Archive:Xbmc-git-svn

This page has notes on what is done for the migration from svn to git.

XBMC Main Repo

This is the script used to sync the xbmc git repository from the svn repository.

#!/bin/sh                       

set -e

# The xbmc svn repo to pull from.
XBMC_SVN_REPO="https://xbmc.svn.sourceforge.net/svnroot/xbmc"

# local git shared repo for xbmc we clone to.
LOCAL_GIT_REPO="$HOME/git-repos/xbmc"        

# The xbmc sourceforge git repo to push to.
XBMC_GIT_SF_REPO="ssh://[email protected]/gitroot/xbmc/xbmc"                                                                              

# The xbmc github repo to push to.
XBMC_GITHUB_REPO="[email protected]:ceros/xbmc.git"

# The xbmc gitorious repo to push to.
XBMC_GITORIOUS_REPO="[email protected]:xbmc/mainline.git"

USAGE="\n\
This script is used to setup a git shared repo to clone an XBMC svn repo.\n\
Once setup, this script can also be used to continually fetch new revisions\n\
and can be used to push to another shared git repo.                           
Usage: xbmc-git-svn [OPTION]\n\                                               
\n\                                                                           
 -h, --help                 Display this help message.\n\                     
 --initial-setup            Only setup the git repo and exit.\n\              
 --perform-push             Performs a git push after fetching new svn revs.\n\
 --local-git-repo GIT_REPO  Override the default local git repo used, which\n\ 
                            is $LOCAL_GIT_REPO\n"                              

while [ "$#" -gt "0" ]
do                    
    case "$1" in      
        --initial-setup)
            INITIAL_SETUP=1
            shift          
            ;;             
        --perform-push)    
            PERFORM_PUSH=1 
            shift          
            ;;             
        --local-git-repo)  
            LOCAL_GIT_REPO="$2"
            shift; shift       
            ;;                 
        -h|--help|*)           
            printf "${USAGE}"  
            exit 1             
            ;;                 
    esac                       
done                           

# This sets up the local git repo to pull from svn and push into the git repo.
initialize_repo () {                                                          
    # Setup the repo with standard layout and prefix of svn/                  
    git svn init "$XBMC_SVN_REPO" --stdlayout --prefix=svn/                   

    # Make tags go into refs/remotes/svn-tags instead of refs/remotes/svn
    git config svn-remote.svn.tags tags/*:refs/remotes/svn-tags/*        

    # Don't follow parents when cloning the svn. This greatly decreases the
    # time needed to clone the svn repo. We'll graft any development history
    # as needed later.
    git config svn-remote.svn.followparent false

    # Settings we apply to keep memory usage sane on the xbmc.org servers
    git config core.packedGitWindowSize 32m                              
    git config core.packedGitLimit 64m                                   
    git config pack.threads 2                                            
    git config pack.windowMemory 64m                                     

    # The remote repositories to push to
    git remote add xbmc-git-sf "$XBMC_GIT_SF_REPO"
    git remote add xbmc-github "$XBMC_GITHUB_REPO"
    git remote add xbmc-gitorious "$XBMC_GITORIOUS_REPO"

    # Immediately exit after initialization
    exit 0
}

# Do all git commands in git repo
test -d "$LOCAL_GIT_REPO" || mkdir -p "$LOCAL_GIT_REPO"
cd "$LOCAL_GIT_REPO"

if [ "$INITIAL_SETUP" ]; then
    initialize_repo
fi

# This fetches new changes from svn and always reset the master branch to
# linuxport
git svn fetch

# This pushes to the remote git repos. master and all svn branches end up in
# refs/heads. All svn tags end up in refs/tags. This will make it possible to
# simply do 'git clone $URL xbmc' to grab all branches and tags
if [ "$PERFORM_PUSH" ]; then
    git push xbmc-git-sf master
    git push xbmc-git-sf refs/remotes/svn/*:refs/heads/*
    git push xbmc-git-sf refs/remotes/svn-tags/*:refs/tags/*
#     git push xbmc-github master
#     git push xbmc-github refs/remotes/svn/*:refs/heads/*
#     git push xbmc-github refs/remotes/svn-tags/*:refs/tags/*
#     git push xbmc-gitorious master
#     git push xbmc-gitorious refs/remotes/svn/*:refs/heads/*
#     git push xbmc-gitorious refs/remotes/svn-tags/*:refs/tags/*
fi

Graft points

Here is a list of graft points we had to use to fix the development history in our git repo.

linuxport switched to trunk at rev 23097. Have child rev (23097) follow parent rev (23094) which was last change made to linuxport before the move.

b35077e53e80b7f707b2b3c4f72a19f4fd55fa57 cc7bfb716f6c2986b1074dd99bfe45462c1f8c75

Here is the graft points for the xbox branch. The xbox branch was created from trunk before the move (r23095). It's parent should be the last change from trunk before the move (r23073).

5692c65551cc915c2a1d4400f06e0e4831bc553d fd899ba70e579fa9049f8b6b612970a2f12fecf1

This is the graft point for when the linuxport branch was created from trunk. The revisions are r8638 (child) and r8637 (parent).

9364c718f19787944aa07f5bb3b6eaf885762ac6 3473363e3ad856c5ad48536d947d80aba6fc1f82

This is the graft point for branch prettylights from linuxport (r22888 from r22886).

abb7ef866645079f01142f60dbc491c93fd893e7 40181a51c8b53b038036c1bbfbc325d263d4f7bf

This is the graft point for branch xdmx from linuxport (r21840 from r21839).

1f8744a36fd28e08128f6682c1581bed3c87de5d 15a2b8d2aeff9f668bb724e1178649617b026eab

This is the graft point for branch unified-pvr from linuxport (r16244 from r16241).

b296e1e14284662dc5758545d1607587b6a23cff f8740632c2992bcfa16a16be7386dec4ea55e04c

This is the graft point for branch gamelibrary2 from linuxport (r22707 from r22704).

4af107c5612a529f46e915b879f6c32ccd64fcf9 27546838dc1a10a268018ce25513c29e6d16f47b

This is the graft point for branch directx from linuxport (r21185 from r21183).

c1089bcaff608fdc1b252be2bcb78ae8e0ad7f50 edf1f8d96f8e6a317c70fca5096eff6ee1f29bb6

This is the graft point for branch masteraudio from linuxport (r20354 from r20352).

e5490d74743d5b8f38e0b32532a44fa8bbd27d37 13e12a4384c05583b8befdbb3d26a4b1c0bcd804

This is the graft point for branch xbmc_on_arm from linuxport (r20267 from r20266).

738e10f2fc728a7f75eca5eb90d938be508221f7 173d289115b36bb1cdf648e6378843c1cacfb938

This is the graft point for branch gamelibrary from linuxport (r16997 from r16996).

d0810422564dfa6d229865d20a65ea55d763f8f5 ad1c0a25f4e2045750dd026a544b9d7fe72104d1

This is the graft point for branch 9.04_Babylon-xbox from xbox (r19588 from r19585).

e71d56a2c0c407131b6cce84fd3787c7a8530e17 09386e4852f03726f0d880d1596aca623e2e3b7f

This is the graft point for branch 9.04_Babylon-linux-osx-win32 from linuxport (r19571 from r19570).

98fec40172bb4a322717ca07f8763532f979ffd7 9c8b5bc934fb3dda3838edea96a4e4e694d1d02f

This is the graft point for branch NetworkManager from linuxport (r20601 from r20596).

b21325f3707443c83352dc4dd5fdc87074c90e30 d839201b42e0b0012b4cc60463b7f222ad8a2cc8

This is the graft point for branch karaoke from linuxport (r16818 from r16817).

90427bda07260181d54eef3a8c9d0384ca01fc05 bd4325eaf159629f660b3de6e0088246ae7c9fc7

Vendor Branches

All vendor sources are being recloned from their respective upstream VCS repositories. We're keeping each vendor source as it's own repository on sourceforge. The "current" branch in the repos is the branch we pin ourselves to make modifications. They are the specific tags or revisions we modify from the respective upstream for xbmc. The "master" branch contains those modifications made from the "current" branch.

FFmpeg and libswscale

Setting up a repo for FFmpeg with libswscale included was somewhat tricky due to libswscale residing in a different location than FFmpeg. We made use of git-submodule to make this work.

libswscale

First we start off by setting up libswscale. Cloning and pushing libswscale is relatively easy. There's no branches or tags to worry about.

$ git svn clone svn://svn.ffmpeg.org/mplayer/trunk/libswscale
$ cd libswscale

We pin ourselves to a particular revision, namely r29464. XBMC svn repo has it as we pinned ourselves to r29476, but the last change made to libswscale was actually r29464.

$ git reset --hard 1e352e2131331e2f5207384702b3f2b57738dcff

Afterwards we create our "current" branch, then push the "master" and "current" branches to reside in refs/heads on our remote branch in sourceforge. The branch used to track libswscale svn, "git-svn", is also pushed to refs/heads.

$ git branch current
$ git remote add sf ssh://[email protected]/gitroot/xbmc/libswscale
$ git push sf master
$ git push sf current
$ git push sf refs/remotes/git-svn:refs/heads/git-svn

FFmpeg

Next we setup FFmpeg. FFmpeg has branches and the trunk branch in svn, but no tags. Here is what's done for our vendor repository of ffmpeg

$ git svn clone  -Ttrunk -bbranches --prefix=svn/ svn://svn.ffmpeg.org/ffmpeg

Next is pinning down what revision we're working with. Here we pin ourselves to r19851

$ cd ffmpeg
$ git reset --hard 237f339e69bac75bfce0ba686d8c1aec40025b65

Now we create the current branch, push master and current to sf, then push the branches used to track ffmpeg svn up to sf.

$ git branch current
$ git remote add sf ssh://[email protected]/gitroot/xbmc/ffmpeg
$ git push sf master
$ git push sf current
$ git push sf refs/remotes/svn/*:refs/heads/svn/*

libswscale as submodule

Now we want to add libswscale into our ffmpeg repo as a submodule. This is simple enough.

$ git submodule add git://xbmc.git.sourceforge.net/gitroot/xbmc/libswscale libswscale

We check that it was written correctly.

$ git status    
# On branch master                                                              
# Changes to be committed:                                                      
#   (use "git reset HEAD <file>..." to unstage)                                 
#                                                                               
#       new file:   .gitmodules                                                 
#       new file:   libswscale                                                  
#                                                                               
$ cat .gitmodules
[submodule "libswscale"]
        path = libswscale
        url = git://xbmc.git.sourceforge.net/gitroot/xbmc/libswscale
$ ls libswscale
bfin               options.c  rgb2rgb_template.c  swscale.h           yuv2rgb.c
colorspace-test.c  ppc        sparc               swscale_internal.h
Makefile           rgb2rgb.c  swscale.c           swscale_template.c
mlib               rgb2rgb.h  swscale-example.c   x86

Then we commit via our ffmpeg repo (the superproject) and push.

$ git commit
$ git push sf master

Finally, we want the submodule added for the "current" branch as well. We checkout current and ignore the warning saying that libswscale couldn't be removed (it shouldn't be). Then we simply merge with master and push current to sf.

$ git checkout current
$ git merge master
$ git push sf current

Now for anyone wanting to checkout the vendor repo for FFmpeg with libswscale as a submodule, it's a matter of using these four commands.

$ git clone git://xbmc.git.sourceforge.net/gitroot/xbmc/ffmpeg ffmpeg
$ cd ffmpeg
$ git submodule init
$ git submodule update

Python

We're currently fetching from svn for now. We'll switch to using the hg-git plugin or hg-fast-export once the Python migration to mercurial is complete.

Python has a standard layout for their svn repo (branches, tags, and trunk). We want to change where tags gets placed, so here we start by creating a directory for python, then doing a 'git svn init'.

$ mkdir python
$ cd python
$ git svn init -s --prefix=svn/ http://svn.python.org/projects/python

Next we modify where the tags will go.

$ git config svn-remote.svn.tags python/tags/*:refs/remotes/svn-tags/*

Now we start fetching.

$ git svn fetch

We pin ourselves to release 2.4.6 and create a "current" branch.

$ git reset --hard svn-tags/r246
$ git branch current

Now we push to sourceforge.

$ git remote add sf ssh://[email protected]/gitroot/xbmc/python
$ git push sf master
$ git push sf current
$ git push sf refs/remotes/svn/*:refs/heads/svn/*
$ git push sf refs/remotes/svn-tags/*:refs/tags/svn-tags/*

Fetch after git clone

If you cloned the python repo, either directly or via git submodule within the xbmc git repo (once it's up), you may setup the python repo to fetch from python. Here we do the initial setup, then change the locations for the tags.

$ git svn init -s --prefix=origin/svn/ http://svn.python.org/projects/python
$ git config svn-remote.svn.tags python/tags/*:refs/tags/svn-tags/*

Then you push in a similar manner as if you had cloned the python svn repo directly. Just supply the correct refspec for the branches and tags.

$ git push master
$ git push current
$ git push origin refs/remotes/origin/svn/*:refs/heads/svn/*
$ git push origin refs/tags/svn-tags/*:refs/tags/svn-tags/*

libdvdcss

Fetching libdvdcss is rather simple.

$ mkdir libdvdcss
$ cd libdvdcss
$ git svn init -s --prefix=svn/ svn://svn.videolan.org/libdvdcss

We always change the default location where tags get placed.

$ git config svn-remote.svn.tags tags/*:refs/remotes/svn-tags/*

Now we fetch.

$ git svn fetch

Now we pin ourselves to the 1.2.10 tag and create the "current" branch.

$ git reset --hard svn-tags/1.2.10
$ git branch current

Finally we push to sourceforge.

$ git remote add sf ssh://[email protected]/gitroot/xbmc/libdvdcss
$ git push sf master
$ git push sf current
$ git push sf refs/remotes/svn/*:refs/heads/svn/*
$ git push sf refs/remotes/svn-tags/*:refs/tags/svn-tags/*

libdvdread

There's no branches or tags for libdvdread, so it's just a simple clone.

$ git svn clone svn://svn.mplayerhq.hu/dvdnav/trunk/libdvdread libdvdread

We reset to a particular commit made back in Feb. 15, 2009 (revision 1168). Then we create the "current" branch.

$ cd libdvdread
$ git reset --hard 3a2808d6556a7fa3123bdc1b2ea767fe3a6002a8
$ git branch current

Now we push everything to sourceforge.

$ git remote add sf ssh://[email protected]/gitroot/xbmc/libdvdread
$ git push sf master
$ git push sf current
$ git push sf refs/remotes/git-svn:refs/heads/git-svn

libdvdnav

There's no branches or tags for libdvdnav either, so it's just a simple clone.

$ git svn clone svn://svn.mplayerhq.hu/dvdnav/trunk/libdvdnav libdvdnav

We reset to a particular commit made back in Feb. 2, 2009 (revision 1167). Then we create the "current" branch.

$ cd libdvdread
$ git reset --hard 2674174be71aea34a6a52c02d0f494245255fd60
$ git branch current

Now we push everything to sourceforge.

$ git remote add sf ssh://[email protected]/gitroot/xbmc/libdvdnav
$ git push sf master
$ git push sf current
$ git push sf refs/remotes/git-svn:refs/heads/git-svn

libcdio

Development for libcdio is done with a git repository so this one is really simple.

$ git clone git://git.sv.gnu.org/libcdio.git libcdio

We pin ourselves to release 0.80 and create the "current" branch.

$ cd libcdio
$ git reset --hard release_0_80
$ git branch current

Now we push to sourceforge.

$ git remote add sf ssh://[email protected]/gitroot/xbmc/libcdio
$ git push sf master
$ git push sf current
$ git push sf refs/tags/release_0_80:refs/tags/upstream/release_0_80

Here we didn't push the branches or all tags from the libcdio upstream git since they're using a git repository anyway. Instead, anyone wanting to track changes from upstream can simply do the following in the libcdio repository cloned from xbmc.

$ git remote add upstream git://git.sv.gnu.org/libcdio.git
$ git fetch upstream

ogg

ogg development is done in an svn repository with a non standard layout. There's no branches to track.

$ mkdir ogg
$ cd ogg
$ git svn init -Ttrunk/ogg -ttags/ogg --prefix=svn/ http://svn.xiph.org

We modify where the trunk and tags get stored in the git repo.

$ git config svn-remote.svn.fetch trunk/ogg:refs/remotes/svn/ogg
$ git config svn-remote.svn.tags tags/ogg/*:refs/remotes/svn-tags/*

Now we fetch

$ git svn fetch

We track the 1.1.3 release of ogg and create the "current" branch.

$ git reset --hard svn-tags/libogg-1.1.3
$ git branch current

Now we push to sourceforge.

$ git remote add sf ssh://[email protected]/gitroot/xbmc/ogg
$ git push sf master
$ git push sf current
$ git push sf refs/remotes/svn/*:refs/heads/svn/*
$ git push sf refs/remotes/svn-tags/*:refs/tags/svn-tags/*

vorbis

vorbis development is done in an svn repository with a non standard layout. There's no branches to track. This is pretty much the same way to clone as ogg.

$ mkdir vorbis
$ cd vorbis
$ git svn init -Ttrunk/vorbis -ttags/vorbis --prefix=svn/ http://svn.xiph.org

We modify where the trunk and tags get stored in the git repo.

$ git config svn-remote.svn.fetch trunk/vorbis:refs/remotes/svn/vorbis
$ git config svn-remote.svn.tags tags/vorbis/*:refs/remotes/svn-tags/*

Now we fetch

$ git svn fetch

More instructions to follow.