Archive:Xbmc-git-svn

This page has notes on what is done for the migration from svn to git.

XBMC Main Repo

This is the script used to sync the xbmc git repository from the svn repository.

#!/bin/sh                                                                   

set -e

# The xbmc svn repo to pull from.
XBMC_SVN_REPO="https://xbmc.svn.sourceforge.net/svnroot/xbmc"

# local git shared repo for xbmc we clone to.
LOCAL_GIT_REPO="$HOME/git-repos/xbmc"        

# The xbmc sourceforge git repo to push to.
XBMC_GIT_SF_REPO="ssh://[email protected]/gitroot/xbmc/xbmc"

# The xbmc github repo to push to.
XBMC_GITHUB_REPO="[email protected]:ceros/xbmc.git"

# The xbmc gitorious repo to push to.
XBMC_GITORIOUS_REPO="[email protected]:xbmc/xbmc.git"

USAGE="\n\
This script is used to setup a git shared repo to clone an XBMC svn repo.\n\
Once setup, this script can also be used to continually fetch new revisions\n\
and can be used to push to another shared git repo.                           
Usage: xbmc-git-svn [OPTION]\n\                                               
\n\                                                                           
 -h, --help                 Display this help message.\n\                     
 --initial-setup            Only setup the git repo and exit.\n\              
 --initial-checkout         Perform the initial checkout.\n\                  
 --perform-push             Performs a git push after fetching new svn revs.\n\
 --local-git-repo GIT_REPO  Override the default local git repo used, which\n\ 
                            is $LOCAL_GIT_REPO\n"                              

while [ "$#" -gt "0" ]
do                    
    case "$1" in      
        --initial-setup)
            INITIAL_SETUP=1
            shift          
            ;;             
        --initial-checkout)    
            INITIAL_CHECKOUT=1 
            shift              
            ;;                 
        --perform-push)        
            PERFORM_PUSH=1     
            shift              
            ;;                 
        --local-git-repo)      
            LOCAL_GIT_REPO="$2"
            shift; shift       
            ;;                 
        -h|--help|*)           
            printf "${USAGE}"  
            exit 1             
            ;;                 
    esac                       
done                           

# This sets up the local git repo to pull from svn and push into the git repo.
initialize_repo () {                                                          
    # Setup the repo with standard layout and prefix of svn/                  
    git svn init --stdlayout --prefix=origin/ "$XBMC_SVN_REPO"                

    # Make tags go into refs/remotes/svn-tags instead of refs/remotes/svn
    git config svn-remote.svn.tags tags/*:refs/tags/*                    

    # Settings we apply to keep memory usage sane only on the xbmc.org
    # servers                                                         
    git config core.packedGitWindowSize 32m                           
    git config core.packedGitLimit 64m                                
    git config pack.threads 2                                         
    git config pack.windowMemory 64m

    # The remote repositories to push to. Sourceforge git repo will be labeled
    # as 'origin'.
    git remote add origin "$XBMC_GIT_SF_REPO"
    git remote add xbmc-github "$XBMC_GITHUB_REPO"
    git remote add xbmc-gitorious "$XBMC_GITORIOUS_REPO"

    # Immediately exit after initialization
    exit 0
}

# Do all git commands in git repo
cd "$LOCAL_GIT_REPO"

do_initial_checkout () {
    # This should only be run once.
    if git log 545fb262092af486f386c7540455da47245ddc4e >/dev/null 2>&1; then
        echo "Initial checkout has already been done."
        exit 1
    fi
    git svn fetch
    echo "928c8da7f7cd9dc734e7fd7c4b0f6842c0e3fa81 a2c7102f3b29b8a36c2aaf1e2362a94632443707" > .git/info/grafts
    git filter-branch -- --all
    git for-each-ref --format='%(refname)' refs/original | while read ref; do git update-ref -d "$ref"; done
    git gc --expire=now

    # Immediately exit after initial checkout
    exit 0
}

if [ "$INITIAL_SETUP" ]; then
    initialize_repo
elif [ "$INITIAL_CHECKOUT" ]; then
    do_initial_checkout
fi

# This fetches new changes from svn
git svn fetch
git svn rebase

# This pushes to the remote git repos. master and all svn branches end up in
# refs/heads. All svn tags end up in refs/tags. This will make it possible to
# simply do 'git clone $URL xbmc' to grab all branches and tags
if [ "$PERFORM_PUSH" ]; then
    git push origin master refs/remotes/origin/*:refs/heads/* refs/tags/*:refs/tags/*
    git push xbmc-github master refs/remotes/origin/*:refs/heads/* refs/tags/*:refs/tags/*
    git push xbmc-gitorious master refs/remotes/origin/*:refs/heads/* refs/tags/*:refs/tags/*
fi

Vendor Branches

All vendor sources are being recloned from their respective upstream VCS repositories. We're keeping each vendor source as it's own repository on sourceforge. The "current" branch in the repos is the branch we pin ourselves to make modifications. They are the specific tags or revisions we modify from the respective upstream for xbmc. The "master" branch contains those modifications made from the "current" branch.

faad2

cvs from sourceforge

FFmpeg and libswscale

Setting up a repo for FFmpeg with libswscale included was somewhat tricky due to libswscale residing in a different location than FFmpeg. We made use of git-submodule to make this work.

libswscale

First we start off by setting up libswscale. Cloning and pushing libswscale is relatively easy. There's no branches or tags to worry about.

$ git svn clone svn://svn.ffmpeg.org/mplayer/trunk/libswscale
$ cd libswscale

We pin ourselves to a particular revision, namely r29464. XBMC svn repo has it as we pinned ourselves to r29476, but the last change made to libswscale was actually r29464.

$ git reset --hard 1e352e2131331e2f5207384702b3f2b57738dcff

Afterwards we create our "current" branch, then push the "master" and "current" branches to reside in refs/heads on our remote branch in sourceforge. The branch used to track libswscale svn, "git-svn", is also pushed to refs/heads.

$ git branch current
$ git remote add sf ssh://[email protected]/gitroot/xbmc/libswscale
$ git push sf master
$ git push sf current
$ git push sf refs/remotes/git-svn:refs/heads/git-svn

FFmpeg

Next we setup FFmpeg. FFmpeg has branches and the trunk branch in svn, but no tags. Here is what's done for our vendor repository of ffmpeg

$ git svn clone  -Ttrunk -bbranches --prefix=svn/ svn://svn.ffmpeg.org/ffmpeg

Next is pinning down what revision we're working with. Here we pin ourselves to r19851

$ cd ffmpeg
$ git reset --hard 237f339e69bac75bfce0ba686d8c1aec40025b65

Now we create the current branch, push master and current to sf, then push the branches used to track ffmpeg svn up to sf.

$ git branch current
$ git remote add sf ssh://[email protected]/gitroot/xbmc/ffmpeg
$ git push sf master
$ git push sf current
$ git push sf refs/remotes/svn/*:refs/heads/svn/*

libswscale as submodule

Now we want to add libswscale into our ffmpeg repo as a submodule. This is simple enough.

$ git submodule add git://xbmc.git.sourceforge.net/gitroot/xbmc/libswscale libswscale

We check that it was written correctly.

$ git status    
# On branch master                                                              
# Changes to be committed:                                                      
#   (use "git reset HEAD <file>..." to unstage)                                 
#                                                                               
#       new file:   .gitmodules                                                 
#       new file:   libswscale                                                  
#                                                                               
$ cat .gitmodules
[submodule "libswscale"]
        path = libswscale
        url = git://xbmc.git.sourceforge.net/gitroot/xbmc/libswscale
$ ls libswscale
bfin               options.c  rgb2rgb_template.c  swscale.h           yuv2rgb.c
colorspace-test.c  ppc        sparc               swscale_internal.h
Makefile           rgb2rgb.c  swscale.c           swscale_template.c
mlib               rgb2rgb.h  swscale-example.c   x86

Then we commit via our ffmpeg repo (the superproject) and push.

$ git commit
$ git push sf master

Finally, we want the submodule added for the "current" branch as well. We checkout current and ignore the warning saying that libswscale couldn't be removed (it shouldn't be). Then we simply merge with master and push current to sf.

$ git checkout current
$ git merge master
$ git push sf current

Now for anyone wanting to checkout the vendor repo for FFmpeg with libswscale as a submodule, it's a matter of using these four commands.

$ git clone git://xbmc.git.sourceforge.net/gitroot/xbmc/ffmpeg ffmpeg
$ cd ffmpeg
$ git submodule init
$ git submodule update

liba52

svn from videolan

libcdio

Development for libcdio is done with a git repository so this one is really simple.

$ git clone git://git.sv.gnu.org/libcdio.git libcdio

We pin ourselves to release 0.80 and create the "current" branch.

$ cd libcdio
$ git reset --hard release_0_80
$ git branch current

Now we push to sourceforge.

$ git remote add sf ssh://[email protected]/gitroot/xbmc/libcdio
$ git push sf master
$ git push sf current
$ git push sf refs/tags/release_0_80:refs/tags/upstream/release_0_80

Here we didn't push the branches or all tags from the libcdio upstream git since they're using a git repository anyway. Instead, anyone wanting to track changes from upstream can simply do the following in the libcdio repository cloned from xbmc.

$ git remote add upstream git://git.sv.gnu.org/libcdio.git
$ git fetch upstream

libdvdcss

Fetching libdvdcss is rather simple.

$ mkdir libdvdcss
$ cd libdvdcss
$ git svn init -s --prefix=svn/ svn://svn.videolan.org/libdvdcss

We always change the default location where tags get placed.

$ git config svn-remote.svn.tags tags/*:refs/remotes/svn-tags/*

Now we fetch.

$ git svn fetch

Now we pin ourselves to the 1.2.10 tag and create the "current" branch.

$ git reset --hard svn-tags/1.2.10
$ git branch current

Finally we push to sourceforge.

$ git remote add sf ssh://[email protected]/gitroot/xbmc/libdvdcss
$ git push sf master
$ git push sf current
$ git push sf refs/remotes/svn/*:refs/heads/svn/*
$ git push sf refs/remotes/svn-tags/*:refs/tags/svn-tags/*

libdvdnav

There's no branches or tags for libdvdnav either, so it's just a simple clone.

$ git svn clone svn://svn.mplayerhq.hu/dvdnav/trunk/libdvdnav libdvdnav

We reset to a particular commit made back in Feb. 2, 2009 (revision 1167). Then we create the "current" branch.

$ cd libdvdread
$ git reset --hard 2674174be71aea34a6a52c02d0f494245255fd60
$ git branch current

Now we push everything to sourceforge.

$ git remote add sf ssh://[email protected]/gitroot/xbmc/libdvdnav
$ git push sf master
$ git push sf current
$ git push sf refs/remotes/git-svn:refs/heads/git-svn

libdvdread

There's no branches or tags for libdvdread, so it's just a simple clone.

$ git svn clone svn://svn.mplayerhq.hu/dvdnav/trunk/libdvdread libdvdread

We reset to a particular commit made back in Feb. 15, 2009 (revision 1168). Then we create the "current" branch.

$ cd libdvdread
$ git reset --hard 3a2808d6556a7fa3123bdc1b2ea767fe3a6002a8
$ git branch current

Now we push everything to sourceforge.

$ git remote add sf ssh://[email protected]/gitroot/xbmc/libdvdread
$ git push sf master
$ git push sf current
$ git push sf refs/remotes/git-svn:refs/heads/git-svn

libmpeg2

svn from videolan

libprojectm

svn from sourceforge

libsquish

svn from googlecode

ogg

ogg development is done in an svn repository with a non standard layout. There's no branches to track.

$ mkdir ogg
$ cd ogg
$ git svn init -Ttrunk/ogg -ttags/ogg --prefix=svn/ http://svn.xiph.org

We modify where the trunk and tags get stored in the git repo.

$ git config svn-remote.svn.fetch trunk/ogg:refs/remotes/svn/ogg
$ git config svn-remote.svn.tags tags/ogg/*:refs/remotes/svn-tags/*

Now we fetch

$ git svn fetch

We track the 1.1.3 release of ogg and create the "current" branch.

$ git reset --hard svn-tags/libogg-1.1.3
$ git branch current

Now we push to sourceforge.

$ git remote add sf ssh://[email protected]/gitroot/xbmc/ogg
$ git push sf master
$ git push sf current
$ git push sf refs/remotes/svn/*:refs/heads/svn/*
$ git push sf refs/remotes/svn-tags/*:refs/tags/svn-tags/*

Python

We're currently fetching from svn for now. We'll switch to using the hg-git plugin or hg-fast-export once the Python migration to mercurial is complete.

Python has a standard layout for their svn repo (branches, tags, and trunk). We want to change where tags gets placed, so here we start by creating a directory for python, then doing a 'git svn init'.

$ mkdir python
$ cd python
$ git svn init -s --prefix=svn/ http://svn.python.org/projects/python

Next we modify where the tags will go.

$ git config svn-remote.svn.tags python/tags/*:refs/remotes/svn-tags/*

Now we start fetching.

$ git svn fetch

We pin ourselves to release 2.4.6 and create a "current" branch.

$ git reset --hard svn-tags/r246
$ git branch current

Now we push to sourceforge.

$ git remote add sf ssh://[email protected]/gitroot/xbmc/python
$ git push sf master
$ git push sf current
$ git push sf refs/remotes/svn/*:refs/heads/svn/*
$ git push sf refs/remotes/svn-tags/*:refs/tags/svn-tags/*

Fetch after git clone

If you cloned the python repo, either directly or via git submodule within the xbmc git repo (once it's up), you may setup the python repo to fetch from python. Here we do the initial setup, then change the locations for the tags.

$ git svn init -s --prefix=origin/svn/ http://svn.python.org/projects/python
$ git config svn-remote.svn.tags python/tags/*:refs/tags/svn-tags/*

Then you push in a similar manner as if you had cloned the python svn repo directly. Just supply the correct refspec for the branches and tags.

$ git push master
$ git push current
$ git push origin refs/remotes/origin/svn/*:refs/heads/svn/*
$ git push origin refs/tags/svn-tags/*:refs/tags/svn-tags/*

timidity

cvs from sourceforge

vorbis

vorbis development is done in an svn repository with a non standard layout. There's no branches to track. This is pretty much the same way to clone as ogg.

$ mkdir vorbis
$ cd vorbis
$ git svn init -Ttrunk/vorbis -ttags/vorbis --prefix=svn/ http://svn.xiph.org

We modify where the trunk and tags get stored in the git repo.

$ git config svn-remote.svn.fetch trunk/vorbis:refs/remotes/svn/vorbis
$ git config svn-remote.svn.tags tags/vorbis/*:refs/remotes/svn-tags/*

Now we fetch

$ git svn fetch

We pin ourselves to tag 1.2.0 and create the "current" branch.

$ git reset --hard svn-tags/libvorbis-1.2.0
$ git branch current

Now we push to sourceforge.

$ git remote add sf ssh://[email protected]/gitroot/xbmc/vorbis
$ git push sf master
$ git push sf current
$ git push sf refs/remotes/svn/*:refs/heads/svn/*
$ git push sf refs/remotes/svn-tags/*:refs/tags/svn-tags/*

zlib

unknown VCS