How I moved my code repository to Google Code

From WhyNotWiki
Jump to: navigation, search

2007-06-14 15:08

Contents

Reset Google Code repository to revision 0

Make sure you have svnsync

I didn't have it, because I was using svn client ~1.3.2 and svnsync didn't become available until 1.4...

So I upgraded to Subversion 1.4.4, which was quite an ordeal for me...

Create a repository with just the code you want to export

svnsync only syncs entire repositories (that I'm aware); it won't let you simply specify the directories that you want to sync.

This is what happens if you try to...

> svnsync init https://rake-completion.googlecode.com/svn/ http://code.qualitysmith.com/gemables/rake-command-completion
svnsync: Session is rooted at 'http://code.qualitysmith.com/gemables/rake-command-completion' but the repos root is 'http://code.qualitysmith.com'

This can be annoying; for instance, if this is a company repository with many projects stored in it, and you want to release/publish the source code for this single project/directory.

Fortunately, there is a workaround: dump the entire repository; use svndumpfilter to get only the directories you want to publish; load that dump into a new repository (see Subversion / Dump and loading for instructions on how to do this).

Do the svnsync

> svnsync init https://rake-completion.googlecode.com/svn/ file:///var/www/svn/code-copy/
Copied properties for revision 0.

> svn checkout https://rake-completion.googlecode.com/svn/ rake-completion --username ...
Checked out revision 0.

Hmm... That didn't exactly work.... Or did it?

> svnsync sync https://rake-completion.googlecode.com/svn/
Committed revision 1.
Copied properties for revision 1.
...
Committed revision 1410.
Copied properties for revision 1410.
svnsync: CHECKOUT of '/svn/!svn/bln/1410': 502 Bad Gateway (https://rake-completion.googlecode.com)

> svnsync sync https://rake-completion.googlecode.com/svn/
...
Copied properties for revision 1618.
Committed revision 1619.
svnsync: PROPFIND request failed on '/svn'
svnsync: PROPFIND of '/svn': 502 Bad Gateway (https://rake-completion.googlecode.com)

> svnsync sync https://rake-completion.googlecode.com/svn/
...
Committed revision 2754.
Copied properties for revision 2754.
svnsync: OPTIONS request failed on '/svn'
svnsync: OPTIONS of '/svn': 502 Bad Gateway (https://rake-completion.googlecode.com)

> svnsync sync https://rake-completion.googlecode.com/svn/
...
Committed revision 2773.
Copied properties for revision 2773.
svnsync: applying log message to /svn/!svn/wbl/e68844b5-f432-0410-a716-9391fd35c517/2773: 502 Bad Gateway (https://rake-completion.googlecode.com)

> svnsync sync https://rake-completion.googlecode.com/svn/
...
Copied properties for revision 2884.
Committed revision 2885.
svnsync: DAV request failed; it's possible that the repository's pre-revprop-change hook either failed or is non-existent
svnsync: At least one property change failed; repository is unchanged

The syncing was a really slow process! I didn't time it, but I would estimate that it took > 3 hours!

And it got interrupted several times with errors, as noted above, so I had to restart the process (not from the beginning, mind you, but I had to "cause the process to start up again"...).

Worst of all, it got stuck at revision 2885 and wouldn't go any further. I tried restarting the process a bunch of times, but every time I kept getting that last error (DAV request failed). It should have kept going until revision 3036, so I something isn't right.

Actually, coincidentally, 2885 is the last revision to actually have any real content in it -- all the other revisions from 2886 to 3036 are simply padding revisions... But I'm not sure that Subversion knew that! Even if it did, why would it throw this error? It had no qualms about committing all the other useless padding revisions... So why wouldn't it commit useless padding revisions 2886-3036 too? I'm reluctant to believe that it actually looked ahead through all future revisions up to the last revision in the source repository, saw that they were all empty, and that's the reason it had the error.

To test this theory, I committed a non-empty revision 3037 to the source repository and tried starting the process again...

> sudo svn ci -m '' code-copy/gemables/rake_command_completion/Readme
Sending        code-copy/gemables/rake_command_completion/Readme
Transmitting file data .
Committed revision 3037.

> svnsync sync https://rake-completion.googlecode.com/svn/
svnsync: DAV request failed; it's possible that the repository's pre-revprop-change hook either failed or is non-existent
svnsync: At least one property change failed; repository is unchanged

Nope, it still won't budge from 2885.

Well that's interesting. I don't know why it stopped there... But it is a pretty convenient place to have stopped, because it just so happens that it stopped after committing the last useful revision. I guess I'll let svnsync off the hook this time and just be glad it didn't stop at an earlier revision, ... but next time, svnsync, I may not let you off so easy.

I wonder if it will break with this error when I try transfering the next project over... We'll see.

Now to finish up this project...

> svn mv https://rake-completion.googlecode.com/svn/gemables/rake_command_completion https://rake-completion.googlecode.com/svn/trunk -m 'Moved to trunk/'

> svn rm https://rake-completion.googlecode.com/svn/gemables -m 'Removed everything except trunk'


Conclusion: not worth the trouble

It might be worth all this trouble if the changesets that I was copying over were really important or something but, at least in my case, they weren't. Here are 2 reasons why not:

  • Most of the revisions copied over were simply empty ("padding") revisions anyway (more below)
  • Even the revisions that were real revisions, I probably won't ever refer to...

Most of the revisions were padding revisions

Of the [3000] revisions copied over, only 8 of them were actual revisions that contained code for this project...


> svn log ~/code/gemables/rake_command_completion
------------------------------------------------------------------------
r2884 | tyler | 2007-05-02 09:02:06 -0700 (Wed, 02 May 2007) | 1 line

readme cleanup
------------------------------------------------------------------------
r2854 | tyler | 2007-04-30 13:11:44 -0700 (Mon, 30 Apr 2007) | 3 lines
...

------------------------------------------------------------------------
r2475 | tyler | 2007-03-23 13:17:34 -0700 (Fri, 23 Mar 2007) | 2 lines
...

------------------------------------------------------------------------
r2397 | tyler | 2007-03-14 20:42:54 -0700 (Wed, 14 Mar 2007) | 4 lines
...

------------------------------------------------------------------------
r2396 | tyler | 2007-03-14 18:08:05 -0700 (Wed, 14 Mar 2007) | 1 line

Finished packaging and releasing rake_command_completion
------------------------------------------------------------------------
r2391 | tyler | 2007-03-13 18:42:07 -0700 (Tue, 13 Mar 2007) | 5 lines
...

------------------------------------------------------------------------
r2387 | tyler | 2007-03-13 17:01:04 -0700 (Tue, 13 Mar 2007) | 1 line

"Finished" rake_command_completion gem. Got it packagable as a gem.
------------------------------------------------------------------------
r2386 | tyler | 2007-03-13 15:04:27 -0700 (Tue, 13 Mar 2007) | 1 line

Starting rake_command_completion gem
------------------------------------------------------------------------
 

This is the unfortunate result of doing an svndumpfilter to extract just the revisions pertaining to the directory/paths you are interested in... It doesn't actually remove the revisions that aren't of interest; it just replaces them with "padding" revisions that have no content.

Usually, I don't really mind that... But it's kind of annoying when using svnsync because it has to commit each revision one-at-a-time ... which is really slow.

It would be much more efficient to just do an svnadmin load on the target repository ... but that requires shell access and unfortunately neither Google Code nor RubyForge allow that...

Next time: Try svndumpfilter --drop-empty-revs --renumber-revs ...

An anonymous contributer gave this tip: "to avoid all those empty revisions, you probably want to look into the --drop-empty-revs and --renumber-revs options to svndumpfilter. I just did this, and they worked out great: my new repository has just the 120 or so useful revisions for my project, and not the 3000-odd in the original repository."

Ads
Personal tools