From WhyNotWiki

Vendor branch management  edit   (Category  edit)

Synchronization with Subversion  edit   (Category  edit)

How I started out: I started out with MediaWiki 1.6.5, then modified a few source files.

What I changed to: I put my entire MediaWiki installation under version control, imported vender drops for MediaWiki 1.6.5 and 1.9.0, and used them to upgrade my installation while keeping all of my previous customizations!

This wouldn't have been possible (at least not easy) without using a version control system like Subversion. If I had simply copied the 1.9.0 files over top of my customized 1.6.5 files, I would've lost all my customizations. Subversion has this great feature called svn merge.

This is how I wanted the revisions to look like:

1. A fresh "install" of MediaWiki 1.6.5
2. My customizations to 1.6.5, including:
   • LocalSettings.php
   • Removing all the language files so that I don't need
   • Parser.php
   • EditPage.php
3. MediaWiki 1.9.0

Following http://svnbook.red-bean.com/en/1.0/ch07s04.html closely, here's what I did...

[edit] Initial vendor drop (MediaWiki 1.6.5)

[tyler: ~/dev/wiki]> svn import ~/mediawiki-1.6.5/ \
                            http://svn.tylerrick.com/wiki/vendor/current \
                            -m 'Importing initial 1.6.5 vendor drop'
Committed revision 2.

[tyler: ~/dev/wiki]> svn cp http://svn.tylerrick.com/wiki/vendor/current \
                            http://svn.tylerrick.com/wiki/vendor/1.6.5 \
                            -m 'Tagging mediawiki-1.6.5'
Committed revision 3.

[tyler: ~/dev/wiki]> svn cp http://svn.tylerrick.com/wiki/vendor/1.6.5 \
                            http://svn.tylerrick.com/wiki/trunk \
                            -m 'Starting the trunk with a fresh install of MediaWiki 1.6.5'
Committed revision 3.

[edit] Merge my customizations to 1.6.5

I couldn't find svn_load_dirs.pl on Dreamhost, so I downloaded it:

$ wget http://svn.collab.net/repos/svn/tags/1.3.2/contrib/client-side/svn_load_dirs.pl.in -O ~/bin/svn_load_dirs.pl
$ chmod +x ~/bin/svn_load_dirs.pl

I had to modify ~/bin/svn_load_dirs.pl:26 from

to:

Then I ran this command, which did most of the work for me (it also took quite a while...like 10 minutes):

$ svn_load_dirs.pl http://svn.tylerrick.com/wiki/ trunk \
                   ~/tylerrick.com/wiki/mychanges

Checking that the base URL is a Subversion repository.
Running /usr/bin/svn log -r HEAD http://svn.tylerrick.com/wiki

Finding the root URL of the Subversion repository.
Running /usr/bin/svn log -r HEAD http://svn.tylerrick.com
Running /usr/bin/svn log -r HEAD http://svn.tylerrick.com/wiki
Determined that the svn root URL is http://svn.tylerrick.com/wiki.

Native EOL on this system is \012.

Finding if any directories need to be created in repository.
Running /usr/bin/svn log -r HEAD http://svn.tylerrick.com/wiki/trunk
No directories need to be created to prepare repository.
Checking out http://svn.tylerrick.com/wiki/trunk into /tmp/svn_load_dirs_zfdcfNONcX/my_import_wc
Running /usr/bin/svn checkout http://svn.tylerrick.com/wiki/trunk my_import_wc

Loading /home/tylerrick/tylerrick.com/wiki/mychanges.

The following table lists files and directories that
exist in either the Subversion repository or the
directory to be imported but not both.  You now have
the opportunity to match them up as renames instead
of deletes and adds.  This is a Good Thing as it'll
make the repository take less space.

The left column lists files and directories that
exist in the Subversion repository and do not exist
in the directory being imported.  The right column
lists files and directories that exist in the
directory being imported.  Match up a deleted item
from the left column with an added item from the
right column.  Note the line numbers on the left
which you type into this script to have a rename
performed.

     Deleted                           Added
   0 languages/LanguageAb.deps.php____ .htaccess
   1 languages/LanguageAb.php_________ languagesBackup/.htaccess
...
Continue printing (Y/n)? n
Enter two indexes for each column to rename, (R)elist, or (F)inish: f
A   LocalSettings.php.20060529
A   LocalSettings.php
A   .htaccess
U   skins/monobook/main.css
U   skins/common/common.css
...
U   languages/Messages.php
U   includes/Title.php
U   includes/Parser.php
U   includes/EditPage.php
...
D   languages/MessagesUk.php
D   languages/MessagesVi.php
D   languages/MessagesWa.php
D   images/README
Running /usr/bin/svn add -N --targets /tmp/svn_load_dirs_zfdcfNONcX/targets.00001
Running /usr/bin/svn propset svn:executable --file /tmp/svn_load_dirs_zfdcfNONcX/rDlnxHMkcl maintenance/storage/make-blobs
Running /usr/bin/svn rm --targets /tmp/svn_load_dirs_zfdcfNONcX/targets.00002
Running /usr/bin/svn propget svn:eol-style includes/Title.php
Running /usr/bin/svn propget svn:eol-style includes/Parser.php
Running /usr/bin/svn propget svn:eol-style includes/EditPage.php
Running /usr/bin/svn propget svn:eol-style languages/Messages.php
Running /usr/bin/svn propget svn:eol-style skins/common/common.css
Running /usr/bin/svn propget svn:eol-style skins/monobook/main.css
Running /usr/bin/svn commit -m Load /home/tylerrick/tylerrick.com/wiki/mychanges into trunk.

Running /usr/bin/svn update
Cleaning up /tmp/svn_load_dirs_zfdcfNONcX
        (in cleanup) Can't return to /usr/local/tmp/svn_load_dirs_zfdcfNONcX/my_import_wc from /tmp/svn_load_dirs_zfdcfNONcX (No such file or directory) at /home/tylerrick/bin/svn_load_dirs.pl line 1979

[edit] Grab version 1.9.0 vendor drop

The way I didn't do it:

    [tyler: ~/dev/wiki]> cp -R mediawiki-1.9.0/. vendor/current/
    [tyler: ~/dev/wiki]> cd vendor/current/

and

    [tyler: ~/dev/wiki/vendor/current]> svn st
    ?      opensearch_desc.php
    ?      StartProfiler.php
    M      languages/Language.php
    M      languages/LanguageConverter.php
    M      languages/Names.php
    ... like a 100 other files

I didn't do it that way because then I'd have to manually svn add and svn rm files...and that's not very fun.

The way I did do it:

$ svn_load_dirs.pl -t '1.9.0' http://svn.tylerrick.com/wiki/vendor current \
                              ~/mediawiki-1.9.0

     Deleted                              Added
...
  22 languages/LanguageAv.php____________ includes/api/ApiQuery.php
  23 languages/LanguageAy.deps.php_______ includes/api/ApiQueryAllpages.php
  24 languages/LanguageAy.php____________ includes/api/ApiQueryBacklinks.php
  25 languages/LanguageAz.php____________ includes/api/ApiQueryBase.php
...
  97 languages/LanguageNds_nl.php________ languages/classes/LanguageAz.php
  98 languages/LanguageNl.php____________ languages/classes/LanguageBe.php
  99 languages/LanguageNn.php____________ languages/classes/LanguageBg.php
 100 languages/LanguageNo.php____________ languages/classes/LanguageBs.php
 101 languages/LanguageNon.deps.php______ languages/classes/LanguageCs.php
 102 languages/LanguageNon.php___________ languages/classes/LanguageCu.php
 103 languages/LanguageNv.php____________ languages/classes/LanguageEo.php
 104 languages/LanguageOc.php____________ languages/classes/LanguageEt.php
 105 languages/LanguageOr.php____________ languages/classes/LanguageFi.php
 106 languages/LanguageOs.deps.php_______ languages/classes/LanguageFr.php
 107 languages/LanguageOs.php____________ languages/classes/LanguageGa.php
 108 languages/LanguagePa.php____________ languages/classes/LanguageGsw.php
 109 languages/LanguagePl.php____________ languages/classes/LanguageHe.php
Continue printing (Y/n)? n

At this point I could have spent an hour telling it that Deleted file #25 (languages/LanguageAz.php) corresponds to Added file #97 (languages/messages/MessagesAz.php), etc., etc., so that it would do an svn mv instead of an svn rm followed by svn add...but that's a waste of time in this case so I didn't bother. If you actually wanted to retain as much version/ancestry history as possible, that's what you'd want to do though.

Enter two indexes for each column to rename, (R)elist, or (F)inish: f
U   trackback.php
U   thumb.php
U   redirect.php
U   profileinfo.php
A   opensearch_desc.php
U   install-utils.inc
U   index.php
U   img_auth.php
A   api.php
U   UPGRADE
A   StartProfiler.php
U   RELEASE-NOTES
U   README
A   Makefile
U   INSTALL
U   HISTORY
U   FAQ
U   AdminSettings.sample
U   tests/SearchMySQL4Test.php
U   tests/SearchEngineTest.php
U   tests/RunTests.php
U   tests/LocalTestSettings.sample
U   tests/DatabaseTest.php
...
D   languages/LanguageAb.php
D   languages/LanguageAf.php
D   languages/LanguageAr.php
D   languages/LanguageAs.php
D   languages/LanguageAv.php
D   languages/LanguageAy.php
D   languages/LanguageAz.php
D   languages/LanguageBa.php
...
D   skins/MonoBook.tpl
Running /usr/bin/svn add -N --targets /tmp/svn_load_dirs_M8KZpp5CYr/targets.00001
Running /usr/bin/svn propset svn:executable --file /tmp/svn_load_dirs_M8KZpp5CYr/tToxlY2nSF bin/ulimit-tvf.sh
Running /usr/bin/svn propset svn:executable --file /tmp/svn_load_dirs_M8KZpp5CYr/Wqs2fjiROp maintenance/storage/make-blobs
Running /usr/bin/svn rm --targets /tmp/svn_load_dirs_M8KZpp5CYr/targets.00002
Running /usr/bin/svn propget svn:eol-style FAQ

Running /usr/bin/svn update
Running /usr/bin/svn cp -m Tag vendor/current as vendor/1.9.0.
 http://svn.tylerrick.com/wiki/vendor/current http://svn.tylerrick.com/wiki/vendor/1.9.0
Checking out http://svn.tylerrick.com/wiki/vendor/1.9.0 into /tmp/svn_load_dirs_M8KZpp5CYr/my_tag_wc_named_1.9.0
Running /usr/bin/svn checkout http://svn.tylerrick.com/wiki/vendor/1.9.0 my_tag_wc_named_1.9.0
Running diff -u -x .svn -r /home/tylerrick/mediawiki-1.9.0 /tmp/svn_load_dirs_M8KZpp5CYr/my_tag_wc_named_1.9.0
Cleaning up /tmp/svn_load_dirs_M8KZpp5CYr

[edit] Integrate version 1.9.0 vendor drop into trunk

This is the coolest part of the process, I'd have to say! It's where we actually merge my changes with the vendor's most recent changes. There's a chance that this may result in conflicts, but we'll see...

[tyler: ~/dev/wiki/trunk]> svn merge http://svn.tylerrick.com/wiki/vendor/1.6.5 \
                                     http://svn.tylerrick.com/wiki/vendor/current \
                                     .
# (Recall that current is now at 1.9.0)
...
U    maintenance/interwiki.sql
U    maintenance/storage/compressOld.inc
A    maintenance/storage/checkStorage.php
 G   maintenance/storage/make-blobs
U    maintenance/archives/patch-log_params.sql
U    maintenance/archives/patch-restructure.sql
A    maintenance/archives/patch-user_editcount.sql
...
A    skins/monobook/document.png
C    skins/monobook/main.css
U    skins/CologneBlue.php
U    skins/Standard.php
C    skins/common/common.css
A    skins/common/ajaxsearch.js
U    skins/common/cologneblue.css
A    skins/common/ajaxwatch.js
...

Skipped missing target: 'languages/LanguageSv.php'
Skipped missing target: 'languages/LanguageDe.php'
Skipped missing target: 'languages/LanguageLa.php'
Skipped missing target: 'languages/MessagesEt.php'
Skipped missing target: 'languages/MessagesIs.php'
Skipped missing target: 'languages/LanguageTa.php'
Skipped missing target: 'languages/LanguageLi.php'
...
A    languages/messages
A    languages/messages/MessagesIi.php
A    languages/messages/MessagesEn.php
A    languages/messages/MessagesAs.php
A    languages/messages/MessagesMl.php
A    languages/messages/MessagesEt.php
A    languages/messages/MessagesAy.php
A    languages/messages/MessagesIs.php
...

I'm somewhat disappointed that it added back the 138 language files that I deleted back in version 1.6.5... But I'm not surprised that it did, since those files were moved between 1.6.5 and 1.8.2 and I didn't tell it about the move.

I'm very curious if it would have skipped adding languages/messages/* (the desired behavior) if I had matched up the old file paths with the new paths. But it's too late to test that now...

[edit] Resolving conflicts

Next I check for conflicts.

$ svn st | grep "^C"
C      skins/monobook/main.css
C      skins/common/common.css

Oh yeah, there were totally conflicts! Let's go and fix 'em...

$ svn st | grep "^C" | sed 's/^C//' | vimopen

[edit] Committing

[tyler: ~/dev/wiki/trunk]> svn ci -m 'Merging MediaWiki-1.9.0 into the trunk'
...
Committed revision 9.

[edit] Delete the silly language files

... because they contain strange characters that break my terminal if I output them (like when I do multi-file searches)...

[tyler: ~/dev/wiki/trunk]> svn rm languages/classes/Language*
[tyler: ~/dev/wiki/trunk]> svn rm languages/messages/*
[tyler: ~/dev/wiki/trunk]> svn revert languages/messages/MessagesEn.php
[tyler: ~/dev/wiki/trunk]> svn revert languages/messages/MessagesDe.php
[tyler: ~/dev/wiki/trunk]> svn revert languages/messages/MessagesEnRTL.php
[tyler: ~/dev/wiki/trunk]> svn ci -m "Delete language files that I don't need"

[edit] Finish MediaWiki-1.9.0 upgrade: Database update

Create an AdminSettings.php file.

Don't use the wrong version of php.

[tylerrick: ~/tylerrick.com/wiki/trunk/maintenance]$ php update.php
X-Powered-By: PHP/4.4.4
Content-type: text/html

Sorry! This version of MediaWiki requires PHP 5; you are running 4.4.4.

If you are sure you already have PHP 5 installed, it may be installed
in a different path from PHP 4. Check with your system administrator.

[tylerrick: ~/tylerrick.com/wiki/trunk/maintenance]$ /usr/local/php5/bin/php update.php
X-Powered-By: PHP/5.1.2
Content-type: text/html

MediaWiki 1.9.0 Updater

Going to run database updates for tyler_wiki
Depending on the size of your database this may take a while!
Abort with control-c in the next five seconds...0
...hitcounter table already exists.
...querycache table already exists.
...objectcache table already exists.
...categorylinks table already exists.
...logging table already exists.
...user_newtalk table already exists.
...transcache table already exists.
...trackbacks table already exists.
...externallinks table already exists.
...job table already exists.
Creating langlinks table...ok
Creating querycache_info table...ok
Creating filearchive table...ok
Creating redirect table...ok
Creating querycachetwo table...ok
...have ipb_id field in ipblocks table.
...have ipb_expiry field in ipblocks table.
Adding ipb_enable_autoblock field to table ipblocks...ok
...have rc_type field in recentchanges table.
...have rc_ip field in recentchanges table.
...have rc_id field in recentchanges table.
...have rc_patrolled field in recentchanges table.
Adding rc_old_len field to table recentchanges...ok
...have user_real_name field in user table.
...have user_token field in user table.
...have user_email_token field in user table.
...have user_registration field in user table.
...have log_params field in logging table.
...have ar_rev_id field in archive table.
...have ar_text_id field in archive table.
...have page_len field in page table.
...have rev_deleted field in revision table.
...have img_width field in image table.
...have img_metadata field in image table.
...have img_media_type field in image table.
...have ss_total_pages field in site_stats table.
...have iw_trans field in interwiki table.
...have ipb_range_start field in ipblocks table.
...have ss_images field in site_stats table.
Adding ipb_anon_only field to table ipblocks...ok
Adding user_newpass_time field to table user...ok
Adding user_editcount field to table user...ok
...already have interwiki table
...indexes seem up to 20031107 standards
Already have pagelinks; skipping old links table updates.
...image primary key already set.
The watchlist table is already set up for email notification.
...watchlist talk page rows already present
...user table does not contain old email authentication field.
Logging table has correct title encoding.
...page table already exists.
revision timestamp indexes already up to 2005-03-13
...rev_text_id already in place.
...page_namespace is already a full int (int(11)).
...ar_namespace is already a full int (int(11)).
...rc_namespace is already a full int (int(11)).
...wl_namespace is already a full int (int(11)).
...qc_namespace is already a full int (int(11)).
...log_namespace is already a full int (int(11)).
...already have pagelinks table.
...templatelinks table already exists
No img_type field in image table; Good.
Already have unique user_name index.
...user_groups table already exists.
...user_groups is in current format.
...wl_notificationtimestamp is already nullable.
...timestamp key on logging already exists.
Setting page_random to a random value on rows where it equals 0...changed 0 rows
Checking for additional recent changes indices...
...index on ( rc_namespace, rc_user_text ) not found; creating
...index on ( rc_user_text, rc_timestamp ) not found; creating
Checking for backlinking indices...
Checking if pagelinks index pl_namespace includes field pl_from...
...index pl_namespace on table pagelinks has no field pl_from; adding
Deleting old default messages...Done
Purging caches...done.
Done.

[edit] .

See WhyNotWiki / MediaWiki / Upgrades done for information about subsequent updates.