Wget – WhyNotWiki

Ethan Caldwell Blog

Save as a different filename with -O option.

Don’t forget to enclose the URL in if it has >1 parameters (& mark)!

# Not gonna work:
wget -O nt.gif http://www.neuroticweb.com/recursos/css-rounded-box/rounded.php
cn=nt&co=e4ecec&ci=A9B8CF

# Gonna work:
wget -O nt.gif 'http://www.neuroticweb.com/recursos/css-rounded-box/rounded.php
cn=nt&co=e4ecec&ci=A9B8CF'

Warning about & characters

Many URLs contain & characters. That’s fine as far as your web browser is concerned, but on the command line, & takes on a special meaning: it causes the command before the & to be started and backgrounded and then the part after the & (which in the case of a URL, wouldn’t be a valid command but it would be treated as a command nonetheless) to be started and foregrounded.

> echo 1 & echo 2
[2] 11524
2
1
[2]-  Done                    echo 1
> wget http://sourceforge.net/project/downloading.php?groupname=wikipedia&filename=mediawiki-1.6.5.tar.gz&use_mirror=easynews
[2] 11320
[3] 11321
...
[2]   Exit 1                  wget http://sourceforge.net/project/downloading.php?groupname=wikipedia
[3]-  Done                    filename=mediawiki-1.6.5.tar.gz

To get around this, simply and enclose the URL in quotes:

> wget "http://sourceforge.net/project/downloading.php?groupname=wikipedia&filename=mediawiki-1.6.5.tar.gz&use_mirror=easynews"

Retrying a troublesome download

http://playingwithsid.blogspot.com/2007/09/wget-re-tries.html. Retrieved on 2007-05-11 11:18.

So, the wget program grabbing the file from the Internet running in background sputters after 20 or so tries and gives up. This calls for extreme measures.

$ wget -t 0 -c http://examplesite.com/pub/xxxx.bz2

The -t ( --tries ) set to 0 (zero) which means it keeps on trying infinitely until the hell freezes over.

And -c ( --continue ) is to continue grabbing the half-downloaded file.

Downloading from sourceforge

Use the direct link it gives you (“Your download should begin shortly. If you are experiencing problems with the download please use this direct link.”).

So

http://easynews.dl.sourceforge.net/sourceforge/wikipedia/mediawiki-1.6.5.tar.gz

instead of

http://sourceforge.net/project/downloading.php?groupname=wikipedia&filename=mediawiki-1.6.5.tar.gz&use_mirror=easynews