Difference between revisions of "Wget"

From Christoph's Personal Wiki
Jump to: navigation, search
Line 1: Line 1:
 
{{lowercase|wget}}
 
{{lowercase|wget}}
'''wget''' - The non-interactive network downloader.
+
'''wget''' — The non-interactive network downloader.
  
 
== Download multiple files ==
 
== Download multiple files ==
* Create variable that holds all urls and then using 'BASH foo loop' to download all files:
+
* Create variable that holds all URLs and then using 'BASH for loop' to download all files:
 
  % URLS="<nowiki>http://www.example.com/foo.tar.gz ftp://ftp.example.org/pub/bar.tar.gz</nowiki>"
 
  % URLS="<nowiki>http://www.example.com/foo.tar.gz ftp://ftp.example.org/pub/bar.tar.gz</nowiki>"
 
* Use for loop as follows:
 
* Use for loop as follows:
 
  % for u in $URLS; do wget $u; done
 
  % for u in $URLS; do wget $u; done
  
* You can also put a list of the URLs in a file and download using the <tt>-i</tt> option:
+
* You can also put a list of the URLs in a file and download using the <code>-i</code> option:
 
  % wget -i download.txt
 
  % wget -i download.txt
  

Revision as of 07:12, 16 January 2007

The correct title of this article is wget. The initial letter is capitalized due to technical restrictions.

wget — The non-interactive network downloader.

Download multiple files

  • Create variable that holds all URLs and then using 'BASH for loop' to download all files:
% URLS="http://www.example.com/foo.tar.gz ftp://ftp.example.org/pub/bar.tar.gz"
  • Use for loop as follows:
% for u in $URLS; do wget $u; done
  • You can also put a list of the URLs in a file and download using the -i option:
% wget -i download.txt

Automating/scripting download process

#!/bin/sh
# wget-list: manage the list of downloaded files

# invoke wget-list without arguments

while [ `find .wget-list -size +0` ]
 do
  url=`head -n1 .wget-list`
   wget -c $url
   sed -si 1d .wget-list
 done
#/bin/sh
# wget-all: process .wget-list in every subdirectory
# invoke wget-all without arguments

find -name .wget-list -execdir wget-list ';'
#!/bin/sh
# wget-dirs: run wget-all in specified directories
# invoking: wget-dirs <path-to-directory> ...

for dir in $*
  do
      pushd $dir
      wget-all
      popd
  done
wget-all

See also

External links

This article is curently a "stub". This means it is an incomplete article needing further elaboration.

I always welcome suggestions, comments, and criticism. If you have something to contribute to this site, please follow this link: Contributing Information. Thank you!