How to Download a File From the Linux Command Line

The 'wget' command is a jack-of-all-trades for file acquisition

Using wget in the Linux command line

Lifewire

Use the Linux command wget to download files to you computer. Run it interactively through a shell prompt, or batch your downloads into a text file to automate the download.

For most distributions, wget installs by default, but if you can't find it, just install it through your package manager.

How to Download a File From the Command Line

The most straightforward approach to using wget is to specify the command as well as a remote file. The file must be a fully-formed URL including the protocol. For example:

wget http://www.test.com/files/sample-data.txt

If you do not specify a filename, wget tries to siphon everything from the specified URL, which usually leads to disastrous results for your disk space and bandwidth metering. However, if you're unsure of the filename,

You could, however, download all files with a particular extension by adding two flags. The -r flag downloads recursively, meaning it checks the location and then all folders and subfolders within it. The -A flag specifies a comma-separated list of extensions to to grab, excluding all other files. (The -R flag works in opposite fashion, specifying extensions to reject but grabbing everything else).

wget -r -A "txt" http://www.test.com

Use --accept-regex= and --reject-regex= and add a regular expression in quotes, for even more precise matching.

Batch Downloads

Specify a list of files to download using the -i switch. Start by creating a regular text file with a list of downloads, one full URL per line.

Use wget to download all of the files using the following command:

wget -i filelist.txt

The trouble with downloading files from the internet is that sometimes the file or URL is unavailable. The timeout for the connection can take a while and if you are trying to download lots of files it is counter-productive to wait for the default timeout.

Limiting Downloads

Specify your own timeout (the amount of time the system waits for a file to begin, in seconds) using the following syntax:

wget -T 5 -i filelist.txt

To limit the size of the files, use the --quota= flag:

wget --quota=100m -i filelist.txt

The above command will stop the download of files once 100 megabytes has been reached. You can also specify the quota in bytes (use b instead of m) or kilobytes (use k instead of m).

You may not have a download limit but you might have a slow internet connection. To download files without devouring all the available bandwidth on your connection, specify a limit that sets a maximum download rate.

For example:

wget --limit-rate=20k -i filelist.txt

The above command limits the download rate to 20 kilobytes per second. Specify the amount in bytes, kilobytes or megabytes.

Use wget -c to continue a download where you had left off—a tool useful, for example, with dropped connections.