Example Uses of the Linux Curl Command

Use 'curl' to upload and download files using common internet protocols

The Curl Command

 Gary Newell

The curl command transfers files over a network or from the internet a broad range of common protocols including HTTP, HTTPS, FTP, and even SMB. Use it to access FTP sites, send emails, connect to SAMBA addresses, upload and download files, and related tasks. 

The curl command has a lot of parameters to learn. You can see a full listing of them at the curl manual page.

Basic 'curl' Command Usage

In its basic form, curl downloads web page content straight to the terminal window.

For example, enter the following command into a terminal window:

curl http://www.google.com

The HTML of Google's landing page displays to your standard output.

Direct the Contents of 'curl' to a File

curl redirection

When you download an object like an ISO image, send it to a file instead of to the standard output.

To save the content to a file, specify the -o switch as follows:

curl -o <filenametocreate> <URL>

To download the page linked to in the basic command usage section, all you have to do is enter the following command:

curl -o google.htm http://www.google.com

After the file has downloaded, open it in an editor or its default program determined by the file type.

You can simplify this further by using the -O switch as follows:

curl -O http://www.google.com

This switch uses the filename portion of the URL and makes it the filename that the URL is saved to.

The -o and -O switches must reference a specific file; they cannot point generically to a folder or a top-level URL. Curl reports a no-length error in such situations.

Run 'curl' in the Background

By default, the curl command shows a progress bar telling you how much of the transfer is left and how much data has been transferred.

If you just want the command to run without a progress bar, you'll need to run it in silent mode. Then run it as a background command.

To run a command silently use the -s switch:

curl -s -O <URL>

To execute the command in the background, use the ampersand as follows:

curl -s -O <URL> &

Running the curl command without the progress bar is more streamlined, but you won't know how long the transfer will take or the estimated time it should complete.

Downloading Several URLs With 'curl'

curl for two sites

In its simplest form, download several URLs simultaneously by specifying them in sequence:

curl -O http://www.mysite.com/page1.html -O http://www.mysite.com/page2.html

To download a large number of files with a common naming pattern, use square brackets to supply a range. For example, to get image files suffixed 1 to 100, specify the following:

curl -O http://www.mysite.com/images/image[1-100].jpg

Use curly brackets to specify multiple sites with similar formats.

For example, to download the index pages for both www.google.com and www.bing.com, use the following command:

curl -O http://www.{google,bing}.com

Your shell may rewrite these commands as necessary. Learn more about regular expressions for relevant context.

Displaying Progress

By default the curl command returns the following information as it downloads a URL:

  • Total %: Percent of total transfer completed.
  • Total bytes: Total size of the transfer in bytes.
  • Received/Transferred %: Percentage of transfer completed.
  • Received/Transferred bytes: Number of bytes downloaded.
  • Average download speed: Download speed in bytes per second.
  • Average upload speed: Upload speed in bytes per second.
  • Total time: Estimated time to complete the current operation.
  • Time spent: Time passed since the start of transfer.
  • Time left: Expected time to complete the transfer.
  • Current speed: Current transfer speed in bytes per second.

If you would prefer a simple progress bar, just specify the -# switch as follows:

curl -# -O <URL>

Handling Redirects

The curl command is clever in that it can follow redirects. All you have to do is use the -L switch as follows:

curl -OL <URL>

Download Files From an FTP Server

The curl command handles more than just HTTP file transfers. It can handle FTP, GOPHER, SMB, HTTPS and many other formats.

To download files from an FTP server use the following command:

curl -u user:password -o <URL>

If you specify the name of a file as part of the URL then it will download the file but if you specify the name of a folder it will return a folder listing.

Use curl to upload files to an FTP server by using the following command:

curl -u user:password -T <filename(s)> <URL>

The filenames and <URL> use the same pattern matching as for downloading several HTTP files.