curl is a cross-platform command line for getting and sending files using URL syntax. It is a recursive acronym that stands for Curl URL Request Library, and it is a very extensive program, supporting a wide range of internet protocols including HTTP, HTTPS, FTP, FTPS, SCP, SFTP, TFTP, LDAP, LDAPS, DICT, TELNET, FILE, IMAP, POP3, SMTP and RTSP.
curl has a ton of features and tricks, including user authentication, proxy support, FTP upload and download, HTTP post, cookies, file transfer resume, SSL connections and much more. This article will serve as an introduction to curl, for those unfamiliar with the program, while power users can also benefit.
To install curl on a Debian/Ubuntu system, run the following command:
At its most basic, curl expects a url argument, and it will attempt to fetch whatever file is available at the given url.
By default, the fetched file contents are dumped to the command line. If an output file is specified, the program will show a progress meter which shows the amount of data transferred, transfer speeds, estimated time left, and time spent, among others. To save the fetched file to your system, with a given filename, use the
To save a file with the same name as it has on the server, use the
To get a file from an ftp server that requires username and password:
To specify multiple URLs or parts of a URL, put the part sets in braces:
You can also specify a range in an alphanumeric series by using square brackets:
To save the fetched files into multiple documents, use the ‘#’ symbol followed by a number in the filename specifier. This will then get replaced by the current string in the given URL
You can specify the User-Agent string to be used while identifying to http servers, for http connections use the
To send data using HTTP POST, use the
If you are, for example, logging in to a website, you most likely want to store the supplied cookies for later use. To store the received cookies, use the
To use these stored cookies, use the
To upload a file, use the
-T option. For http(s) servers, the
PUT command will be used instead:
To force the use SSL/TLS for connections, use the
--ssl-reqd flag. You can also specify multiple files in the same manner you specify multiple URLs:
In addition, you can specify the login credentials with the -u option:
curl is a very simple, solid and yet feature-full program. There are so many different options available, and it handles a very large number of protocols, yet we have just barely scratched the surface of its abilities. This is an introduction to curl, for users who haven’t used it before, and a reminder for those who are familiar with it’s possibilities. Don’t forget to check out the manpages or type the following for more info:
If you have a particular use for this, please share with us in the comments.