With a fast Internet connection you will be able to download a large file without any issue. However, there are times when the network (or ISP) that you are connected to restricts your bandwidth or file download size. This is usually the case for educational institutions and places that offer free WiFi. So what can you do about it?
For such restrictive situations where you desperately need to download that large file to your computer, one of the solutions is to use cURL to split the file into smaller parts and combine them together again after all the parts are downloaded.
Note: this tutorial is done on Ubuntu, though it will work on any other Linux distro as well as OS (including Windows and Mac OS X).
Split and download large file with cURL
1. To get started, first make sure that cURL is installed in your system.
sudo apt-get install curl
You can also download cURL packages and the installer here.
2. As an illustration, I will assume that my network has a 200MB file download limit, and I am going to download the Ubuntu 15.04 ISO file (Download link: http://mirror.pnl.gov/releases/15.04/ubuntu-15.04-desktop-amd64.iso) which is 1.1GB.
The plan is to split the ISO file into 6 parts, each of them 200MB. In the terminal the command is:
curl --range 0-199999999 -o ubuntu-iso.part1 http://mirror.pnl.gov/releases/15.04/ubuntu-15.04-desktop-amd64.iso
--range flag tell cURL to download only the first 200MB of the file and save it as “ubuntu-iso.part1.”
We will do the same thing for the other parts.
curl --range 200000000-399999999 -o ubuntu-iso.part2 http://mirror.pnl.gov/releases/15.04/ubuntu-15.04-desktop-amd64.iso curl --range 400000000-599999999 -o ubuntu-iso.part3 http://mirror.pnl.gov/releases/15.04/ubuntu-15.04-desktop-amd64.iso curl --range 600000000-799999999 -o ubuntu-iso.part4 http://mirror.pnl.gov/releases/15.04/ubuntu-15.04-desktop-amd64.iso curl --range 800000000-999999999 -o ubuntu-iso.part5 http://mirror.pnl.gov/releases/15.04/ubuntu-15.04-desktop-amd64.iso curl --range 1000000000- -o ubuntu-iso.part6 http://mirror.pnl.gov/releases/15.04/c
You will notice that the last command doesn’t come with an end range. That means it will download from 1.0GB onward to the end of the file.
Once you are done downloading all the parts, you should now have 6 files in your computer.
The last thing to do is to combine them back to a single file. This can be done with the
cat ubuntu-iso.part? > ubuntu-15.04-desktop-amd64.iso
Note: for Windows users you can use the
copy command instead of
And a md5 checksum of the combined file shows that it is the same file as the one in the server.
There you have it: the large file that you wouldn’t be able to get using the usual download method.