How to Manage Amazon S3 From the Command Line

Amazon S3 is a good place to back up and store your files cheaply on the cloud. While some people use it to store their personal data, there are others that use it to store images and scripts for their websites, and even use it as a CDN. On your desktop, you can easily access and manage your Amazon S3 account with apps like S3Fox, CloudBerry, DragonDisk, etc., but what if you need to access it from the command line? This is where S3cmd comes in useful.

S3cmd is a free command line tool for uploading, retrieving and managing data in Amazon S3. Other than S3, it also supports other cloud storage service providers that use the S3 protocol, such as Google Cloud Storage or DreamHost DreamObjects. The best use for this tool is on remote server where everything is done on the command-line. Alternatively, you can also use it to create batch scripts and automated backup to S3.

S3cmd is written in Python, so it is just about supported in all operating systems, as long as Python is installed.

S3cmd is found in most Linux repositories. In Debian/Ubuntu, or any other apt-based distro, you can install S3cmd with the command:

sudo apt-get install s3cmd

For Fedora, Centos, or any other yum-based distro:

sudo yum install s3cmd

Alternatively, you can download the source code, unzip the package and run the installer:

sudo python install

Before you get started, you will need to have your Amazon S3’s Access and Secret keys ready. You can locate the Access and Secret keys in the Amazon Management Console.

To get started, open the terminal and type:

s3cmd --configure

It will first prompt you to enter your Access and Secret keys.


Next, you have to enter an encryption key and the path to the GPG program.


After the encryption key is the prompt for using HTTPS protocol. The default option is “No”, but you can set to “Yes” for a more secure connection to Amazon S3. Do note that using the HTTPS protocol will slow down the transfer speed and can’t be used if you are using a proxy.


Once you have configured and tested the settings, you can proceed to use S3cmd. If you need to change the settings, you can either edit the .s3cfg file in your Home directory, or run the s3cmd --configure command again.

First, to view the buckets in your S3 account, you can use the ls command.

s3cmd ls


To create a bucket, use the mb command:

s3cmd mb s3://test-s3cmd-bucket

Note that you will need to prefix the bucket name with “s3://”


To list the content in a bucket, use the ls command together with the bucket name. For example:

s3cmd ls s3://test-s3cmd-bucket


Uploading file to bucket

The easiest way to upload a file via S3cmd is with the put command. For example:

s3cmd put text.txt s3://test-s3cmd-bucket

In addition, you can use the --encrypt parameter to encrypt the file before uploading to S3.

s3cmd put --encrypt text.txt s3://test-s3cmd-bucket

The default permission for the put command is private, which means the file can only be viewed by you. If you need the file to be publicly accessible, you can add the parameter --acl-public

s3cmd put --acl-public text.txt s3://test-s3cmd-bucket

For more granular control, use the --acl-grant=PERMISSION:EMAIL or USER_CANONICAL_ID parameter. For example:

s3cmd put text.txt s3://test-s3cmd-bucket

The “PERMISSION” can be “read,” “write,” “read_acp,” “write_acp,” “full_control,” “all.”

Retrieving files from bucket

To retrieve a file, use the get command.

s3cmd get s3://test-s3cmd-bucket/text.txt text.txt

To download all files in the bucket, simply append the --recursive parameter.

s3cmd get --recursive s3://test-s3cmd-bucket

Deleting files from bucket

The delete command for s3cmd is simply del. For example:

s3cmd del s3://test-s3cmd-bucket/text.txt

You can also use the --recursive parameter to delete all files in the bucket.

s3cmd del --recursive s3://test-s3cmd-bucket

For more S3cmd commands, check out its usage guide here.

Let’s say you store all your important files in a folder and you want it to be synced to S3. S3cmd comes with a sync command that can synchronize the local folder to the remote destination.

s3cmd sync --delete-removed ~/SecretFolder s3://my-secret-bucket/

All you have to do is to create a cronjob to run the sync command regularly.

1. Open the crontab.

crontab -e

2. Add the following line to the end of the crontab. Save and exit the crontab.

5 * * * * s3cmd sync --delete-removed ~/SecretFolder s3://my-secret-bucket/

That’s it. Your system will now sync the secret folder to S3 every 5 minutes. You can change the value to run the sync command at your preferred interval. Every file you removed from the secret folder will be removed from S3 too.

For those who need to work in the command line environment, S3cmd is really a great tool for you to access and manage Amazon S3 from the command line. Not only is it easy to use, it also comes with plenty of options for advanced usage and scripting needs.