s3cmd failed to upload

Posted under others On By

Using s3cmd 1.0.1, I was unable to upload a 2.7G file. It’s not even 5GB.

WARNING: Upload failed: /private.raw.tar.gz ([Errno 32] Broken pipe)
WARNING: Retrying on lower speed (throttle=1.25)
WARNING: Waiting 15 sec…

Turns out there is a new feature with s3cmd 1.5 which is to split large files into multiple smaller ones, each one being 15M. Only then was I able to upload my file:

$ s3cmd --no-encrypt put private.raw.tar.gz s3://xxxxx
WARNING: Module python-magic is not available. Guessing MIME types based on file extensions.
WARNING: Redirected to: xxxxx.s3-us-west-2.amazonaws.com
private.raw.tar.gz -> s3://xxxxx/private.raw.tar.gz [part 1 of 188, 15MB]
15728640 of 15728640 100% in 26s 575.98 kB/s done

And it looks like the new version needs python-magic.

 579 total views,  2 views today

Leave a comment

Your email address will not be published. Required fields are marked *