Menu
blog.headdesk.me
blog.headdesk.me

s3cmd failed to upload

Posted on 2014/03/07

Using s3cmd 1.0.1, I was unable to upload a 2.7G file. It’s not even 5GB.

WARNING: Upload failed: /private.raw.tar.gz ([Errno 32] Broken pipe)
WARNING: Retrying on lower speed (throttle=1.25)
WARNING: Waiting 15 sec…

Turns out there is a new feature with s3cmd 1.5 which is to split large files into multiple smaller ones, each one being 15M. Only then was I able to upload my file:

$ s3cmd --no-encrypt put private.raw.tar.gz s3://xxxxx
WARNING: Module python-magic is not available. Guessing MIME types based on file extensions.
WARNING: Redirected to: xxxxx.s3-us-west-2.amazonaws.com
private.raw.tar.gz -> s3://xxxxx/private.raw.tar.gz [part 1 of 188, 15MB]
15728640 of 15728640 100% in 26s 575.98 kB/s done

And it looks like the new version needs python-magic.

facebookShare on Facebook
TwitterTweet

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Full text search

Recent Posts

  • Dumping AWS Organization tree
  • Free is the most expensive
  • Terraform conditional resource and blocks
  • Upgrade Ubuntu 16.04 to latest release
  • Inspect and control network traffic on AWS
  • aws (8)
  • coffee (1)
  • headfi (1)
  • linux (7)
  • others (55)
  • security (2)
  • tech (36)
  • wordpress (2)

apache aws awscli azure backup cloud coffee coreos distributed filesystem docker ec2 EL8 elasticcache etckeeper featured heartbleed kernel linux mail meltdown mysql php pine python rdp rds Redhat Red Hat RHEL RHEL7 rpm Ryzen snapshot spectre SSL systemd tech terraform ubuntu ubuntu upgrade vector vpn wordpress xtreemfs yum

©2022 blog.headdesk.me | Powered by SuperbThemes & WordPress