Amazon S3

The S3 transport allows transfer of files from the localhost to an Amazon S3 bucket or from a remote bucket to the localhost (transfer of directories is not supported yet).

This transport requires an AWS account. You'll have to choose a region, create a bucket, and get an access key id and secret key. Refer to the AWS S3 documentation to create an S3 bucket.

Bucket policy

If you plan on using the credentials of your primary AWS account, no further configuration is needed as this account owns your S3 buckets.

However, a more secure approach is to create a specific user using IAM and to add an access policy to the bucket. The S3 transport only requires a limited set of permissions: list the bucket contents, get, put and delete an object.

{
  "Version": "2008-10-17",
  "Id": "ARBITRARY_POLICY_ID",
  "Statement": [
    {
      "Sid": "ARBITRARY_STMT_ID",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::AWS_ACCOUNT_ID:user/IAM_USER"
      },
      "Action": [
        "s3:ListBucket",
        "s3:DeleteObject",
        "s3:GetObject",
        "s3:PutObject"
      ],
      "Resource": [
        "arn:aws:s3:::BUCKET_NAME/*",
        "arn:aws:s3:::BUCKET_NAME"
      ]
    }
  ]
}

copy

Unlike the move verb, copy preserves the input resource during the transfer. It also means that subsequent agents will assume the resource is still located on its original location.

Copy a file from the localhost to a remote bucket

job "daily-report" do
  resource "file", path: "/var/daily/report"
  copy to: "my-bucket/path/to/directory", using: "s3", region: "eu-central-1", access_key_id: "ACCESS_KEY_ID", secret_key: "SECRET"
end

Copy a file from a remote bucket to the localhost

job "daily-report" do
  resource "s3_object", bucket: "my-bucket", path: "path/to/directory", region: "eu-central-1"
  copy to: "localhost", using: "s3", access_key_id: "ACCESS_KEY_ID", secret_key: "SECRET"
end

move

❗️

Destructive action

Unlike the copy verb, move destroys the input resource once the file has been copied to the target location.

Move a file from the localhost to a remote bucket

job "daily-report" do
  resource "file", path: "/var/daily/report"
  move to: "my-bucket/path/to/directory", using: "s3", region: "eu-central-1", access_key_id: "ACCESS_KEY_ID", secret_key: "SECRET"
end

Move a file from a remote bucket to the localhost

job "daily-report" do
  resource "s3_object", bucket: "my-bucket", path: "path/to/directory", region: "eu-central-1"
  move to: "localhost", using: "s3", access_key_id: "ACCESS_KEY_ID", secret_key: "SECRET"
end

Options

You'll likely want to Specify default values globally for access to your S3 buckets (access_key_id, secret_key, region).