The "Copy EBS Snapshots to S3" action is used to copy the contents of one or more EBS snapshots to Amazon S3.
How Does It Work?
In order to copy the contents of the EBS snapshots to S3, one or more helper EC2 instances must be launched in the AWS account.
The helper EC2 instances do the following:
- Create a temporary EBS volume from the EBS snapshot.
- Mount the new EBS volume.
- Copy the contents to Amazon S3.
- Terminate the EC2 instance and delete the temporary EBS volume.
Important: The contents of the EBS snapshot never touch the Skeddly infrastructure.
Before the "Copy EBS Snapshots to S3" action can be executed, the action must be configured with the following:
- One or more VPC subnets in which the helper EC2 instances will execute,
- One security group in the same VPC as the subnet(s)
Note: The VPC subnets do not need to be in the same VPC or availability zones as the original EC2 instances or EBS volumes. Only the AWS region must be the same as the EBS snapshots.
We have created a CloudFormation template that will create the above resources and tag them so they can be found by Skeddly.
This template should be used in each region from which EBS snapshots are to be copied.
Also, for copy operations that take longer than 60 minutes, an IAM role needs to be assigned to the helper EC2 instance.
Use this CloudFormation template to create the necessary IAM role.
This template should be created once, in any region.
Configuring the Action
The "General Information" and "Schedule and Triggers" sections are configured just like any other Skeddly action.
Credentials and Regions
In this section, select the desired AWS account and AWS region(s) in which the action will be executed.
This action supports multiple AWS regions.
This section is used to select the EBS snapshots that are to be copied to S3. Each field are combined in an "AND" combination:
- Volume IDs - IDs of the EBS volumes of which EBS snapshots will be copied. If left blank, EBS snapshots from any EBS volume can be copied.
- Resource Tag Comparisons - Comparisons to perform on the resource tags of the EBS snapshots. Only those EBS snapshots that match all comparisons can be copied.
- Newer Than - Indicates the maximum snapshot age for EBS snapshots. Any EBS snapshot older than specified will not be copied. Leave this field blank to copy EBS snapshots of any age.
This section is used to specify the S3 target information.
- Bucket Name - Specifes the name of the bucket to which EBS snapshots will be copied.
- Prefix - The folders and sub-folders in the S3 bucket into which the files will be uploaded.
- Upload Format - Possible options include "Flat", "Tar", and "Tar Gzipped"
- Archive Filename - If using a "Tar" or "Tar Gzipped" upload format, the files on each EBS snapshot are combined into a single file with this filename.
- Server-Side Encryption - Indicates whether server-side encryption should be specified when uploading the files to S3.
- Storage Class - Indicates the storage class for the uploaded files.
Pro Tip: To help improve performance and reduce data transfer, you can use the
$(REGION) macro in the bucket name to use a different bucket for each region's snapshots.
For example, you can specify the bucket name as
my-bucket-$(REGION) . In this case, the following buckets would be used (depending on the location of the EBS snapshots):
Skeddly won't validate the bucket location, so create each of the above buckets in the appropriate region.
Helper Instance Information
This section is used to specify the network configuration to use for the helper EC2 instances that Skeddly will use to execute the copy process.
- Instance Type - The instance type to use for the helper EC2 instances
- Network Parameters Identification Method - Method to use to identify the subnets, security groups, and IAM roles to use for the helper EC2 instances.
- Max. Helpers Per Region - Indicates the maximum number of helpers regions that will be launched simultaneously in each region. By default, AWS limits each region to 20 running EC2 instances.
Pro Tip: Use the CloudFormation template in the "Prerequisites" section above to create one CloudFormation stack in each AWS region from which EBS snapshots will be copied.
When creating the stacks, leave the parameters with the default values. Then, leave the "Helper Instance Information" section with the default values.
Some file system types may not be supported. File systems known to not work are:
- ReFS for Windows
The "Copy EBS Snapshots to S3" action only works in the AWS commercial regions. It does not work in the GovCloud or Chinese regions.