🔶

AWS DataSync

  • Fully managed service to transfer large amounts of data between on-premises systems, AWS services, or other cloud providers.
  • On-premises/other cloud → AWS: supports NFS, SMB, HDFS, S3 API, etc. → requires a DataSync Agent.
  • AWS → AWS: transfers between supported AWS storage services without an agent.

Supported Targets

  • Amazon S3 (all storage classes, including Glacier)
  • Amazon EFS
  • Amazon FSx (Windows, Lustre, NetApp ONTAP, OpenZFS)

Key Features

  • Scheduled replication tasks (hourly, daily, weekly)
  • Preserves file permissions and metadata (NFS POSIX, SMB ACLs, etc.)
  • Each agent task can use up to 10 Gbps of bandwidth (limit configurable)
  • TLS encryption for data in transit

AWS DataSync – On-Premises to AWS

Architecture:
  1. NFS or SMB Server connects to a DataSync Agent running on-premises or on devices like AWS Snowcone.
  1. The agent communicates securely with the AWS DataSync service.
  1. Data is transferred to:
      • Amazon S3 (Standard, Intelligent-Tiering, Standard-IA, One Zone-IA, Glacier, Glacier Deep Archive)
      • Amazon EFS
      • Amazon FSx

AWS DataSync – AWS to AWS Transfers

  • Can move data and metadata between:
    • Amazon S3Amazon S3
    • Amazon S3Amazon EFS
    • Amazon S3Amazon FSx
    • Amazon EFSAmazon FSx
  • No agent needed for AWS-to-AWS transfers.
  • Maintains metadata and file attributes.