This page documents known limitations of Transfer Appliance. If you anticipate ordering many appliances for a large transfer and cannot follow the limitations listed below, you must perform an initial proof-of-concept transfer with a single Transfer Appliance and representative data to ensure performance is suitable before you order multiple appliances.
Maximum file size
Cloud Storage supports a maximum single-object size of 5 tebibytes. Transfers from Transfer Appliance to Cloud Storage fail for objects larger than 5 TiB.
Maximum files per directory
Cloud Storage supports a maximum of 500,000 files or directories in a single directory to avoid poor ingest performance.
Maximum path length
The maximum object name length for data copied to Cloud Storage is 1024 bytes. The object prefix for the destination object is included in the length limitation because the object prefix is incorporated in the object's name in Cloud Storage.
Minimum file size
The minimum file size for files copied onto the appliance is 1MB. To copy many smaller files, we recommend that you archive the files before copying them.
Supported file metadata
Transfer Appliance preserves symlinks, numeric MODE and the last
modified time (mtime) from Transfer Appliance file system to copy
into the corresponding Cloud Storage destination object. Other file metadata
are not preserved in the transfer.
Supported file names
File names should be Unicode-compatible and not contain newlines. If your source directory contains file names with newlines, the copy from the appliance to Cloud Storage for that directory fails.
We strongly recommend that you replace newlines in your file names before copying data to Transfer Appliance.
For more information, see Cloud Storage's requirements for object naming.
Supported file types
Transfer Appliance supports transferring regular files. Non-regular files, such as devices, named pipes, or sockets are not supported.
Transfer Appliance does not create empty directories in Cloud Storage when we transfer from the appliance to your Cloud Storage bucket, because objects don't reside within subdirectories within a bucket. For more information, see Object namespace.
Object Versioning
Using Transfer Appliance with a bucket where object versioning is enabled may result in multiple versions of the same object being created and storage costs associated with these versions. We recommend that you disable object versioning for the bucket you specified as the destination at the time of ordering your Transfer Appliance.
To learn more, see Object versioning. Follow these instructions to disable object versioning for your bucket.