Migrating data is often the first step towards adopting the cloud. However, when uploading data to the cloud, sometimes even the fastest available public internet connections fall short. For example, on a leased T3 line, migrating 100 TB of data can take up to 8 months – an untenable situation! Oracle Cloud FastConnect offers a great alternative to quickly upload data to the cloud. But it’s understandable that using FastConnect may not always be feasible for you, especially when you don’t expect to upload data frequently or when the data migration is a part of an effort to retire your on-premise datacenter.
A few short months ago, when we announced the availability of Data Transfer Disk, we promised that there was more to come. Today, I am excited to announce the general availability of Oracle Cloud Infrastructure Data Transfer Appliance.
Oracle Cloud Infrastructure Data Transfer Appliance is a PB-scale offline data transfer service. You can now use an Oracle-branded, purpose-built storage appliance to cost-effectively and easily migrate your data to the cloud. Each transfer appliance supports migrating up to 150 TB of data. To migrate PB-scale data sets, you can simply order multiple transfer appliances. The best part is that we charge you exactly $0 to use the service. That’s right, Oracle Cloud customers are able to use the Data Transfer Appliance for free. We even pay for the cost of shipping the appliance.
From the time you receive the transfer appliance, you have up to 30 days to copy your data and ship it back to the nearest Oracle data transfer site. When we receive the data transfer appliance, we upload the data to your Oracle Cloud Object or Archive Storage using high-speed internet connections. Large datasets that would’ve taken weeks or months to upload can now be uploaded in a fraction of the time.
The data transfer appliance is a 2u device that can rest standalone on a desk or fit in a standard rack. Weighing just 38 pounds, the appliance is easily handed by one person. The appliance was built with safety at the forefront. It’s tamper resistant and tamper evident. Only the serial port and the network ports are exposed. Any attempt to access the transfer appliance hardware in non-standard ways is detected. All the data copied to the transfer appliance is encrypted by default. The encryption passphrase is stored separately, never on the device with the data. The transfer appliance is shipped to you in a ruggedized case to shield it from the G-forces of transportation. You must ship the transfer appliance back to Oracle in the same shipping case.
Oracle Cloud Infrastructure Data Transfer Appliance Shipping Case
How It Works
Order the Data Transfer Service
To use the data transfer appliance to ship your data, place an order for the desired quantity of data transfer appliances. Your Oracle sales rep can help you with the order. Make sure that you have also purchased sufficient Oracle cloud credits, so that we can upload your data to your Oracle cloud tenancy. Placing an order for the data transfer service entitles you to the use of this service.
Requesting the Transfer Appliance
To request an appliance, log into the Oracle Cloud Infrastructure Console and create a Transfer Job of the type Appliance, in a region of your choice. While creating the Transfer Job, you must also specify the bucket to which the data must be uploaded. Currently, all data from a single transfer appliance can be uploaded to only one bucket.
Next, select the transfer job that you created and click the Request Transfer Appliance button. Specify the address to which the appliance must be shipped. A transfer appliance label is generated with the status Requested, which indicates that Oracle has received your request. When the status of the appliance changes from Requested to Oracle Preparing, your request has been accepted and the transfer appliance you requested will be shipped shortly. If you are requesting more than one transfer appliance, you can request that the appliances be shipped to multiple locations.
Preparing the Transfer Appliance
When you receive the data transfer appliance, it comes with a security tag with a unique number engraved on it. Verify that the tag label matches the number posted in the Oracle Cloud Console. If the number matches, retrieve the transfer appliance from the case, plug it into your network, and assign an IP to it through the serial console.
You can use the provided USB – serial cable and your favorite terminal emulator to access the serial console. You need to unlock the transfer appliance before you can use it. Download the Data Transfer Utility on a Linux host and follow the instructions to prepare the transfer appliance. Retrieve the encryption passphrase using the Data Transfer Utility. This encryption passphrase is used to encrypt the data on the transfer appliance. When the transfer appliance is unlocked and ready for use, create a dataset. A dataset is essentially an NFSv3 mount point. Currently, we support creating one dataset per transfer appliance. That’s it! You just configured the Data Transfer Appliance as an NFS filer.
Copying Data to the Data Transfer Appliance
Mount the NFSv3 dataset on any Linux compatible host of your choice and copy data to it using regular file system commands. We preserve the source data file/folder hierarchy by storing objects as a flattened file name, For exampe, a file in the folder hierarchy Logs->July2018->DBLog001.txt will be stored as an object name /Logs/July2018/DBLog001.txt, which simulates a virtual folder hierarchy in Oracle Object or Archive Storage.
Once you have copied all the data to the transfer appliance, seal the dataset. Sealing the dataset creates a manifest file that contains an index of all the files copied, including the file MD5 hashes, which are used to verify the integrity of data as we upload data to your Oracle cloud tenancy. Finally, Finalize the transfer appliance. At this point, you can no longer access the appliance for dataset operations. The transfer appliance is now ready to be shipped back to the Oracle transfer site.
Shipping the Appliance Back to Oracle
When we ship you the data transfer appliance, included in the shipping case is a return shipping label, which you must use to ship the transfer appliance back to the nearest Oracle data transfer site. If you misplace the return shipping label, reach out to us and we are happy to provide you a copy of the shipping label. Make sure that you return the transfer appliance within the allocated 30 days period. If you need more time, request an extension by creating a support request (SR).
Chain of Custody
Using the Oracle Cloud Console or the Data Transfer Utility, you can track the status of the data transfer process throughout its lifecycle, from the time you requested the appliance to the time the data is uploaded to your Oracle cloud tenancy.
Confirmation that Data was Uploaded to your Oracle Cloud Tenancy
When Oracle processes your transfer appliance and uploads data to your Oracle cloud tenancy, a data upload summary is posted to the same bucket where the data was uploaded. The following is a sample of the upload summary:
The upload summary provides a summary and detailed view of the successful and unsuccessful file uploads. It provides information on why some files were skipped so that you can take the necessary corrective action. Before you delete the primary copy of the data, it’s important that you review the upload summary and verify the content in your Object Storage bucket.
Once the upload process is complete, your transfer appliance status changes to Complete. Once the transfer job is complete, you must close it out. Closing a transfer job requires that the status of every associated transfer appliance is in a completed state.
If you need help, reach out to the Oracle support channels.
The Data Transfer Appliance service is available for use in the US regions (Phoenix and Ashburn), but we will be rolling out the service to other Oracle Cloud Infrastructure regions soon. For more information, please refer to the FAQs and Data Transfer Appliance product documentation.