5 ways to transfer your data to Cloud Storage
You have a lot of data and would like to transfer it to the cloud. How do you do that? In this post, you’ll learn how to get your data into Cloud Storage with a variety of upload methods.
When you upload an object to your Cloud Storage bucket, it will consist of the data you want to store, along with any associated metadata. When it comes to the actual uploading, you’ve got a few different options to choose from, which we’ll go over below.
1. Cloud console
The Cloud Console provides you with an in-browser experience where you can easily click to create buckets and folders and then choose or drag and drop the files from your local machine to upload. Folder uploads are also supported if you’re using the Chrome browser.
2. gsutil
For production environments, you may want an automated, command-line solution. For this, you can use the gsutil tool. gsutil is a Python application that lets you access Cloud Storage from the command line, providing you with the ability to do all sorts of things like creating buckets, moving objects or even editing metadata.
To use it, run the gsutil program with a variety of command-line options. For example, the command below uploads a directory of files from your local machine to your Cloud Storage bucket using parallel upload.
And the command below lists out specific objects that have a version-specific URL using a wildcard.
More cool stuff you can do with the gsutil tool can be found here.
3. Client libraries
At some point, you might need to interface with Cloud Storage directly from your code, rather than going out to a command-line option. You can include the client libraries into your code and call a simple API to get data into a bucket or folder. It supports C++, C#, Go, Java, Node.js, PHP, Python, and Ruby.
For example, check out this Python code to upload an object to a Cloud Storage bucket:
Check out even more code samples here.
4. JSON and XML
And finally, if none of that does the trick, there’s always the JSON and XML APIs which can let you kick off an HTTP POST request to upload data directly to a bucket or folder. It’s a bit more complex, but it’s there if you need it.
5. Cloud Storage Transfer Appliance
For those who have a lot of data, it’s worth noting that it might not be feasible to upload all of that data directly from your on-prem systems to Google Cloud. For that, you can use the Cloud Storage Transfer Appliance. Google ships you a fancy device, you connect it, add your data and send it back to Google.
Don’t worry if your data is in another cloud. Google has easy-to-use guides to help you get up and running with supporting a multi-cloud environment and getting that data over to Cloud Storage.
