Cloud storage for MySQL Enterprise Backup (MEB) users

Jothir wrote great blog post, introducing MEB's "Backup to Cloud" feature. And while this blog post is great source of information for Amazon users who consider using MEB it is not so clear how to setup the cloud storage itself for those MEB users who are only considering storing backups in the Cloud. I actually was one of them and decided to share my experience.

OK, so what do you need to start using MEB with Amazon S3 service?

  1. Amazon AWS account. You can register it for free at https://portal.aws.amazon.com/gp/aws/developer/registration/index.html
  2. A bucket where backup image will be stored. To create it, choose S3 service under "Services" menu, then click "Create Bucket" button and enter the name. You will pass this name to --cloud-bucket option.
  3. Access key and secret access key for a user which will connect to AWS. You can use root user for such purpose or create a special one. Latter option is the safest. To create a user choose "AIM" service, then "Manage Users" under "Create individual AIM users" tab, then click "Create new users" and enter user names. After you created a user you will have an option to see its security credentials. It will be somethiing like this:
    Access Key ID:
    AKIAJ2DK2OBDEGQSBGSQ
    Secret Access Key:
    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

    You will pass "Access Key ID" to --cloud-access-key-id option and "Secret Access Key" to --cloud-secret-access-key option.
  4. After you created a user you need to grant necessary permissions to it. There are many ways to manage AWS user permissions, but for our test we can choose simplest way: click on "Attach User Policy" button under "User Policies" part, then select "Amazon S3 Full Access". We are ready to try backup now.
  5. To summarize, following Cloud options are required:
    --cloud-service=s3 - it is always s3 and must be specified
    --cloud-aws-region='us-west-2' - here is the tricky part: you could specify different region when created the bucket, but default region should be used, so AWS is able to find the bucket
    --cloud-bucket=ourbucket - name of the bucket
    --cloud-access-key-id=AKIAJ2DK2OBDEGQSBGSQ - access key ID
    --cloud-secret-access-key=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 - Secret Access Key
    --cloud-object-key=test - any name: this will be name of the image in the bucket
  6. Add common MEB options, such as:
    --backup-dir=/path/to/backup/dir/to/store/temporary/files
    --backup-image=- - so we are storing in stdout
    backup-to-image - this is not an option, but the command =)
  7. Our final command is:
    $mysqlbackup --cloud-service=s3 --cloud-aws-region='us-west-2' \
    --cloud-bucket=ourbucket --cloud-access-key-id=AKIAJ2DK2OBDEGQSBGSQ \
    --cloud-secret-access-key=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 \
    --cloud-object-key=test --backup-dir=/home/sveta/BACKUPDIR/cloud_test \
    --backup-image=- backup-to-image


    Here is the cloud operation part of the MEB output:

    ...
    Uploading part %:10 20 30 40 50 60 70 80 90
    mysqlbackup: INFO: Finalizing data upload.
    mysqlbackup: INFO: Cloud operation complete.
    ...


You can see the file in your bucket using AWS console.

Nice thing is that you don't need to care about URLs, remote streaming and things like this: MEB will take care about everything itself.
Comments:

It can also work without needing to give Full access on S3.
http://bugs.mysql.com/bug.php?id=73615

And if you're going to use a non-US bucket you might hit this bug:
http://bugs.mysql.com/bug.php?id=73616

Posted by Daniël van Eeden on September 06, 2014 at 03:38 AM PDT #

Post a Comment:
  • HTML Syntax: NOT allowed
About

MySQL MEB Team Blog

Search

Categories
Archives
« September 2015
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today