Applies To: OBJECT STORAGE
rclone is a powerful command line based utility that allows you to connect to and manage various Cloud based storage systems, including our Object Storage (Memstore).
To add your Object Storage to your rclone configuration, you will need to run the rclone configuration utility;
(Note: For Windows users, you many need to specify the full path to the rclone executable file, if its not included in your systems path variable)
When prompted for the following settings, enter the below details;
- Enter "n" to create a New remote.
- Provide a suitable name to allow you to easily identify it e.g. "Memstore".
- For the "Type of storage to configure" enter the value for the "Openstack Swift (Rackspace Cloud Files, Memset Memstore, OVH)" option.
- For the "Get swift credentials from environment variables in standard OpenStack form" enter "false".
- For the "User name to log in (OS_USERNAME)", enter your chosen username (not the Access Username) e.g. "admin".
- For the "API key or password (OS_PASSWORD)" enter the password for your selected user.
- For the "Authentication URL for server (OS_AUTH_URL)" enter the value for the "Memset Memstore UK v2" option.
- If the option is not available the URL is "https://auth.storage.memset.com/v2.0"
- For the "Tenant name - optional for v1 auth, this or tenant_id required otherwise (OS_TENANT_NAME or OS_PROJECT_NAME)" enter your Object Storage Product Name e.g. "mstesteaa1"
- For all other values, leave these as the default values, and press the Enter key to continue.
- Once Configuration is complete, you will then be prompted to confirm your Configuration, an example of the completed rclone configuration can be seen below;
[Memstore] type = swift env_auth = false user = admin key = MyPassword123 auth = https://auth.storage.memset.com/v2.0 tenant = mstesteaa1
Listing the Containers and Files and Folders within a Container
Once your Object Storage has been configured in rclone, you can then use the rclone interface to List all the Containers with the "lsd" command;
rclone lsd "Memstore:"
This will then produce an output similar to the below, listing the containers available;
$ rclone lsd "Memstore:" 493279 2020-01-10 11:03:15 7 Test 422672615 2020-01-10 11:03:15 31 backups 1115 2020-01-10 11:03:15 5 fireworks 73211637413 2020-01-10 11:03:15 1144 miniserver-snapshots
To list the files, within a container we can use the "ls" command (like you would on a Linux Terminal) e.g. ;
rclone ls "Memstore:backups/"
This will then produce an output similar to the below, listing the files and folders available within that container;
$ rclone ls "Memstore:backups/" 804 2019-04-07/accounts/.master.meta 12393177 2019-04-07/accounts/demo.tar.gz 12375947 2019-04-07/accounts/stwordpress.tar.gz 59535360 2019-04-07/system/system_files.tar 804 2019-04-08/accounts/.master.meta 12393181 2019-04-08/accounts/demo.tar.gz 12375913 2019-04-08/accounts/stwordpress.tar.gz 59607040 2019-04-08/system/system_files.tar 804 2019-04-09/accounts/.master.meta 12393171 2019-04-09/accounts/demo.tar.gz 12375999 2019-04-09/accounts/stwordpress.tar.gz 59688960 2019-04-09/system/system_files.tar 805 2019-04-10/accounts/.master.meta 12393189 2019-04-10/accounts/demo.tar.gz 12375912 2019-04-10/accounts/stwordpress.tar.gz 59678720 2019-04-10/system/system_files.tar 805 2019-04-11/accounts/.master.meta 12393808 2019-04-11/accounts/demo.tar.gz 12375895 2019-04-11/accounts/stwordpress.tar.gz 59709440 2019-04-11/system/system_files.tar 602881 readme.txt/README.txt
Uploading and Downloading Files and Folders
rclone support a variety of options to allow you to Copy, Sync and Move (as well as others) files from one destination to another. A simple example of this can be seen below, where we copy (Upload) the file "upload.me" to the "Backups" container;
rclone copy "/home/user/upload.me" "Memstore:backups/"
The main principle here, is you specify the transfer method, then the source and destination and the Upload / Download will start. The Command above is in its simplest form, you will likely want to change these settings to include Verbose Output, specify the maximum number of transfers at a time etc. However you can reverse the Source and Destination sections above to Download the file from your Object Storage to your local device.
Below are some useful links mostly to the various rclone options mentioned above, but also to some Graphical User Interfaces available for rclone.