Applies To: OBJECT STORAGE

rclone is a powerful command line based utility that allows you to connect to and manage various Cloud based storage systems, including our Object Storage (Memstore).


Configuring rclone

To add your Object Storage to your rclone configuration, you will need to run the rclone configuration utility;

rclone config
BASH

(Note: For Windows users, you many need to specify the full path to the rclone executable file, if its not included in your systems path variable)

When prompted for the following settings, enter the below details;

  • Enter "n" to create a New remote.
  • Provide a suitable name to allow you to easily identify it e.g. "Memstore".
  • For the "Type of storage to configure" enter the value for the "Openstack Swift (Rackspace Cloud Files, Memset Memstore, OVH)" option.
  • For the "Get swift credentials from environment variables in standard OpenStack form" enter "false".
  • For the "User name to log in (OS_USERNAME)", enter your chosen username (not the Access Username) e.g. "admin".
  • For the "API key or password (OS_PASSWORD)" enter the password for your selected user.
  • For the "Authentication URL for server (OS_AUTH_URL)" enter the value for the "Memset Memstore UK v2" option.
  • For the "Tenant name - optional for v1 auth, this or tenant_id required otherwise (OS_TENANT_NAME or OS_PROJECT_NAME)" enter your Object Storage Product Name e.g. "mstesteaa1"
  • For all other values, leave these as the default values, and press the Enter key to continue.
  • Once Configuration is complete, you will then be prompted to confirm your Configuration, an example of the completed rclone configuration can be seen below;
[Memstore]
type = swift
env_auth = false
user = admin
key = MyPassword123
auth = https://auth.storage.memset.com/v2.0
tenant = mstesteaa1
TEXT

Listing the Containers and Files and Folders within a Container

Once your Object Storage has been configured in rclone, you can then use the rclone interface to List all the Containers with the "lsd" command;

rclone lsd "Memstore:"
BASH

This will then produce an output similar to the below, listing the containers available;

$ rclone lsd "Memstore:"
      493279 2020-01-10 11:03:15         7 Test
   422672615 2020-01-10 11:03:15        31 backups
        1115 2020-01-10 11:03:15         5 fireworks
 73211637413 2020-01-10 11:03:15      1144 miniserver-snapshots
BASH

To list the files, within a container we can use the "ls" command (like you would on a Linux Terminal) e.g. ;

rclone ls "Memstore:backups/"
BASH

This will then produce an output similar to the below, listing the files and folders available within that container;

$ rclone ls "Memstore:backups/"
      804 2019-04-07/accounts/.master.meta
 12393177 2019-04-07/accounts/demo.tar.gz
 12375947 2019-04-07/accounts/stwordpress.tar.gz
 59535360 2019-04-07/system/system_files.tar
      804 2019-04-08/accounts/.master.meta
 12393181 2019-04-08/accounts/demo.tar.gz
 12375913 2019-04-08/accounts/stwordpress.tar.gz
 59607040 2019-04-08/system/system_files.tar
      804 2019-04-09/accounts/.master.meta
 12393171 2019-04-09/accounts/demo.tar.gz
 12375999 2019-04-09/accounts/stwordpress.tar.gz
 59688960 2019-04-09/system/system_files.tar
      805 2019-04-10/accounts/.master.meta
 12393189 2019-04-10/accounts/demo.tar.gz
 12375912 2019-04-10/accounts/stwordpress.tar.gz
 59678720 2019-04-10/system/system_files.tar
      805 2019-04-11/accounts/.master.meta
 12393808 2019-04-11/accounts/demo.tar.gz
 12375895 2019-04-11/accounts/stwordpress.tar.gz
 59709440 2019-04-11/system/system_files.tar
   602881 readme.txt/README.txt
BASH

Uploading and Downloading Files and Folders

rclone support a variety of options to allow you to Copy, Sync and Move (as well as others) files from one destination to another. A simple example of this can be seen below, where we copy (Upload) the file "upload.me" to the "Backups" container;

rclone copy "/home/user/upload.me" "Memstore:backups/"
BASH

The main principle here, is you specify the transfer method, then the source and destination and the Upload / Download will start.  The Command above is in its simplest form, you will likely want to change these settings to include Verbose Output, specify the maximum number of transfers at a time etc. However you can reverse the Source and Destination sections above to Download the file from your Object Storage to your local device.

Below are some useful links mostly to the various rclone options mentioned above, but also to some Graphical User Interfaces available for rclone.