Backends

The Backend class provides the abstraction to access a storage array with an specific configuration, which usually constraint our ability to operate on the backend to a single pool.

Note

While some drivers have been manually validated most drivers have not, so there’s a good chance that using any non tested driver will show unexpected behavior.

If you are testing cinderlib with a non verified backend you should use an exclusive pool for the validation so you don’t have to be so careful when creating resources as you know that everything within that pool is related to cinderlib and can be deleted using the vendor’s management tool.

If you try the library with another storage array I would love to hear about your results, the library version, and configuration used (masked IPs, passwords, and users).

Initialization

Before we can have access to an storage array we have to initialize the Backend, which only has one defined parameter and all other parameters are not defined in the method prototype:

class Backend(object):
    def __init__(self, volume_backend_name, **driver_cfg):

There are two arguments that we’ll always have to pass on the initialization, one is the volume_backend_name that is the unique identifier that cinderlib will use to identify this specific driver initialization, so we’ll need to make sure not to repeat the name, and the other one is the volume_driver which refers to the Python namespace that points to the Cinder driver.

All other Backend configuration options are free-form keyword arguments. Each driver and storage array requires different information to operate, some require credentials to be passed as parameters, while others use a file, and some require the control address as well as the data addresses. This behavior is inherited from the Cinder project.

To find what configuration options are available and which ones are compulsory the best is going to the Vendor’s documentation or to the OpenStack’s Cinder volume driver configuration documentation.

Cinderlib supports references in the configuration values using the forms:

• $[<config_group>.]<config_option>

• ${[<config_group>.]<config_option>}

Where config_group is backend_defaults for the driver configuration options.

Attention

The rbd_keyring_file configuration parameter does not accept templating.

Examples:

• target_ip_address='$my_ip'

• volume_group='my-${backend_defaults.volume_backend_name}-vg'

Attention

Some drivers have external dependencies which we must satisfy before initializing the driver or it may fail either on the initialization or when running specific operations. For example Kaminario requires the krest Python library, and Pure requires purestorage Python library.

Python library dependencies are usually documented in the driver-requirements.txt file, as for the CLI required tools, we’ll have to check in the Vendor’s documentation.

Cinder only supports using one driver at a time, as each process only handles one backend, but cinderlib has overcome this limitation and supports having multiple Backends simultaneously.

Let’s see now initialization examples of some storage backends:

LVM

import cinderlib

lvm = cinderlib.Backend(
    volume_driver='cinder.volume.drivers.lvm.LVMVolumeDriver',
    volume_group='cinder-volumes',
    target_protocol='iscsi',
    target_helper='lioadm',
    volume_backend_name='lvm_iscsi',
)

XtremIO

import cinderlib

xtremio = cinderlib.Backend(
    volume_driver='cinder.volume.drivers.dell_emc.xtremio.XtremIOISCSIDriver',
    san_ip='10.10.10.1',
    xtremio_cluster_name='xtremio_cluster',
    san_login='xtremio_user',
    san_password='xtremio_password',
    volume_backend_name='xtremio',
)

Kaminario

import cinderlib

kaminario = cl.Backend(
    volume_driver='cinder.volume.drivers.kaminario.kaminario_iscsi.KaminarioISCSIDriver',
    san_ip='10.10.10.2',
    san_login='kaminario_user',
    san_password='kaminario_password',
    volume_backend_name='kaminario_iscsi',
)

For other backend configuration examples please refer to the Validated drivers page.

Available Backends

Usual procedure is to initialize a Backend and store it in a variable at the same time so we can use it to manage our storage backend, but there are cases where we may have lost the reference or we are in a place in our code where we don’t have access to the original variable.

For these situations we can use cinderlib’s tracking of Backends through the backends class dictionary where all created Backends are stored using the volume_backend_name as the key.

for backend in cinderlib.Backend.backends.values():
    initialized_msg = '' if backend.initialized else 'not '
    print('Backend %s is %sinitialized with configuration: %s' %
          (backend.id, initialized_msg, backend.config))

Installed Drivers

Available drivers for cinderlib depend on the Cinder version installed, so we have a method, called list_supported_drivers to list information about the drivers that are included with the Cinder release installed in the system.

The method accepts parameter output_version where we can specify the desired output format:

• 1 for human usage (default value).

• 2 for automation tools.

The main difference are the values of the driver options and how the expected type of these options is described.

import cinderlib

drivers = cinderlib.list_supported_drivers()

And what we’ll get is a dictionary with the class name of the driver, a description, the version of the driver, etc.

Here’s the entry for the LVM driver:

{'LVMVolumeDriver':
    {'ci_wiki_name': 'Cinder_Jenkins',
     'class_fqn': 'cinder.volume.drivers.lvm.LVMVolumeDriver',
     'class_name': 'LVMVolumeDriver',
     'desc': 'Executes commands relating to Volumes.',
     'supported': True,
     'version': '3.0.0',
     'driver_options': [
         {'advanced': 'False',
          'default': '64',
          'deprecated_for_removal': 'False',
          'deprecated_opts': '[]',
          'deprecated_reason': 'None',
          'deprecated_since': 'None',
          'dest': 'spdk_max_queue_depth',
          'help': 'Queue depth for rdma transport.',
          'metavar': 'None',
          'mutable': 'False',
          'name': 'spdk_max_queue_depth',
          'positional': 'False',
          'required': 'False',
          'sample_default': 'None',
          'secret': 'False',
          'short': 'None',
          'type': 'Integer(min=1, max=128)'},
     ],
    }
},

The equivalent for the LVM driver for automation would be:

import cinderlib

drivers = cinderlib.list_supported_drivers(2)

 {'LVMVolumeDriver':
     {'ci_wiki_name': 'Cinder_Jenkins',
      'class_fqn': 'cinder.volume.drivers.lvm.LVMVolumeDriver',
      'class_name': 'LVMVolumeDriver',
      'desc': 'Executes commands relating to Volumes.',
      'supported': True,
      'version': '3.0.0',
      'driver_options': [
         {'advanced': False,
          'default': 64,
          'deprecated_for_removal': False,
          'deprecated_opts': [],
          'deprecated_reason': None,
          'deprecated_since': None,
          'dest': 'spdk_max_queue_depth',
          'help': 'Queue depth for rdma transport.',
          'metavar': None,
          'mutable': False,
          'name': 'spdk_max_queue_depth',
          'positional': False,
          'required': False,
          'sample_default': None,
          'secret': False,
          'short': None,
          'type': {'choices': None,
                   'max': 128,
                   'min': 1,
                   'num_type': <class 'int'>,
                   'type_class': Integer(min=1, max=128),
                   'type_name': 'integer value'}}
      ],
     }
 },

Stats

In Cinder all cinder-volume services periodically report the stats of their backend to the cinder-scheduler services so they can do informed placing decisions on operations such as volume creation and volume migration.

Some of the keys provided in the stats dictionary include:

• driver_version

• free_capacity_gb

• storage_protocol

• total_capacity_gb

• vendor_name volume_backend_name

Additional information can be found in the Volume Stats section within the Developer’s Documentation.

Gathering stats is a costly operation for many storage backends, so by default the stats method will return cached values instead of collecting them again. If latest data is required parameter refresh=True should be passed in the stats method call.

Here’s an example of the output from the LVM Backend with refresh:

>>> from pprint import pprint
>>> pprint(lvm.stats(refresh=True))
{'driver_version': '3.0.0',
 'pools': [{'QoS_support': False,
            'filter_function': None,
            'free_capacity_gb': 20.9,
            'goodness_function': None,
            'location_info': 'LVMVolumeDriver:router:cinder-volumes:thin:0',
            'max_over_subscription_ratio': 20.0,
            'multiattach': False,
            'pool_name': 'LVM',
            'provisioned_capacity_gb': 0.0,
            'reserved_percentage': 0,
            'thick_provisioning_support': False,
            'thin_provisioning_support': True,
            'total_capacity_gb': '20.90',
            'total_volumes': 1}],
 'sparse_copy_volume': True,
 'storage_protocol': 'iSCSI',
 'vendor_name': 'Open Source',
 'volume_backend_name': 'LVM'}

Available volumes

The Backend class keeps track of all the Backend instances in the backends class attribute, and each Backend instance has a volumes property that will return a list all the existing volumes in the specific backend. Deleted volumes will no longer be present.

So assuming that we have an lvm variable holding an initialized Backend instance where we have created volumes we could list them with:

for vol in lvm.volumes:
    print('Volume %s has %s GB' % (vol.id, vol.size))

Attribute volumes is a lazy loadable property that will only update its value on the first access. More information about lazy loadable properties can be found in the Resource tracking section. For more information on data loading please refer to the Metadata Persistence section.

Note

The volumes property does not query the storage array for a list of existing volumes. It queries the metadata storage to see what volumes have been created using cinderlib and return this list. This means that we won’t be able to manage pre-existing resources from the backend, and we won’t notice when a resource is removed directly on the backend.

Attributes

The Backend class has no attributes of interest besides the backends mentioned above and the id, config, and JSON related properties we’ll see later in the Serialization section.

The id property refers to the volume_backend_name, which is also the key used in the backends class attribute.

The config property will return a dictionary with only the volume backend’s name by default to limit unintended exposure of backend credentials on serialization. If we want it to return all the configuration options we need to pass output_all_backend_info=True on cinderlib initialization.

If we try to access any non-existent attribute in the Backend, cinderlib will understand we are trying to access a Cinder driver attribute and will try to retrieve it from the driver’s instance. This is the case with the initialized property we accessed in the backends listing example.

Other methods

All other methods available in the Backend class will be explained in their relevant sections:

• load and load_backend will be explained together with json, jsons, dump, dumps properties and to_dict method in the Serialization section.

• create_volume method will be covered in the Volumes section.

• validate_connector will be explained in the Connections section.

• global_setup has been covered in the Initialization section.

• pool_names tuple with all the pools available in the driver. Non pool aware drivers will have only 1 pool and use the name of the backend as its name. Pool aware drivers may report multiple values, which can be passed to the create_volume method in the pool_name parameter.