Deploy Software Releases using the CLI

After a software release has been uploaded to the central repository, it can be deployed first on the System Controller and then on a specific subcloud or across a group of subclouds through different orchestration methods.

Deploy a Software Release on the System Controller

Follow the instructions at Orchestrated Deployment Host Software Deployment to complete the software deployment.

Note

The step to upload the software release can be skipped since it has already been done.

It is recommended to upgrade Kubernetes following the deployment of a major release. See Kubernetes Version Upgrade Cloud Orchestration Overview.

The System Controller is now ready for new subcloud deployment.

Note

Starting release r9.0, low-latency configuration via remote install

is no longer supported.

For new subcloud deployments that involve remote install using Redfish,

avoid using install_type of 4 and 5 in install-values.yaml.

For existing subclouds that were originally deployed with remote install,

use dcmanager subcloud update --install-values to update the install_type so they can be restored in the future.

Orchestrate the Deployment of a Software Release Across the Subclouds

After the System Controller has been successfully updated with a new major or patch software release; distributed software deployment orchestration can be initiated to bring the managed subclouds to the same software level as the System Controller. The subclouds must, however, be prestaged with the target software before orchestrated deployment can take place.

For more information on Prestaging Subcloud Orchestration see Orchestrate Subcloud Prestage Using the CLI.

Before orchestrating a major software release deployment across the subclouds, it is strongly recommended that a backup of each subcloud to be upgraded is taken. See Backup a Subcloud/Group of Subclouds using DCManager CLI for instructions.

Prerequisites

The following conditions must be met for the orchestrated software deployment across the subclouds to be successful.

  • All target subclouds are managed, online and healthy. The Kubernetes cluster on each subcloud is fully operational and there are no management affecting alarms.

  • Subcloud certificates are up-to-date or will not expire during the deployment of the new software release.

  • All target subclouds have been prestaged with the software release and the new container images required for that release. See Orchestrate Subcloud Prestage Using the CLI for instructions.

  • For the deployment of major software release, ensure that the controller-0 is the active controller on each subcloud.

  • Optional step: Only for Major Release Upload the new software release to the central software release repository:

    ~(keystone_admin)]$ software --os-region-name SystemController upload <bootimage.iso> <bootimage.sig>

Procedure

  1. Review software sync status of the subclouds.

    After new software release has been deployed on the System Controller, wait for approximately 60 seconds for the software_sync_status of all subclouds to be updated.

    To identify which subclouds are deploy-current (in-sync), use the dcmanager subcloud list command. For example:

    ~(keystone_admin)]$ dcmanager subcloud list
+----+-----------+--------------+--------------------+-------------+
| id | name      | management   | availability       | sync        |
+----+-----------+--------------+--------------------+-------------+
|  1 | subcloud-1| managed      | online             | out-of-sync |
|  2 | subcloud-2| managed      | online             | out-of-sync |
|  3 | subcloud-3| managed      | online             | out-of-sync |
|  4 | subcloud-4| managed      | online             | out-of-sync |
+----+-----------+--------------+--------------------+-------------+

    To see synchronization details for a subcloud, use the following command:

    ~(keystone_admin)]$ dcmanager subcloud show subcloud1
+-----------------------------+----------------------------------+
| Field                       | Value                            |
+-----------------------------+----------------------------------+
| id                          | 2                                |
| name                        | subcloud1-stx-latest             |
| description                 | Ottawa Site                      |
| location                    | YOW                              |
| software_version            | 24.09                            |
| management                  | unmanaged                        |
| availability                | online                           |
| deploy_status               | complete                         |
| management_subnet           | 192.168.101.0/24                 |
| management_start_ip         | 192.168.101.2                    |
| management_end_ip           | 192.168.101.50                   |
| management_gateway_ip       | 192.168.101.1                    |
| systemcontroller_gateway_ip | 192.168.204.101                  |
| group_id                    | 1                                |
| peer_group_id               | None                             |
| created_at                  | 2025-01-21 18:34:30.410350       |
| updated_at                  | 2025-01-21 19:03:08.836619       |
| backup_status               | None                             |
| backup_datetime             | None                             |
| prestage_status             | None                             |
| prestage_versions           | None                             |
| dc-cert_sync_status         | in-sync                          |
| firmware_sync_status        | unknown                          |
| identity_sync_status        | unknown                          |
| kubernetes_sync_status      | unknown                          |
| kube-rootca_sync_status     | unknown                          |
| load_sync_status            | unknown                          |
| patching_sync_status        | unknown                          |
| platform_sync_status        | unknown                          |
| software_sync_status        | unknown                          |
| region_name                 | 0aa48e9b72bf48bbaad9a624cdd0cacb |
+-----------------------------+----------------------------------+

  2. To create a sw-deploy strategy, use the dcmanager sw-deploy-strategy create <release_id> command.

    The sw-deploy strategy for a Distributed Cloud system controls how updates/upgrades are applied to subclouds.

    ~(keystone_admin)]$ dcmanager sw-deploy-strategy create \
[--subcloud-apply-type <type>] \
[–-max-parallel-subclouds <i>] \
[–-stop-on-failure <level>] \
[--group group] \
[<subcloud>]

    where:

    subcloud-apply-type

    parallel or serial— determines whether the subclouds will be processed in parallel or serially.

    If this is not specified using the CLI, the values for subcloud_update_type defined for each subcloud group will be used by default.

    max-parallel-subclouds

    Sets the maximum number of subclouds that can be upgraded in parallel (default 2).

    If this is not specified using the CLI, the values for max_parallel_subclouds defined for each subcloud group will be used by default.

    stop-on-failure

    true(default) or false— determines whether upgrade orchestration failure for a subcloud prevents application to subsequent subclouds.

    group

    Optionally pass the name or ID of a subcloud group to the dcmanager sw-deploy-strategy create command. This results in a strategy that is only applied to all subclouds in the specified group. The subcloud group values are used for subcloud apply type and max parallel subclouds parameters.

    For example:

    ~(keystone_admin)]$ dcmanager sw-deploy-strategy create
+------------------------+----------------------------+
| Field                  | Value                      |
+------------------------+----------------------------+
| strategy type          | sw-deploy                  |
| subcloud apply type    | None                       |
| max parallel subclouds | 2                          |
| stop on failure        | False                      |
| release_id             | WRCP-24.09.200             |
| state                  | initial                    |
| created_at             | 2025-01-21T20:00:31.141872 |
| updated_at             | None                       |
+------------------------+----------------------------+

  3. To show the settings for the sw-deploy-strategy, use the dcmanager sw-deploy-strategy show command.

    For example:

    ~(keystone_admin)]$ dcmanager sw-deploy-strategy show
+------------------------+----------------------------+
| Field                  | Value                      |
+------------------------+----------------------------+
| strategy type          | sw-deploy                  |
| subcloud apply type    | parallel                   |
| max parallel subclouds | 2                          |
| stop on failure        | False                      |
| state                  | initial                    |
| created_at             | 2020-02-02T14:42:13.822499 |
| updated_at             | None                       |
+------------------------+----------------------------+

    Note

    The values for subcloud apply type and max parallel subclouds will be taken from the subcloud group if specified through the --group parameter.

  4. Review the sw-deploy strategy for the subclouds.

    To show the subclouds that will be processed when the sw-deploy strategy is applied, use the dcmanager strategy-step list command. For example:

    ~(keystone_admin)]$ dcmanager strategy-step list
+----------------------+-------+---------+---------+------------+-------------+
| cloud                | stage | state   | details | started_at | finished_at |
+----------------------+-------+---------+---------+------------+-------------+
| subcloud1-stx-latest |     1 | initial |         | None       | None        |
+----------------------+-------+---------+---------+------------+-------------+

    Note

    All the subclouds that are included in the same stage will be deployed in parallel.

  5. To apply the software deploy strategy, use the dcmanager sw-deploy-strategy apply command.

    ~(keystone_admin)]$ dcmanager sw-deploy-strategy apply
+------------------------+----------------------------+
| Field                  | Value                      |
+------------------------+----------------------------+
| subcloud apply type    | parallel                   |
| max parallel subclouds | 2                          |
| stop on failure        | False                      |
| state                  | applying                   |
| created_at             | 2020-02-02T14:42:13.822499 |
| updated_at             | 2020-02-02T14:42:19.376688 |
+------------------------+----------------------------+

  6. To show the step currently being performed on each of the subclouds, use the dcmanager strategy-step list command.

    For example:

    ~(keystone_admin)]$ dcmanager strategy-step list
+------------------+--------+-----------------------+-------------------+----------------------------+----------------------------+
| cloud            | stage  | state                 | details           | started_at                 | finished_at                |
+------------------+--------+-----------------------+-------------------+----------------------------+----------------------------+
| subcloud-1       | 3      | complete              |                   | 2024-11-06 12:57:20.342429 | 2024-11-06 12:57:41.054664 |
| subcloud-4       | 2      | apply VIM sw-deploy   |                   | 2024-11-06 12:57:20.342429 | None                       |
| subcloud-5       | 1      | initial               |                   | None                       | None                       |
| subcloud-6       | 1      | initial               |                   | None                       | None                       |
+------------------+--------+-------------+-----------------------------+----------------------------+----------------------------+

  7. To show the step currently being performed on a subcloud, use the dcmanager strategy-step show <subcloud> command.

    ~(keystone_admin)]$ dcmanager strategy-step show <subcloud>

  8. When all the subclouds within the distributed software deploy orchestration indicate they have entered either the complete or failed state, delete the sw-deploy strategy, using the dcmanager sw-deploy-strategy delete command.

    ~(keystone_admin)]$ dcmanager sw-deploy-strategy delete
+------------------------+----------------------------+
| Field                  | Value                      |
+------------------------+----------------------------+
| subcloud apply type    | parallel                   |
| max parallel subclouds | 2                          |
| stop on failure        | False                      |
| state                  | deleting                   |
| created_at             | 2020-03-23T20:04:50.992444 |
| updated_at             | 2020-03-23T20:05:14.157352 |
+------------------------+----------------------------+

Postrequisites

If the subclouds are upgraded with a major software release, it is recommended to also upversion Kubernetes right after the software deployment. See Kubernetes Upversion Orchestration.

Error Handling During an Orchestrated Subcloud Software Deployment

If a failure occurs, follow the general steps below:

  1. Allow the failed strategy to complete on its own.

  2. Check the output using dcmanager strategy-step list command.

  3. To see the cause of failure for a subcloud, use the dcmanager subcloud errors <subcloud-name/subcloud-id> command.

  4. Resolve the condition that caused the failure.

  5. Retry the orchestration by deleting the failed strategy and create a new one for the failed subcloud(s).