Orchestrate Subcloud Prestage Using the CLI

This section describes the prestage strategy for a single subcloud, all subclouds or a specific subcloud group.

Prerequisites

There are two types of subcloud prestage:

• Prestage for software deployment occurs when the subcloud is running an older software than the system controller during prestaging. This prestage transfers software update(s) and container images to the subcloud while preparing for new software deployment. The prestaged data is stored in various locations on the subcloud. For example, metadata is stored in /opt/software/<sw-version> and new images are added to the local docker registry.

• Prestage for install occurs when the subcloud is running the same or older software than the system controller during prestaging. This prestage transfers the entire platform software and container images of the specified release while preparing for the subcloud redeployment. This is used if the subcloud needs to be reinstalled and restored to a specific release. The prestaged data is stored in the subcloud persistent filesystem /opt/platform-backup/<sw-version>.

For more information on prerequisites for prestage upgrade and reinstall, see Prestage a Subcloud.

Note

Any existing strategy must be deleted first as only one type of strategy can exist at a time.

Procedure

1. Create a prestage strategy.

   Prestage strategy can be created for a single subcloud, all subclouds, or a specific subcloud group.

   To create a prestage strategy for a specific subcloud, use the following command:

   ~(keystone_admin)]$ dcmanager prestage-strategy create --for-install --release nn.nn subcloud7
   Enter the sysadmin password for the subcloud:
   Re-enter sysadmin password to confirm:
   +--------------------------+-----------------------------+
   | Field                    | Value                       |
   +--------------------------+-----------------------------+
   | strategy type            | prestage                    |
   | subcloud apply type      | None                        |
   | max parallel subclouds   | 2                           |
   | stop on failure          | False                       |
   | prestage software version| 22.12                       |
   | state                    | initial                     |
   | created_at               | 2024-10-25T05:37:54.607307  |
   | updated_at               | None                        |
   +--------------------------+-----------------------------+
   

   Note

   --release nn.nn specifies only the major release version.

   To create a prestage strategy for all subclouds, use the following command:

   ~(keystone_admin)]$ dcmanager prestage-strategy create --for-sw-deploy
   Enter the sysadmin password for the subcloud:
   Re-enter sysadmin password to confirm:
   +--------------------------+-----------------------------+
   | Field                    | Value                       |
   +--------------------------+-----------------------------+
   | strategy type            | prestage                    |
   | subcloud apply type      | None                        |
   | max parallel subclouds   | 50                          |
   | stop on failure          | False                       |
   | prestage software version| 24.09                       |
   | state                    | initial                     |
   | created_at               | 2024-10-25T05:37:54.607307  |
   | updated_at               | None                        |
   +--------------------------+-----------------------------+
   

   Note

   If the --release option is not specified, the release version of the system controller will be used.

   To create a prestage strategy for a specific subcloud group, use the following command:

   ~(keystone_admin)]$ dcmanager prestage-strategy create --group First_10_Subclouds
   Enter the sysadmin password for the subcloud:
   Re-enter sysadmin password to confirm:
   +------------------------+-----------------------------+
   | Field                  | Value                       |
   +------------------------+-----------------------------+
   | strategy type          | prestage                    |
   | subcloud apply type    | parallel                    |
   | max parallel subclouds | 10                          |
   | stop on failure        | False                       |
   | state                  | initial                     |
   | created_at             | 2202-03-22T18:54:45.037336  |
   | updated_at             | None                        |
   +------------------------+-----------------------------+
   

   Note

   If the prestage type is not specified, the default prestage type (--for-install) will be applied.

   Note

   Unlike other types of orchestration, prestage orchestration requires sysadmin password as all communications with the subclouds are done using ansible over the oam network to avoid disruptions to management traffic.

2. Apply the strategy.

   ~(keystone_admin)]$ dcmanager prestage-strategy apply
   
   +------------------------+-----------------------------+
   | Field                  | Value                       |
   +------------------------+-----------------------------+
   | strategy type          | prestage                    |
   | subcloud apply type    | None                        |
   | max parallel subclouds | None                        |
   | stop on failure        | False                       |
   | state                  | applying                    |
   | created_at             | 2202-03-22T18:33:20:100712  |
   | updated_at             | 2202-03-22T18:36:03.895542  |
   +------------------------+-----------------------------+
   

3. Monitor the progress of the strategy.

   • The overall status of the strategy can be viewed with the following command:

     ~(keystone_admin)]$ dcmanager prestage-strategy show
     +---------------------------+----------------------------+
     | Field                     | Value                      |
     +---------------------------+----------------------------+
     | strategy type             | prestage                   |
     | subcloud apply type       | None                       |
     | max parallel subclouds    | 2                          |
     | stop on failure           | False                      |
     | prestage software version | 24.09                      |
     | state                     | complete                   |
     | created_at                | 2025-01-21 20:27:05.649975 |
     | updated_at                | 2025-01-21 20:30:04.113626 |
     +---------------------------+----------------------------+
     

   • To view the progress of each strategy step, use the following command:

     ~(keystone_admin)]$ dcmanager strategy-step list
     
     +-----------+-------+---------------------+---------+----------------------------+-------------+
     | cloud     | stage | state               | details | started_at                 | finished_at |
     +-----------+-------+---------------------+---------+----------------------------+-------------+
     | subcloud1 |   1   | prestaging-packages |         | 2202-03-22 18:55:11.523970 | None        |
     +-----------+-------+---------------------+---------+----------------------------+-------------+
     

4. (Optional) Abort the strategy, if required.

   The abort command can be used to abort the prestage orchestration strategy after the current step of the currently applying state is completed.

   ~(keystone_admin)]$ dcmanager prestage-strategy abort
   +---------------------------+----------------------------+
   | Field                     | Value                      |
   +---------------------------+----------------------------+
   | strategy type             | prestage                   |
   | subcloud apply type       | None                       |
   | max parallel subclouds    | 2                          |
   | stop on failure           | False                      |
   | prestage software version | 24.09                      |
   | state                     | abort requested            |
   | created_at                | 2025-01-21T20:27:05.649975 |
   | updated_at                | 2025-01-21T20:27:31.917127 |
   +---------------------------+----------------------------+
   

5. Delete the strategy.

   Once the strategy status is either “complete”, “aborted” or “failed”, the strategy can be deleted with the following command:

   ~(keystone_admin)]$ dcmanager prestage-strategy delete
   
   +------------------------+-----------------------------+
   | Field                  | Value                       |
   +------------------------+-----------------------------+
   | strategy type          | prestage                    |
   | subcloud apply type    | None                        |
   | max parallel subclouds | None                        |
   | stop on failure        | False                       |
   | state                  | deleting                    |
   | created_at             | 2202-03-22T19:09:03.576053  |
   | updated_at             | 2202-03-22T19:09:09.436732  |
   +------------------------+-----------------------------+
   

Troubleshoot Subcloud Prestage Orchestration

If an orchestrated prestage fails for a subcloud, check the log specified in the error message for reasons of failure. After the issue has been resolved, prestage can be retried using one of the following methods:

• Prestage each failed subcloud individually.

  Use the dcmanager subcloud prestage command on the failed subclouds. See Prestage a Subcloud for more information.

• Use prestage orchestration to prestage all failed subclouds.

  Create a subcloud group, for example, prestage-retry, add the failed subcloud(s) to group prestage-retry, and finally create and apply the prestage strategy to the group, following the procedure described on this page.

  Warning

  Do not retry orchestration with an existing group unless the subclouds that have been successfully prestaged are removed from the group. Otherwise, prestage will be repeated for ALL subclouds in the group.