Manage Unified Limits Quotas

Note

This section provides deployment information about the quota feature. For end-user information about quotas, including information about the type of quotas available, refer to the user guide.

Since the Nova 28.0.0 (2023.2 Bobcat) release, it is recommended to use Keystone unified limits for Nova quota limits.

For information about legacy quota limits, see the legacy quota documentation.

Quotas

To prevent system capacities from being exhausted without notification, you can set up quotas. Quotas are operational limits. The number of servers allowed for each project can be controlled so that cloud resources are optimized, for example. Quotas can be enforced at both the global (default) level and at the project level.

Unified limits

Unified limits is a modern quota system in which quota limits are centralized in the Keystone identity service. There are three steps for quota enforcement in this model:

  1. Quota limits are retrieved by calling the Keystone unified limits API

  2. Quota usage is counted from the Placement API service

  3. Quota is enforced locally using the oslo.limit limit enforcement library

In unified limits, the terminology is a bit different from legacy quotas:

  • A registered limit is a global or default limit that applies to all projects

  • A limit is a project-scoped limit that applies to a particular project

Cloud operators will need to manage their quota limits in the Keystone service by calling the API directly or by using the OpenStackClient (OSC) registered limit and limit commands.

Roles

By default Keystone API policy, a user must have the following roles and scopes in order to perform actions with unified limits.

Action

Role

Scope

List registered limits

*

*

Get registered limit

*

*

Create registered limit

admin

system=all

Update registered limit

admin

system=all

Delete registered limit

admin

system=all

List limits

*

*

Get limit

*

*

Create limit

admin

system=all

Update limit

admin

system=all

Delete limit

admin

system=all

Configuration

To enable unified limits quotas, some Nova configuration of the nova-api and nova-conductor services is necessary.

Set the quota driver to the nova.quota.UnifiedLimitsDriver:

[quota]
driver = nova.quota.UnifiedLimitsDriver

Add a configuration section for oslo.limit:

[oslo_limit]
username = nova
user_domain_name = $SERVICE_DOMAIN_NAME
auth_url = $KEYSTONE_SERVICE_URI
auth_type = password
password = $SERVICE_PASSWORD
system_scope = all
endpoint_id = $SERVICE_ENDPOINT_ID

Note

The Nova service endpoint ID can be obtained by openstack endpoint list --service nova -f value -c ID

Ensure that the nova service user has the reader role with system scope:

openstack role add --user nova --user-domain $SERVICE_DOMAIN_NAME \
   --system all reader

Setting quota limits on resources

Any resource that can be requested in the cloud must have a registered limit set. Quota checks on cloud resources that do not have registered limits will continue to fail until registered limits are set because oslo.limit considers an unregistered resource to have a limit of 0.

Types of quota

Unified limit resource names for resources that are tracked as resource classes in the Placement API service follow the naming pattern of the class: prefix followed by the name of the resource class. For example: class:VCPU, class:PCPU, class:MEMORY_MB, class:DISK_GB, class:VGPU.

Quota Name

Description

class:VCPU

Number of shared CPU cores (VCPUs) allowed per project

class:PCPU

Number of dedicated CPU cores (PCPUs) allowed per project

servers

Number of instances allowed per project

server_key_pairs

Number of key pairs allowed per user

server_metadata_items

Number of metadata items allowed per instance

class:MEMORY_MB

Megabytes of instance ram allowed per project

server_groups

Number of server groups per project

server_group_members

Number of servers per server group

class:DISK_GB

Gigabytes of instance disk allowed per project

class:$RESOURCE_CLASS

Any resource class in the Placement API service can have a quota limit specified for it (example: class:VGPU)

OpenStack CLI commands

For full OpenStackClient documentation, see https://docs.openstack.org/python-openstackclient/latest/index.html.

Registered Limits

To list default limits for Nova:

openstack registered limit list --service nova

To show details about a default limit:

openstack registered limit show $REGISTERED_LIMIT_ID

To create a default limit:

openstack registered limit create --service nova --default-limit $LIMIT \
   $RESOURCE

To update a default limit:

openstack registered limit set --default-limit $LIMIT $REGISTERED_LIMIT_ID

To delete a default limit:

openstack registered limit delete $REGISTERED_LIMIT_ID

Limits

To list project limits for Nova:

openstack limit list --service nova

To list limits for a particular project:

openstack limit list --service nova --project $PROJECT_ID

To show details about a project limit:

openstack limit show $LIMIT_ID

To create a project limit:

openstack limit create --service nova --project $PROJECT_ID \
   --resource-limit $LIMIT $RESOURCE

To update a project limit:

openstack limit set --resource-limit $LIMIT $LIMIT_ID

To delete a project limit:

openstack limit delete $LIMIT_ID

Quota enforcement

When enforcing limits for a given resource and project, the following checks are made in order:

  1. Limits (project-specific)

    Depending on the resource, is there a project-specific limit on the resource in Keystone limits? If so, use that as the limit. If not, proceed to check the registered default limit.

  2. Registered limits (default)

    Depending on the resource, is there a default limit on the resource in Keystone limits? If so, use that as the limit. If not, oslo.limit will consider the limit as 0, the quota check will fail, and a quota limit exceeded exception will be raised.

Warning

Every resource that can be requested in the cloud must at a minimum have a registered limit set. Any resource that does not have a registered limit set will fail quota enforcement because oslo.limit considers an unregistered resource to have a limit of 0.

Rechecking quota

If quota.recheck_quota = True (this is the default), Nova will perform a second quota check after allocating resources. The first quota check is performed before resources are allocated. Rechecking quota ensures that quota limits are strictly enforced and prevents any possibility of resource allocation going over the quota limit in the event of racing parallel API requests.

It can be disabled by setting quota.recheck_quota = False if strict quota enforcement is not important to the operator.

Quota usage from Placement

With unified limits quotas, it is required that quota resource usage is counted from the Placement API service. As such, the quota.count_usage_from_placement configuration option is ignored when quota.driver is set to nova.quota.UnifiedLimitsDriver.

There are some things to note when quota resource usage is counted from the Placement API service:

  • Counted usage will not be accurate in an environment where multiple Nova deployments are sharing a Placement deployment because currently Placement has no way of partitioning resource providers between different Nova deployments. Operators who are running multiple Nova deployments that share a Placement deployment should not use the nova.quota.UnifiedLimitsDriver.

  • Behavior will be different for resizes. During a resize, resource allocations are held on both the source and destination (even on the same host, see https://bugs.launchpad.net/nova/+bug/1790204) until the resize is confirmed or reverted. Quota usage will be inflated for servers in this state.

  • The populate_queued_for_delete and populate_user_id online data migrations must be completed before usage can be counted from Placement. Until the data migration is complete, the system will fall back to legacy quota usage counting from cell databases depending on the result of an EXISTS database query during each quota check. Use nova-manage db online_data_migrations to run online data migrations.

  • Behavior will be different for unscheduled servers in ERROR state. A server in ERROR state that has never been scheduled to a compute host will not have Placement allocations, so it will not consume quota usage for cores and ram.

  • Behavior will be different for servers in SHELVED_OFFLOADED state. A server in SHELVED_OFFLOADED state will not have Placement allocations, so it will not consume quota usage for cores and ram. Note that because of this, it will be possible for a request to unshelve a server to be rejected if the user does not have enough quota available to support the cores and ram needed by the server to be unshelved.

Migration to unified limits quotas

There is a nova-manage command available to help with moving from legacy Nova database quotas to Keystone unified limits quotas. The command will read quota limits from the Nova database and call the Keystone API to create the corresponding unified limits.

$ nova-manage limits migrate_to_unified_limits -h
usage: nova-manage limits migrate_to_unified_limits
[-h] [--project-id <project-id>] [--region-id <region-id>] [--verbose]
[--dry-run]

Copy quota limits from the Nova API database to Keystone.

options:
  -h, --help            show this help message and exit
  --project-id <project-id>
                        Project ID for which to migrate quota limits
  --region-id <region-id>
                        Region ID for which to migrate quota limits
  --verbose             Provide verbose output during execution.
  --dry-run             Show what limits would be created without actually
                        creating them.

Important

Per-user quota limits will not be copied into Keystone because per-user quotas are not supported in unified limits.

Require or ignore resources

The quota.unified_limits_resource_strategy and quota.unified_limits_resource_list configuration options are available for operators to specify which cloud resources they will require to have registered limits set in Keystone. The default strategy is require and the default resource list contains the servers resource.

When unified_limits_resource_strategy = require, if a resource in unified_limits_resource_list is requested and has no registered limit set, the quota limit for that resource will be considered to be 0 and all requests to allocate that resource will be rejected for being over quota. Any resource not in the list will be considered to have unlimited quota.

When unified_limits_resource_strategy = ignore, if a resource in unified_limits_resource_list is requested and has no registered limit set, the quota limit for that resource will be considered to be unlimited and all requests to allocate that resource will be accepted. Any resource not in the list will be considered to have 0 quota.

The options should be configured for the nova-api and nova-conductor services. The nova-conductor service performs quota enforcement when quota.recheck_quota is True (the default).

The unified_limits_resource_list list can also be set to an empty list.

Example configuration values:

[quota]
unified_limits_resource_strategy = require
unified_limits_resource_list = servers,class:VCPU,class:MEMORY_MB,class:DISK_GB