devtest_undercloud

  1. Add extra elements for Undercloud UI

    if [ "$USE_UNDERCLOUD_UI" -ne 0 ] ; then
    UNDERCLOUD_DIB_EXTRA_ARGS="$UNDERCLOUD_DIB_EXTRA_ARGS ceilometer-collector \
        ceilometer-api ceilometer-agent-central ceilometer-agent-notification \
        ceilometer-undercloud-config horizon nova-ironic"
fi

  2. Specifiy a client-side timeout in minutes for creating or updating the undercloud Heat stack.

    UNDERCLOUD_STACK_TIMEOUT=${UNDERCLOUD_STACK_TIMEOUT:-60}

  3. Create your undercloud image. This is the image that the seed nova will deploy to become the baremetal undercloud. $UNDERCLOUD_DIB_EXTRA_ARGS is meant to be used to pass additional arguments to disk-image-create.

    NODE_ARCH=$(os-apply-config -m $TE_DATAFILE --key arch --type raw)
    $TRIPLEO_ROOT/diskimage-builder/bin/disk-image-create $NODE_DIST \
        -a $NODE_ARCH -o $TRIPLEO_ROOT/undercloud \
        ntp baremetal boot-stack os-collect-config dhcp-all-interfaces \
        neutron-dhcp-agent $DIB_COMMON_ELEMENTS $UNDERCLOUD_DIB_EXTRA_ARGS 2>&1 | \
        tee $TRIPLEO_ROOT/dib-undercloud.log

  4. If you wanted to build the image and run it elsewhere, you can stop at this point and head onto the overcloud image building.

  5. Load the undercloud image into Glance:

    UNDERCLOUD_ID=$(load-image -d $TRIPLEO_ROOT/undercloud.qcow2)

  6. Set the public interface of the undercloud network node:

    NeutronPublicInterface=${NeutronPublicInterface:-'nic1'}

  7. Set the NTP server for the undercloud::

    UNDERCLOUD_NTP_SERVER=${UNDERCLOUD_NTP_SERVER:-''}

  8. Create secrets for the cloud. The secrets will be written to a file ($TRIPLEO_ROOT/tripleo-undercloud-passwords by default) that you need to source into your shell environment.

    Note

    You can also make or change these later and update the heat stack definition to inject them - as long as you also update the keystone recorded password.

    Note

    There will be a window between updating keystone and instances where they will disagree and service will be down. Instead consider adding a new service account and changing everything across to it, then deleting the old account after the cluster is updated.

    setup-undercloud-passwords $TRIPLEO_ROOT/tripleo-undercloud-passwords
source $TRIPLEO_ROOT/tripleo-undercloud-passwords

  9. Export UNDERCLOUD_CEILOMETER_SNMPD_PASSWORD to your environment so it can be applied to the SNMPd of all Overcloud nodes.

    NEW_JSON=$(jq ‘.undercloud.ceilometer_snmpd_password=”’${UNDERCLOUD_CEILOMETER_SNMPD_PASSWORD}’”’ $TE_DATAFILE) echo $NEW_JSON > $TE_DATAFILE

  10. Pull out needed variables from the test environment definition.

    POWER_MANAGER=$(os-apply-config -m $TE_DATAFILE --key power_manager --type raw)
POWER_KEY=$(os-apply-config -m $TE_DATAFILE --key ssh-key --type raw)
POWER_HOST=$(os-apply-config -m $TE_DATAFILE --key host-ip --type raw)
POWER_USER=$(os-apply-config -m $TE_DATAFILE --key ssh-user --type raw)

  11. Wait for the BM cloud to register BM nodes with the scheduler:

    wait_for -w 60 --delay 1 -- wait_for_hypervisor_stats

  12. We need an environment file to store the parameters we’re going to give heat.:

    HEAT_ENV=${HEAT_ENV:-"${TRIPLEO_ROOT}/undercloud-env.json"}

  13. Read the heat env in for updating.:

    if [ -e "${HEAT_ENV}" ]; then
    ENV_JSON=$(cat "${HEAT_ENV}")
else
    ENV_JSON='{"parameters":{}}'
fi

  14. Detect if we are deploying with a VLAN for API endpoints / floating IPs. This is done by looking for a ‘public’ network in Neutron, and if found we pull out the VLAN id and pass that into Heat, as well as using a VLAN enabled Heat template.

    if (neutron net-list | grep -q public); then
    VLAN_ID=$(neutron net-show public | awk '/provider:segmentation_id/ { print $4 }')
else
    VLAN_ID=
fi

  15. Nova-baremetal and Ironic require different Heat templates and different options.

    if [ -n "$VLAN_ID" ]; then
    HEAT_UNDERCLOUD_TEMPLATE="undercloud-vm-ironic-vlan.yaml"
    ENV_JSON=$(jq .parameters.NeutronPublicInterfaceTag=\"${VLAN_ID}\" <<< $ENV_JSON)
# This should be in the heat template, but see
# https://bugs.launchpad.net/heat/+bug/1336656
# note that this will break if there are more than one subnet, as if
# more reason to fix the bug is needed :).
PUBLIC_SUBNET_ID=$(neutron net-show public | awk '/subnets/ { print $4 }')
VLAN_GW=$(neutron subnet-show $PUBLIC_SUBNET_ID | awk '/gateway_ip/ { print $4}')
BM_VLAN_CIDR=$(neutron subnet-show $PUBLIC_SUBNET_ID | awk '/cidr/ { print $4}')
    ENV_JSON=$(jq .parameters.NeutronPublicInterfaceDefaultRoute=\"${VLAN_GW}\" <<< $ENV_JSON)
else
    HEAT_UNDERCLOUD_TEMPLATE="undercloud-vm-ironic.yaml"
fi
ENV_JSON=$(jq .parameters.IronicPassword=\"${UNDERCLOUD_IRONIC_PASSWORD}\" <<< $ENV_JSON)
REGISTER_SERVICE_OPTS="--ironic-password $UNDERCLOUD_IRONIC_PASSWORD"

STACKNAME_UNDERCLOUD=${STACKNAME_UNDERCLOUD:-'undercloud'}

  16. Choose whether to deploy or update. Use stack-update to update:: HEAT_OP=stack-create

    if heat stack-show $STACKNAME_UNDERCLOUD > /dev/null; then
    HEAT_OP=stack-update
    if (heat stack-show $STACKNAME_UNDERCLOUD | grep -q FAILED); then
        echo "Updating a failed stack. this is a new ability and may cause problems." >&2
    fi
else
    HEAT_OP=stack-create
fi

  17. Set parameters we need to deploy a baremetal undercloud:

    ENV_JSON=$(jq '.parameters = {
"MysqlInnodbBufferPoolSize": 100
} + .parameters + {
"AdminPassword": "'"${UNDERCLOUD_ADMIN_PASSWORD}"'",
"AdminToken": "'"${UNDERCLOUD_ADMIN_TOKEN}"'",
"SnmpdReadonlyUserPassword": "'"${UNDERCLOUD_CEILOMETER_SNMPD_PASSWORD}"'",
"GlancePassword": "'"${UNDERCLOUD_GLANCE_PASSWORD}"'",
"HeatPassword": "'"${UNDERCLOUD_HEAT_PASSWORD}"'",
"NovaPassword": "'"${UNDERCLOUD_NOVA_PASSWORD}"'",
"NeutronPassword": "'"${UNDERCLOUD_NEUTRON_PASSWORD}"'",
"NeutronPublicInterface": "'"${NeutronPublicInterface}"'",
"undercloudImage": "'"${UNDERCLOUD_ID}"'",
"BaremetalArch": "'"${NODE_ARCH}"'",
"PowerSSHPrivateKey": "'"${POWER_KEY}"'",
"NtpServer": "'"${UNDERCLOUD_NTP_SERVER}"'",
"Flavor": "'"${FLAVOR}"'"
}' <<< $ENV_JSON)



#Add Ceilometer to env only if USE_UNDERCLOUD_UI is specified

if [ "$USE_UNDERCLOUD_UI" -ne 0 ] ; then
    ENV_JSON=$(jq '.parameters = .parameters + {
    "CeilometerPassword": "'"${UNDERCLOUD_CEILOMETER_PASSWORD}"'"
    }' <<< $ENV_JSON)
fi

  18. Save the finished environment file.:

    jq . > "${HEAT_ENV}" <<< $ENV_JSON
chmod 0600 "${HEAT_ENV}"

  19. Add Keystone certs/key into the environment file.:

    generate-keystone-pki --heatenv $HEAT_ENV

  20. Deploy an undercloud.

    make -C $TRIPLEO_ROOT/tripleo-heat-templates $HEAT_UNDERCLOUD_TEMPLATE

heat $HEAT_OP -e $HEAT_ENV \
    -t 360 \
    -f $TRIPLEO_ROOT/tripleo-heat-templates/$HEAT_UNDERCLOUD_TEMPLATE \
    $STACKNAME_UNDERCLOUD

    You can watch the console via virsh/virt-manager to observe the PXE boot/deploy process. After the deploy is complete, it will reboot into the image.

  21. Get the undercloud IP from nova list

    # Make time out 60 mins as like the Heat stack-create default timeout.
wait_for_stack_ready -w $(($UNDERCLOUD_STACK_TIMEOUT * 60 )) 10 undercloud
UNDERCLOUD_CTL_IP=$(nova list | grep ctlplane | sed  -e "s/.*=\\([0-9.]*\\).*/\1/")

  22. If we’re deploying with a public VLAN we must use it, not the control plane network (which we may not even have access to) to ping and configure thing.

    if [ -n "$VLAN_ID" ]; then
    UNDERCLOUD_IP=$(heat output-show undercloud PublicIP|sed 's/^"\(.*\)"$/\1/')
else
    UNDERCLOUD_IP=$UNDERCLOUD_CTL_IP
fi

  23. We don’t (yet) preserve ssh keys on rebuilds.

    ssh-keygen -R $UNDERCLOUD_IP
ssh-keygen -R $UNDERCLOUD_CTL_IP

  24. Exclude the undercloud from proxies:

    export no_proxy=$no_proxy,$UNDERCLOUD_IP

  25. Export the undercloud endpoint and credentials to your test environment.

    UNDERCLOUD_ENDPOINT="http://$UNDERCLOUD_IP:5000/v2.0"
NEW_JSON=$(jq '.undercloud.password="'${UNDERCLOUD_ADMIN_PASSWORD}'" | .undercloud.endpoint="'${UNDERCLOUD_ENDPOINT}'" | .undercloud.endpointhost="'${UNDERCLOUD_IP}'"' $TE_DATAFILE)
echo $NEW_JSON > $TE_DATAFILE

  26. Source the undercloud configuration:

    source $TRIPLEO_ROOT/tripleo-incubator/undercloudrc

  27. Perform setup of your undercloud.

    init-keystone -o $UNDERCLOUD_CTL_IP -t $UNDERCLOUD_ADMIN_TOKEN \
    -e admin@example.com -p $UNDERCLOUD_ADMIN_PASSWORD \
    --public $UNDERCLOUD_IP --no-pki-setup

# Creating these roles to be used by tenants using swift
openstack role create swiftoperator
openstack role create ResellerAdmin


# Create service endpoints and optionally include Ceilometer for UI support
ENDPOINT_LIST="--glance-password $UNDERCLOUD_GLANCE_PASSWORD
    --heat-password $UNDERCLOUD_HEAT_PASSWORD
    --neutron-password $UNDERCLOUD_NEUTRON_PASSWORD
    --nova-password $UNDERCLOUD_NOVA_PASSWORD
    --tuskar-password $UNDERCLOUD_TUSKAR_PASSWORD"

if [ "$USE_UNDERCLOUD_UI" -ne 0 ] ; then
    ENDPOINT_LIST="$ENDPOINT_LIST --ceilometer-password $UNDERCLOUD_CEILOMETER_PASSWORD"
fi

setup-endpoints $UNDERCLOUD_CTL_IP $ENDPOINT_LIST $REGISTER_SERVICE_OPTS \
    --public $UNDERCLOUD_IP
openstack role create heat_stack_user

user-config

BM_NETWORK_CIDR=$(os-apply-config -m $TE_DATAFILE --key baremetal-network.cidr --type raw --key-default '192.0.2.0/24')
if [ -n "$VLAN_ID" ]; then
    # No ctl plane gateway - public net gateway is needed.
    # XXX (lifeless) - Neutron still configures one, first position in the subnet.
    BM_NETWORK_GATEWAY=
else
    # Use a control plane gateway.
    BM_NETWORK_GATEWAY=$(os-apply-config -m $TE_DATAFILE --key baremetal-network.gateway-ip --type raw --key-default '192.0.2.1')
fi
BM_NETWORK_UNDERCLOUD_RANGE_START=$(os-apply-config -m $TE_DATAFILE --key baremetal-network.undercloud.range-start --type raw --key-default '192.0.2.21')
BM_NETWORK_UNDERCLOUD_RANGE_END=$(os-apply-config -m $TE_DATAFILE --key baremetal-network.undercloud.range-end --type raw --key-default '192.0.2.40')

UNDERCLOUD_NAMESERVER=$(os-apply-config -m $TE_DATAFILE --key undercloud.nameserver --type netaddress --key-default "${UNDERCLOUD_NAMESERVER:-}")

NETWORK_JSON=$(mktemp)
jq "." <<EOF > $NETWORK_JSON
{
    "physical": {
        "gateway": "$BM_NETWORK_GATEWAY",
        "metadata_server": "$UNDERCLOUD_CTL_IP",
        "cidr": "$BM_NETWORK_CIDR",
        "allocation_start": "$BM_NETWORK_UNDERCLOUD_RANGE_START",
        "allocation_end": "$BM_NETWORK_UNDERCLOUD_RANGE_END",
        "name": "ctlplane",
        "nameserver": "$UNDERCLOUD_NAMESERVER"
    }
}
EOF
setup-neutron -n $NETWORK_JSON
rm $NETWORK_JSON

if [ -n "$VLAN_ID" ]; then
    BM_VLAN_START=$(jq -r '.["baremetal-network"].undercloud.public_vlan.start' $TE_DATAFILE)
    BM_VLAN_END=$(jq -r '.["baremetal-network"].undercloud.public_vlan.finish' $TE_DATAFILE)
    PUBLIC_NETWORK_JSON=$(mktemp)
    jq "." <<EOF > $PUBLIC_NETWORK_JSON
{
    "physical": {
        "gateway": "$VLAN_GW",
        "metadata_server": "$UNDERCLOUD_CTL_IP",
        "cidr": "$BM_VLAN_CIDR",
        "allocation_start": "$BM_VLAN_START",
        "allocation_end": "$BM_VLAN_END",
        "name": "public",
        "nameserver": "$UNDERCLOUD_NAMESERVER",
        "segmentation_id": "$VLAN_ID",
        "physical_network": "ctlplane",
        "enable_dhcp": false
    }
}
EOF
    setup-neutron -n $PUBLIC_NETWORK_JSON
fi

  28. Nova quota runs up with the defaults quota so overide the default to allow unlimited cores, instances and ram.

    nova quota-update --cores -1 --instances -1 --ram -1 $(openstack project show admin | awk '$2=="id" {print $4}')

  29. Register two baremetal nodes with your undercloud.

    setup-baremetal --service-host undercloud --nodes <(jq '.nodes - [.nodes[0]]' $TE_DATAFILE)