Whenever you deploy VMware Integrated OpenStack there is always a chance that something may go wrong. This post is here to help diagnose and troubleshoot any issue that a user may have during the deployment or during day 2 operations.

The first section we will start with is some handy tips to troubleshoot a new deployment. All of these can be conducted via the root account within the VMware Integrated OpenStack Manager.

If you find that a service is stalling during the deployment of OpenStack we can use the pods command to get a live list of the pods that are booting up and the status of them. You’ll notice in the image below that we have a number of pods all with different status.

Status Legend:

• Running: Service is healthy and running

• PodInitializing: Pod is starting up

• Error: The Pod has encountered and error and is no longer running

• CrashLoopBackOff: The pod has encountered an error and is restarting to try again.

• Completed: Pod has successfully run and completed the task that is required.

The status values that are of use here will your Error & CrashLoopBackOff and we can look into both of these via some simple kubectl commands.

OpenStack & Kubectl

As VIO is a containerised we can run a number of kubectl commands directly from the manager cli. Any deployment that is completed using VIO will have an OpenStack namespace created and through this we can basically see all of the relevant resources for the given deployment including pods, deployments and logs.

Lets say for example we have a pod that is crashing and we want to see the logs of it. We can first use the kubectl -n openstack get pods command to get the pod name. As an example here ill use nova-conductor

We can copy the pod name and then issue kubectl -n openstack logs nova-conductor-7c79f84bcf-8pf6m and it will go ahead and print out the logs for this container. From here we can use filtering commands like grep to filter through the logs and identify any errors.

We can also edit a deployment file which basically serves as an OpenStack service. by using the kubectl get deployment command.

From here we can select a deployment and use the kubectl -n openstack edit deployment <deployment-name> to change the configuration of the service. An example of Nova can be seen below.

VIOCLI

This is not the only way to edit services however. Through the use of viocli we can directly edit services on the fly. Lets say we wanted to update he configuration of neutron. We can do this in a few different ways.

1. GUI: We can login to the VIO LCM GUI and select our OpenStack deployment before going to configure the services

2. viocli update: This serves as a CLI way to edit your configuration by using viocli update neutron we can directly update parameters of our neutron config.

This viocli update command can be used for all of the services within the OpenStack config and example of neutron can be seen below. Once and changes and made and saved within these files the pods will automatically be deployed which makes it incredibly helpful for troubleshooting

High Availability Issues

If your attempting to deploy a high availability cluster and are having issues with services starting up. Ensure that there is no time drift between your nodes.

• viocli get time