Integrating vSphere and NSX API’s with Python (2/2)

This post will focus on the last tasks mentioned in my Disaster Recovery scenario:

  • Recover VMs at the simulated DR site.
  • Connect VMs to the appropriate Logical Switch.
  • Boot VMs and test connectivity.
  • Check route updates on the physical network.

Before going further you may want to check how to perform the first tasks involved in this scenario. This is here.

The previous script must be executed for the next steps to work properly. I didn’t really include any error catching, my focus being around how to work with pyVmomi and also deal with XML parsing when interpreting NSX API responses. I may consider that for future improvements !

As I've stated in the initial post describing my lab environment for this scenario, I've simulated 2 distinct data centers by creating 2 individual Transport Zones housing individual VDS within a unique vCenter. I've then deployed a 2-node ESXi cluster within each data center, connected to the corresponding VDS. The diagram below describes this design:

globalNSX

Actually this allows me to simulate the recovery of VMs located in DC1 by doing the following:

  1. In DC1, shutdown VMs connected to LS1.
  2. Unregister VMs connected to LS1.
  3. Register those VMs in DC2 and connect them to the Logical Switch I've previously created with my Python script. (see previous post)
  4. Boot the VMs up and possibly answer VM question.

Let's try to do that with Python, by using pyVmomi for vSphere related tasks and the now well known Requests library for the NSX bit. We’re going to create 4 functions for that:

  • removeVM(…), which shuts VMs down and unregisters them.
  • recoverVM(…), which registers VMs to the right place.
  • attachVM(…), which attaches VMs to the appropriate Logical Switch.
  • startVM(…), which starts VMs up and answers VM question.
Basic example: Connecting to vCenter

The first step is to install pyVmomi itself. For that just open a command prompt, it doesn't really matter whether you're using Python with Windows or Linux. But still, I'd recommend you to use a Linux environment as it's more stable and you simply experience less problems when working with certain Python libraries. Use pip to install the package:

$ pip install --upgrade pyvmomi

We are now able to use the vSphere API wrapper within our Python script. The next thing to do is to import the pyVmomi modules we need and establish the connection with the vCenter, which at this point creates a Service Instance Object, the singleton root object of the inventory.

#!/usr/bin/env python

from pyVim.connect import SmartConnect, Disconnect
import atexit

vCenter = 'vcsa01.lab.int'
vmUser = 'root'
pwd = 'vmware'

def main():

    si = SmartConnect (host='vCenter', user='vmUser', pwd='password', port='443')
    atexit.register(Disconnect, si)
    print si.content.about

if __name__ == '__main__':
    main()

The script uses the print function to display information about the vCenter connection. Here is an output example if the connection is ok:

$ python myScript.py
(vim.AboutInfo) {
   dynamicType = ,
   dynamicProperty = (vmodl.DynamicProperty) [],
   name = 'VMware vCenter Server',
   fullName = 'VMware vCenter Server 5.5.0 build-1623101',
   vendor = 'VMware, Inc.',
   version = '5.5.0',
   build = '1623101',
   localeVersion = 'INTL',
   localeBuild = '000',
   osType = 'linux-x64',
   productLineId = 'vpx',
   apiType = 'VirtualCenter',
   apiVersion = '5.5',
   instanceUuid = '9F6CC738-EC6F-4842-BB19-16C2BFAB11D8',
   licenseProductName = 'VMware VirtualCenter Server',
   licenseProductVersion = '5.0'
}
Defining static variables

Before going further I wanted to detail various parameters required for the whole script to work. They can be found at the beginning of the script attached to this post, declared as static variables.

Shutdown and Unregister VMs

We're going to shutdown the VMs based on a predefined list (vmList) containing their displayed names. For that we define a Python function called removeVM. In that function, we need to instantiate a Container View that contains all the VMs connected to the vCenter. Then we just filter those VMs to match the VM on which we want to execute the shutdown operation, based on its name.

#The function takes 2 arguments: the VM to be removed and the vSphere Service Instance object.
def removeVM(vm, si):

    #Create root Service Instance Object
    content = si.content
    #Create Virtual Machine container
    vmCont = content.viewManager.CreateContainerView(content.rootFolder, [vim.VirtualMachine], True)
    #Create Virtual Machine view, which contains all the VMs managed by the vCenter
    vmView = vmCont.view
    vmCont.Destroy()

    #Find the Virtual Machine object that matches the VM name passed as an argument
    for myVM in vmView:
        if myVM.name == vm:
            break

    #Return error if no VM has been matched
    if myVM.name != vm:
        return -1
    else:
        if myVM.runtime.powerState != 'poweredOff':
            task = [myVM.PowerOff()]
            WaitForTasks(task, si)
        myVM.Unregister()

The WaitForTasks() function aims at managing non-blocking tasks, such as VM power-on, shutdown, etc. It’s the same function as defined in github samples.

def WaitForTasks(tasks, si):

    pc = si.content.propertyCollector

    taskList = [str(task) for task in tasks]

    objSpecs = [vmodl.query.PropertyCollector.ObjectSpec(obj = task) for task in tasks]
    propSpec = vmodl.query.PropertyCollector.PropertySpec(type = vim.Task, pathSet = [], all = True)
    filterSpec = vmodl.query.PropertyCollector.FilterSpec()
    filterSpec.objectSet = objSpecs
    filterSpec.propSet = [propSpec]
    filter = pc.CreateFilter(filterSpec, True)

    try:
      version, state = None, None

      #Loop looking for updates till the state moves to a completed state.
      while len(taskList):
         update = pc.WaitForUpdates(version)
         for filterSet in update.filterSet:
            for objSet in filterSet.objectSet:
               task = objSet.obj
               for change in objSet.changeSet:
                  if change.name == 'info':
                     state = change.val.state
                  elif change.name == 'info.state':
                     state = change.val
                  else:
                     continue

                  if not str(task) in taskList:
                     continue

                  if state == vim.TaskInfo.State.success:
                     # Remove task from taskList
                     taskList.remove(str(task))
                  elif state == vim.TaskInfo.State.error:
                     raise task.info.error
         # Move to next version
         version = update.version
    finally:
        if filter:
            filter.Destroy()
Recovering Vitual Machines

Recovering VMs on the second simulated data center involves several steps that we're going to code in a function called recoverVM.

The first step consists in building a cluster Container View that will include all the clusters managed by our unique vCenter. Then we're going to skim through this list to match the cluster where we want to recover the VMs, as defined by the variable drCluster.

Once we've got the cluster object defined, we must chose the appropriate host in that cluster. Once again we're going to skim through the list of available hosts to match the host where we want to recover the VM, as defined by the variable drHostAddr.

The last step consists in choosing the right folder where we want to register the VM. As previously, we're going to skim through the list of existing folders to match the folder name defined in the variable vmFolder.

#The funtion takes 6 arguments: the VM to be recovered, the destination ESXi cluster, the destination host,
#the destination VM folder and the datastore housing the VM to be restored
def recoverVM(vm, si, dstCluster, dstHost, dstFolder, dstDatastore):

   content = si.content
    #Create ESXi Cluster Container
    clCont = content.viewManager.CreateContainerView(content.rootFolder.childEntity[0].hostFolder, [vim.ClusterComputeResource], True)
    #Create Cluster view, which contains all the clusters managed by the vCenter
    clView = clCont.view
    clCont.Destroy()

    #Find the Cluster object that matches the Cluster name passed a an argument.
    for cl in clView:
        found = False
        if cl.name == dstCluster:
            #Need the root Resource Pool object for VM registration
            RP = cl.resourcePool
            for host in cl.host:
                if host.name == dstHost:
                    drHost = host
                    found = True
                    break
        if found:
            break

    folders = content.rootFolder.childEntity[0].vmFolder.childEntity

    #Register the VM in the folder passed as an argument
    for folder in folders:
        if folder.name == dstFolder:
            task = folder.RegisterVM_Task(path = '%s %s/%s.vmx' % (dstDatastore, vm, vm), pool=RP, host=drHost, asTemplate=False)
            WaitForTasks([task], si)
Attach VMs to Logical Switch

This part was surprisingly the most challenging. The NSX 6.1 API guide has a section about attaching and detaching a Virtual Machine from a Logical Switch in the Working with Logical Switches chapter, however I couldn't make that work. Apparently there are errors in both the REST URL and the request body specified in the documentation.

But luckily, one of my friend at VMware (@tsugliani) helped me sort this out and get the right URL and body syntax. It should be corrected in the API guide soon. In the meantime, check the attachVM function for the correct version. The next snippet actually contains 2 functions:

  • attachVM(…), which effectively attaches the VM to the Logical Switch.
  • findLSId(…), which identifies the Logical Switch Object ID given its name, as displayed by NSX Manager. This is done with the help of the lxml module, a powerful tool to parse XML with Python. My 2 cents here is to recommend you NOT to manage XML with regular expressions, this is just a mess…

I’ve commented out the script so you can understand the different sections.

#The function take 3 arguments: the name of the VM to be connected, the Logical Switch, and the Service Instance Object
def attachVM(vm, dstLS, si):

 content = si.content
 vmCont = content.viewManager.CreateContainerView(content.rootFolder, [vim.VirtualMachine], True)
 vmView = vmCont.view
 vmCont.Destroy()
 for myVM in vmView:
 if myVM.name == vm:
 break

 if myVM.name != vm:
 return -1
 else:
 #Find the VM instance UUID required to build the XML body used for the NSX API call
 vmInstanceUuid = myVM.config.instanceUuid
 #Determine the Logical Switch Object ID given the Logical Switch name passed as an argument
 myLS = findLSId(dstLS)

 rheaders = {'Content-Type': 'application/xml'}
 #Build the XML body used by NSX API to attach the VM to the Logical Switch
 xml_Reconfig = '''
 <com.vmware.vshield.vsm.inventory.dto.VnicDto>
 <objectId>''' + vmInstanceUuid + '''.000</objectId>
 <vnicUuid>''' + vmInstanceUuid + '''.000</vnicUuid>
 <portgroupId>''' + str(myLS) + '''</portgroupId>
 </com.vmware.vshield.vsm.inventory.dto.VnicDto>
 ''' 

 #Execute the API call to connect the VM to the Logical Switch
 r = requests.post(MANAGER + '/api/2.0/vdn/inventory/ui/vm/vnic/reconfig', data = xml_Reconfig, auth = (USER, PASS), verify = False, headers = rheaders) 

def findLSId(lsName):

 rheaders = {'Content-type': 'application/xml'}
 #Get the list of Logical Switch formatted as XML
 r = requests.get(MANAGER + '/api/2.0/vdn/scopes/' + tz + '/virtualwires', auth = (USER, PASS), verify = False, headers = rheaders)
 #Create XML Element for further parsing
 doc = etree.fromstring(r.content)
 #Find Logical Switch Object ID of the Logical Switch name passed as an argument and return it as a string
 for ls in doc.xpath("/virtualWires/dataPage/virtualWire[name = '" + lsName + "']/objectId"):
 return ls.text
Calling the functions

The last step is to define the main function. After instantiating the Service Instance Object, the idea is to call the previous procedures within a loop skimming through the Virtual Machines list we’ve previously defined.

 def main ():

    #Create Service Instance Object
    si = SmartConnect (host = vCenter, user = vmUser, pwd = password, port = '443')
    atexit.register(Disconnect, si)
    #Execute functions for Virtual Machines defined in the global list
    for vm in vmList:
        removeVM(vm, si)
        recoverVM(vm, si, drCluster, drHostAddr, vmFolder, datastore)
        attachVM(vm, lSwitch, si)
        startVM(vm, si)

If you're still with me, then you’re really brave – CONGRATULATIONS!!!. Feel free to improve the code, and share it back with me!

Comments

comments powered by Disqus