vCenter Virtual Machines

vCenter VM and VM Templates

To learn how to use VMs and VM Templates you can read the Managing Virtual Machines Instances and Managing Virtual Machine Templates, but first take into account the following considerations.

OpenNebula creates VM Templates using the import tools, but for your reference a VM Template definition in OpenNebula that represents a vCenter VM Template should contain the following attributes:

Attribute	Note
CPU	Physical CPUs to be used by the VM. This does not have to relate to the CPUs used by the vCenter VM Template, OpenNebula will change the value accordingly
MEMORY	Physical Memory in MB to be used by the VM. This does not have to relate to the CPUs used by the vCenter VM Template, OpenNebula will change the value accordingly
HYPERVISOR	Should be set to vcenter
NIC	One or more NIC elements. Check VM template reference. Valid MODELs are: virtuale1000, virtuale1000e, virtualpcnet32, virtualsriovethernetcard, virtualvmxnetm, virtualvmxnet2, virtualvmxnet3.
DISK	One or more DISK elements. Check VM template reference.
GRAPHICS	Multi-value - Only VNC supported, check the VM template reference.
CONTEXT	All sections will be honored. You can find more information about contextualization in the vcenter Contextualization section.
VCENTER_RESOURCE_POOL	(Optional) By default, the VM will be deployed to the default resource pool. If this attribute is set, its value will be used to confine this the VM in the referred resource pool. Check this section for more information.
VCENTER_VM_FOLDER	(Optional) If you use folders to group your objects like VMs and you want a VM to be placed inside an specific folder, you can specify a Deployment Folder where the VM will be created. The Deployment Folder is a path which uses slashes to separate folders. More info in the VM cloning Section
VCENTER_TEMPLATE_REF	Managed Object Reference of the vCenter datastore. Please visit the Managed Object Reference section to know more about these references.
VCENTER_TEMPLATE_NAME	Name of VM and VM Template in vCenter
VCENTER_CCR_REF	Managed Object Reference of the vCenter cluster related with the template. Please visit the Managed Object Reference section to know more about these references.
VCENTER_INSTANCE_ID	The vCenter instance ID. Please visit the Managed Object Reference section to know more about these references.

After a VM Template is instantiated, the life-cycle of the resulting virtual machine (including creation of snapshots) can be controlled through OpenNebula. Also, all the operations available in the Admin view can be performed, including:

• network management operations like the ability to attach/detach network interfaces

• capacity (CPU and MEMORY) resizing

• VNC connectivity

• Attach/detach VMDK images as disks

• Resize VM disks (shrink not supported) before the VM is deployed or when the VM is in POWEROFF state.

• Migrate VMs to other vCenter clusters. Migrating to other vCenter instances is not supported.

The monitoring attributes retrieved from a vCenter VM are:

• VCENTER_ESX_HOST: The ESX host where the VM is running

• VCENTER_GUEST_IP: IP address reported by VMWare Tools or Open VM Tools.

• VCENTER_GUEST_STATE: The state of the VM reported by VMWare Tools or Open VM Tools.

• VCENTER_RP_NAME: Resource Pool where the VM is running. Check this section for more information.

• VCENTER_VMWARETOOLS_RUNNING_STATUS: The status of VMWare Tools software or Open VM Tools software.

• VCENTER_VMWARETOOLS_VERSION: The version of the VMWare Tools or Open VM Tools.

• VCENTER_VMWARETOOLS_VERSION_STATUS: The status version of the VMWare Tools or Open VM Tools.

• VCENTER_DRS: The status of the vSphere DRS (Distributed Resource Scheduler).

• VCENTER_HA: The status of the vSphere HA (High Availability).

• NETTX: Bytes transmitted.

• NETRX: Bytes received.

• DISKRDBYTES: VM disks read bytes

• DISKRDIOPS: VM disks read IOPS

• DISKWRBYTES: VM disks write bytes

• DISKWRIOPS: VM disks write IOPS

• VCENTER_VM_NAME: VM name on vCenter

• DISK_x_ACTUAL_PATH: Physical path of the disk inside the vCenter datastore

vCenter Template or Wild VM Importing Procedure

While a VM Template or Wild VM is being imported, OpenNebula will inspect the virtual disks and virtual nics and it will create images and virtual networks referencing the disks and port-groups used by the VM. This process may take some time, please be patient.

You have more information about these procedures:

• vCenter Template import

• Wild VM import

VM Template Cloning Procedure

OpenNebula uses VMware cloning VM Template procedure to instantiate new Virtual Machines through vCenter. From the VMware documentation:

Deploying a virtual machine from a template creates a virtual machine that is a copy of the template. The new virtual machine has the virtual hardware, installed software, and other properties that are configured for the template.

The cloning procedure involves the following steps:

Step 1. Choosing a datastore

By default, the VM will be deployed in the datastore that the OpenNebula’s scheduler chooses according to its policy.

The policy is set in the /etc/one/sched.conf configuration file and by default OpenNebula will try to deploy the VMs trying to distribute them across the available datastores.

You can force that OpenNebula uses specific datastores overriding the scheduler’s policy with the SCHED_DS_REQUIREMENTS as explained here.

It’s compulsory that you import vCenter datastores before trying to deploy a VM and you must be sure that the datastores are shared by every ESX host in the cluster.

Step 2. Specifying how the disks are copied

OpenNebula instructs vCenter to “move all disk backing an disallow sharing”. That means that vCenter will create a full clone of the disks inside the template, and that full clone flattens all disks from the parent-most to the child-most disk.

However if you import the template with Linked Clones support OpenNebula will “move only the child-most disk backing” which means that any parent disk backings should be left in their current locations and if the disks have delta disks on top of them, then linked clones of the template disks will be used.

You have more information about disk moving operations here and the use of Linked Clones here.

Step 3. Selecting a Resource Pool

OpenNebula uses the default cluster resource pool to place the VM instantiated from the VM template, unless VCENTER_RESOURCE_POOL variable is defined in the OpenNebula host template or VM template. You have more information about resource pools here in OpenNebula docs.

Step 4. Deciding the VM folder in vSphere’s VM and Templates view

When the VM is cloned from the VM template the VM is placed in vCenter by default in the same location where the vCenter template is located. However you may place the VM in a different folder using the VCENTER_VM_FOLDER attribute as explained here in OpenNebula docs

Saving a VM Template: Instantiate to Persistent

At the time of deploying a VM Template, a flag can be used to create a new VM Template out of the VM.

onetemplate instantiate <tid> --persistent

You can also use this feature from Sunstone when you instantiate a template:

OpenNebula does the following when you use Instantiate to Persistent:

• A copy of every disk in the template is made and stored as OpenNebula images. Unmanaged disks which are disks that have OPENNEBULA_MANAGED attribute set to NO and that represents the disks that already exists in the vCenter template will be copied as non-persistent images. The rest of the disks will be represented as persistent images. Note that volatile disks won’t have an image associated.

• A new OpenNebula template will be created and the disks added in the previous step will be included in the template.

• Whenever the VM life-cycle ends (a VM terminate action), OpenNebula will instruct vCenter to create a new vCenter template out of the VM, with the settings of the VM.

• The OpenNebula VM template will point to the new vCenter template, so it can be instantiated through OpenNebula.

This functionality is very useful to create new VM Templates from a original VM Template, changing the VM configuration and/or installing new software, to create a complete VM Template catalog.

Important

Don’t detach disks from the VM or resize any disk of the VM once you’ve deployed it with Instantiate as Persistent, as when the VM is terminated the OpenNebula template that was created before the VM was deployed will differ from the template created in vCenter. Differences between the templates may affect operations on VMs based on unsynced templates.

Saving a VM Template: Save As

In Sunstone Cloud View you can poweroff a VM an use the save icon to create a new OpenNebula template from this VM.

OpenNebula will ask if you want to create a copy of the disks, select non-persistent images or persistent images. A message will inform you that the new OpenNebula template has been created and the VM will show the state SAVING_IMAGE.

Refresh the VM state with the icon next to the VM’s name, you’ll see the VM in OFF state when your new OpenNebula template is ready to use in the templates tab.

Your new VM template will contain DISK elements that will point to the disk copies created and NIC elements that will point to the same OpenNebula virtual networks used by the VM this template is based on.

This functionality enables the VM Backup functionality in vCenter VMs.

Note

The new OpenNebula VM Template has a Managed Object Reference to the vCenter template used to create the original VM. This implies that when a VM is deployed from the new OpenNebula template, a VM will be cloned from the original vCenter template where the old disks will be detached and the disk copies that were created previously will be attached.

VM Scheduling

OpenNebula scheduler should only chose a particular OpenNebula host for a OpenNebula VM Template representing a vCenter VM Template, since it most likely only would be available in a particular vCenter cluster.

The scheduler will inspect the VM Template and it will choose to deploy the VM in an OpenNebula host which is member of an OpenNebula cluster that contains the datastores where the DISKs images are stored and that contains the virtual networks used by the NICs of the VM template. When a vCenter cluster is imported into OpenNebula an OpenNebula Host is created and added to a OpenNebula cluster that is created by default if no other cluster is specified in the import process.

Note

If a VM is stuck in the PENDING state, that means that the scheduler hasn’t found a host and datastores that satisfies its requirement. In this case check that the images and networks defined in the VM template are located in an OpenNebula cluster other than the default cluster and check that the OpenNebula host is also assigned to the same OpenNebula cluster.

Since a vCenter cluster is an aggregation of ESX hosts, the ultimate placement of the VM on a particular ESX host would be managed by vCenter, in particular by the Distribute Resource Scheduler (DRS).

In Sunstone, a host abstracting a vCenter cluster will have an extra tab showing the ESX hosts that conform the cluster.

Attaching a CDROM to a Virtual Machine

You can attach a CDROM to a Virtual Machine creating first an OpenNebula image from an ISO file.

Then the CDROM can be attached to a Virtual Machine template or can be attached to deployed Virtual Machine, ONLY if the Virtual Machines is in the POWEROFF state. OpenNebula tries to connect the ISO file as an IDE CD-ROM drive which is not a hot-pluggable device that’s why the Virtual Machine must not be RUNNING (powered on).

Disks monitoring

OpenNebula gathers disks monitoring info for each VM providing metrics like the rate of reading/writing data to the VM’s virtual disks and the read/write IOPS. Real-time data is retrieved from vCenter thanks to the Performance Manager which collects data every 20 seconds and maintains it for one hour.

Important

OpenNebula requires that you set the right Statistics Level in vCenter so disk metrics are generated and stored. Increasing the level implies that more space is needed to store metrics so check that you have enough storage before changing the level.

vCenter Statistics level for 5-minutes data must be set to 2.

The rate of reading/write is provided by vCenter as an average using KB/s unit. The graphs provided by Sunstone are different from those found in vCenter under the Monitor --> Performance tab when selecting Realtime in the Time Range drop-down menu. The reason is that Sunstone uses polling time as time reference while vCenter uses sample time on their graphs, so an approximation to the real values aggregating vCenter’s samples between polls is needed. As a result, peaks may be different in value and different peaks between polls won’t be depicted. Sunstone’s graphs will provide a useful information about disks behaviour which can be examined on vCenter later with more detail.

vCenter Images

You can follow the common Managing Images Section, considering that VMDK snapshots are not supported as well as the following considerations.

OpenNebula scans VM Templates and Wild VMs for existing disks and it will create OpenNebula images that will represent those virtual disks. Thanks to this scanning process, existing disks will be visible for OpenNebula, and therefore can be detached from the deployed VMs. The following information is important about images created when a vCenter template or Wild VM is imported:

• VM disks in imported as part of a VM Template or Wild VM are considered unmanaged images.

• An unmanaged image won’t be cloned by OpenNebula when a VM is instantiated. When OpenNebula deploys a VM, vCenter will clone the vCenter template and it will be responsible of creating the copies of the template disks and attach them to the new Virtual Machine.

• Although the images are considered unmanaged, you can perform operations like detaching the disks.

• Virtual Machines in vCenter will have some variables created by OpenNebula that allows an OpenNebula disk element to be related with a vCenter Virtual Hard Disk. For example the unmanaged DISK with ID=0 has a variable called opennebula.disk.0 in vCenter’s VM that stores a reference to the disk created by vCenter that will help OpenNebula identify what disk has to be detached.

• The VCENTER_IMPORTED attribute is set to YES in the Image template for prevent accidental deletions.

• Although these images represents files that already exists in the datastores, OpenNebula accounts the size of those imported images as if they were new created files hence the datastore capacity is decreased even though no real space in the vCenter datastore is being used by the OpenNebula images. You should understand this limitation if for example an image cannot be imported as OpenNebula reports that no more space is left or if you’re using disk quotas.

• The images that have been imported will have a name generated by OpenNebula. That name contains the name of the VMDK file, the datastore name and the OpenNebula template that is related with that image.

There are three ways of adding VMDK representations in OpenNebula:

• Upload a new VMDK from the local filesystem

• Register an existent VMDK image already in the datastore

• Create a new empty datablock

The following image template attributes need to be considered for vCenter VMDK image representation in OpenNebula:

Attribute	Description
PATH	This can be either: local filesystem path to a VMDK to be uploaded, which can be a single VMDK or tar.gz of vmdk descriptor and flat files (no OVAs supported). If using a tar.gz file which contains the flat and descriptor files, both files must live in the first level of the archived file as folders and subfolders are not supported inside the tar.gz file, otherwise a “Could not find vmdk” error message would show up. path of an existing VMDK file in the vCenter datastore or HTTP url. In this case a ‘’vcenter://’’ prefix must be used (for instance, an image win10.vmdk in a Windows folder should be set to vcenter://Windows/win10.vmdk)
VCENTER_ADAPTER_TYPE	Default adapter type used by virtual disks to plug inherited to VMs for the images in the datastore. It is inherited by images and can be overwritten if specified explicitly in the image. Possible values (careful with the case): lsiLogic, ide, busLogic. More information in the VMware documentation. Known as “Bus adapter controller” in Sunstone.
VCENTER_DISK_TYPE	The type of disk has implications on performance and occupied space. Values (careful with the case): delta,eagerZeroedThick,flatMonolithic,preallocated,raw,rdm,rdmp,seSparse,sparse2Gb,sparseMonolithic,thick,thick2Gb,thin. More information in the VMware documentation. Known as “Disk Provisioning Type” in Sunstone.
VCENTER_IMPORTED	It will be set to YES for images that have been imported when a vCenter template or Wild VM is imported. If this attribute is set to YES, OpenNebula will not delete the VMDK file in vCenter so you don’t actually delete the hard disk attached to a template. If you remove this attribute the VMDK file will be deleted when the Image is deleted in OpenNebula.

VMDK images in vCenter datastores can be:

• Cloned

• Deleted

• Hotplugged to VMs

Images can be imported from the vCenter datastore using the onevcenter tool.

Both “VCENTER_ADAPTER_TYPE” and “VCENTER_DISK_TYPE” need to be set at either the Datastore level, the Image level or the VM Disk level. If not set, default values in /etc/one/vcenter_driver.defaults file should be used.

Note

By default, OpenNebula checks the datastore capacity to see if the image fits. This may cause a “Not enough space in datastore” error. To avoid this error, disable the datastore capacity check before importing images. This can be changed in /etc/one/oned.conf, using the DATASTORE_CAPACITY_CHECK set to “no”, and restarting OpenNebula.

vCenter Tags and Categories

Working with vCenter tags and categories can be extremely helpful. It allows to add valuable information to inventory items and make them searchable and sortable.

To assign a Label and Category to a VM add the following block to the Template definition before instantiating it. Several blocks can be added as needed.

VCENTER_TAG = [
    CATEGORY_NAME = "Category name",
    DESCRIPTION = "Tag description",
    NAME = "Tag name"]

OpenNebula uses the CATEGORY_NAME field to create a Category in case there is no Category with this name. If a CATEGORY_NAME is not supplied, OpenNebula will create a default category called OpenNebula.

OpenNebula uses the NAME and DESCRIPTION fields to create a TAG with this name and description that belongs to the category.

vCenter Live Capacity Resize (CPU & MEMORY)

To enable resize for a virtual machine, add the following block to the template definition before creating an instance or in the USER_TEMPLATE directly in the Virtual Machine.

HOT_RESIZE = [
    CPU_HOT_ADD_ENABLED = "YES",
    MEMORY_HOT_ADD_ENABLED = "YES"]

OpenNebula uses the CPU_HOT_ADD_ENABLED field to activate or deactivate the CPU Hot resize in vCenter. If a CPU_HOT_ADD_ENABLED is not supplied or the value is NO, OpenNebula will deactivate the CPU Hot resize in vCenter.

OpenNebula uses the MEMORY_HOT_ADD_ENABLED field to activate or deactivate the MEMORY Hot resize in vCenter. If a MEMORY_HOT_ADD_ENABLED is not supplied or the value is NO, OpenNebula will deactivate the MEMORY Hot resize in vCenter.

To activate or deactivate these options it is necessary to turn off and resume the Virtual Machine, since vCenter does not allow changing these settings with the Virtual Machine turned on.

Note

CPU Hot Add is a feature that allows the addition of vCPUs to a running virtual machine. Enabling this feature disables vNUMA for that Virtual Machine. You can find more info here in this VMware KB article 2040375

You can find more information on how to resize a Virtual Machine in OpenNebula using the CLI or in Sunstone here