Hosts and Clusters on

Overview

Mon, 17 Feb 2025 00:00:00 +0000

A Host is a server that has the ability to run Virtual Machines and that is connected to OpenNebula’s Front-end server. To learn how to prepare the Hosts, you can read the Installation guide. Hosts are usually grouped in Clusters.

How Should I Read This Chapter

In this chapter there are four guides describing these objects.

Host Management: Host management is achieved through the onehost CLI command or through the Sunstone GUI. You can read about Host Management in more detail in the Hosts guide.
Cluster Management: Hosts can be grouped in clusters. These clusters are managed with the onecluster CLI command or through the Sunstone GUI. You can read about Cluster Management in more detail in the Clusters guide.

You should read all the guides in this chapter to familiarize yourself with these objects. For small and homogeneous clouds you may not need to create new clusters.

Hosts

Mon, 17 Feb 2025 00:00:00 +0000

In order to use your existing physical nodes, you have to add them to OpenNebula as Hosts. To add a Host only its hostname and type is needed.

Warning

Before adding a Linux Host check that you can SSH to it without being prompted for a password.

Creating and Deleting Hosts

Hosts are the servers managed by OpenNebula responsible for running VMs. To use these Hosts in OpenNebula you need to register them so they are monitored and made available to the scheduler.

Clusters

Mon, 17 Feb 2025 00:00:00 +0000

Clusters group together Hosts, datastores, and virtual networks that are configured to work together. A cluster is used to:

Ensure that VMs use resources that are compatible.

Assign resources to user groups by creating Virtual Private Clouds.

Clusters should contain homogeneous resources. Note that some operations like live migrations are restricted to Hosts in the same cluster.

The requirements for live migrating VMs between Hosts of the same cluster are that no differences occur in the following areas of the hypervisors:

Virtual Topology and CPU Pinning

Mon, 17 Feb 2025 00:00:00 +0000

Overview

In this guide you’ll learn to set up OpenNebula to control how VM resources are mapped onto the hypervisor ones. These settings will help you to fine tune the performance of VMs. We will use the following concepts:

Cores, Threads, and Sockets. A computer processor is connected to the motherboard through a socket. A processor can pack one or more cores, each one implements a separate processing unit that shares some cache levels, memory, and I/O ports. CPU cores’ performance can be improved by the use of hardware multi-threading (SMT) that permits multiple execution flows to run simultaneously on a single core.
NUMA. Multi-processor servers are usually arranged in nodes or cells. Each NUMA node holds a fraction of the overall system memory. In this configuration, a processor accesses memory and I/O ports local to its node faster than those of non-local ones.
Hugepages. Systems with big physical memory also use a large number of virtual memory pages. This big number makes the use of virtual-to-physical translation caches inefficient. Hugepages reduces the number of virtual pages in the system and optimizes the virtual memory subsystem.

In OpenNebula the virtual topology of a VM is defined by the number of sockets, cores, and threads. We assume that a NUMA node or cell is equivalent to a socket and they will be used interchangeably in this guide.

PCI Passthrough

Mon, 17 Feb 2025 00:00:00 +0000

It is possible to discover PCI devices in the Hosts and directly assign them to Virtual Machines in the KVM hypervisor.

The setup and environment information is taken from here. You can safely ignore all the VGA-related sections, those for PCI devices that are not graphic cards, or if you don’t want to output video signal from them.

Warning

The overall setup state was extracted from a preconfigured Fedora 22 machine. Configuration for your distro may be different.

Requirements

Virtualization Host must

NVIDIA vGPU & MIG

Mon, 17 Feb 2025 00:00:00 +0000

Configuring the hypervisor for NVIDIA® vGPU and MIG (Multi-Instance GPU) capabilities facilitates centralized management, flexibility in resource allocation, and enhanced security isolation across all virtualized GPU workloads.

BIOS

You need to check that the following settings are enabled in your BIOS configuration:

Enable SR-IOV
Enable IOMMU

Note that the specific menu options where you need to activate these features depend on the motherboard manufacturer.

NVIDIA GPU Passthrough

Thu, 16 Oct 2025 00:00:00 +0000

Here you will find detailed instructions for configuring PCI passthrough for high-performance NVIDIA® GPUs on x86_64 hypervisors. The procedures described here have been validated with NVIDIA H100 GPUs but can be adapted for other similar high-performance NVIDIA GPUs. This allows Virtual Machines to get exclusive access to the GPU, which is recommended for AI/ML workloads. It builds upon the concepts explained in the general PCI Passthrough guide. This guide is not applicable to ARM-based systems.

OpenNebula NVIDIA Fabric Manager

Mon, 01 Jan 0001 00:00:00 +0000

Introduction

The OpenNebula NVIDIA® Fabric Manager integration provides a complete solution for managing NVIDIA NVSwitch fabric within a cloud environment.

NVIDIA Shared NVSwitch Virtualization Model

The OpenNebula integration with Fabric Manager integration follows the NVIDIA Shared NVSwitch Virtualization Model. This model uses a Service VM in each hypervisor for managing the NVSwitches. The NVSwitches are added to the Service VM as PCI passthrough devices to configure the selected GPU partitioning. The Guest VMs are configured with PCI passthrough for the GPUs only, without any visibility of the NVSwitches. See the reference architecture in the image below.