Storage System on

Overview

Mon, 17 Feb 2025 00:00:00 +0000

How Should I Read This Chapter

Before performing the operations outlined in this chapter, familiarize with Node Deployment from the Installation guide.

After that, proceed with the specific Datastore documentation you might be interested in.

Datastore Types

Storage in OpenNebula is designed around the concept of datastores. A datastore is any storage medium to store disk images. OpenNebula distinguishes between three different datastore types:

Images Datastore, which stores the base operating system images, persistent data volumes, CD-ROMs and File Systems.
System Datastore holds disks of running Virtual Machines. Disk are moved from/to the Images when the VMs are deployed/terminated.
Files & Kernels Datastore to store plain files (not disk images), e.g. kernels, ramdisks, or contextualization files. See details here.

Driver Types

Storage portfolio

Use case	Description	Shared	Disk Format	Disk snapshots	VM snapshots	Storage migration	Fault tolerance	HV	Availability
Local storage	Images stored in frontend* and transferred to hosts via SSH on instantiation.	no	raw/qcow2	yes	yes	poweroff/live	no	KVM/LXC	EE/CE
NFS/NAS	Images stored in a NFS share, activated directly.	yes	raw/qcow2	yes	yes	poweroff/live	yes	KVM/LXC	EE/CE
Ceph	Images stored in a Ceph pool, activated directly.	yes	raw (RBD)	yes	no	poweroff	yes	KVM/LXC	EE/CE
SAN - LVM	Images stored as LVs in a SAN, activated directly.	yes	raw (LV)	yes	no	poweroff	yes	KVM	EE only
SAN - LVM (File Mode)	Images stored in frontend*, transferred to hosts via SSH, and copied to the SAN on instantiation.	yes	raw (LV) Images: raw/qcow2	yes**	no	poweroff/live	yes	KVM	EE/CE
SAN - NetApp	Images stored in a NetApp cabin, activated directly.	yes	raw (LUN)	yes	no	poweroff	yes	KVM	EE only
FileSystems - VirtioFS	Images are filesytem paths available on the hosts	yes	filesystems (dir)	no	no	no	yes	KVM	EE/CE

^* Additional options available by mounting remote filesystems in the frontend.

Datastores

Mon, 17 Feb 2025 00:00:00 +0000

Types

OpenNebula features three different datastore types:

The Image Datastore stores the Image repository.
The System Datastore holds disk for running Virtual Machines, usually cloned from the Image Datastore.
The Files & Kernels Datastore to store plain files used in contextualization, or VM kernels used by some hypervisors.

By default, OpenNebula will create an image (default), system (system), and files datastore (files). These datastores are configured to use the SSH protocol to transfer images. You can list the datastores in your cloud with the onedatastore list command:

NFS/NAS Datastore

Mon, 17 Feb 2025 00:00:00 +0000

This storage configuration assumes that your Hosts can access and mount a shared volume located on a NAS (Network Attached Storage) server. You will use this shared volume to store VM disk images files. The Virtual Machines will also boot from the shared volume.

The scalability of this solution will be bound to the performance of your NAS server. However, you can use multiple NAS servers simultaneously to improve the scalability of your OpenNebula cloud. The use of multiple NFS/NAS Datastores will allow you to:

Local Storage Datastore

Mon, 17 Feb 2025 00:00:00 +0000

This storage configuration uses the local storage area of each Host to run VMs. Additionally you’ll need a storage area for the VM disk image repository. Disk images are transferred from the repository to the Hosts using the SSH protocol.

Front-end Setup

The Front-end needs to prepare the storage area for:

Image Datastores to store the image repository.
System Datastores to hold temporary disks and files for VMs stopped and undeployed.

Simply make sure that there is enough space under /var/lib/one/datastores to store images and the disks of the stopped and undeployed Virtual Machines. Note that /var/lib/one/datastores can be mounted from any NAS/SAN server in your network.

Ceph Datastore

Mon, 17 Feb 2025 00:00:00 +0000

The Ceph Datastore driver allows the use of Ceph storage for images and disks of Virtual Machines.

Warning

This driver requires the OpenNebula nodes using the Ceph driver to be Ceph clients of a running Ceph Cluster. More information in Ceph documentation.

Ceph Cluster Setup

This guide assumes that you already have a functional Ceph Cluster in place. Additionally you need to:

Create a pool for the OpenNebula Datastores. Write down the name of the pool to include it in the datastore definitions.

$ ceph osd pool create one 128
$ ceph osd pool application enable one rbd
$ rbd pool init -p one
$ ceph osd lspools
0 data,1 metadata,2 rbd,6 one,

Define a Ceph user to access the datastore pool; this user will also be used by libvirt to access the disk images. For example, create the user libvirt:

On the Ceph Jewel (v10.2.x and earlier):

Raw Device Mapping Datastore

Mon, 17 Feb 2025 00:00:00 +0000

The RDM Datastore is an Image Datastore that enables raw access to block devices on Nodes. This Datastore enables fast VM deployments due to a non-existent transfer operation from the Image Datastore to the System Datastore.

Warning

The Datastore should only be usable by the administrators. Letting users create images in this Datastore is a huge security risk! For example, users could register an image /dev/sda and read the Host filesystem.

Datastore Layout

The RDM Datastore is used to register already existing block devices on the Nodes. The devices should be already set up and available, and VMs using these devices must be enforced to run on the Nodes ready for them. Additional Virtual Machine files, like deployment files or volatile disks, are created as regular files.

Kernels and Files Datastore

Mon, 17 Feb 2025 00:00:00 +0000

The Kernels and Files Datastore lets you store plain files to be used as VM kernels, ramdisks, or any other files that need to be passed to the VM through the contextualization process. The Kernels and Files Datastore does not expose any special storage mechanism but is a simple and secure way to use files within VM templates.

Configuration

The datastores common configuration attributes apply to the Kernels and Files Datastores and can be defined during the creation process or updated once the datastores have been created.

VirtioFS Datastore

Wed, 18 Feb 2026 00:00:00 +0000

Overview

OpenNebula supports sharing host directories with Virtual Machines using virtiofs. This enables high-performance, low-latency file sharing between the host and guests. The feature is exposed through a dedicated datastore configured to provide directory-based disks, which are attached to VMs and exported via virtiofs.

Unlike traditional block-based disks, these disks represent host directories. This makes them suitable for:

Shared data volumes across VMs
HPC / AI workloads requiring fast POSIX access
Container-like storage semantics inside VMs

Requirements

Before creating and using a VirtioFS Datastore, ensure the following: