Overview

Overview of AI factory deployment and validation.

The purpose of the AI Factory blueprint collection is to provide a step-by-step process for setting up a simple AI Factory system and getting it up and running quickly, including:

  • Identifying the minimum hardware and networking requirements for your AI Factory. These baseline specifications also serve as a reference for more advanced deployments. OpenNebula supports high-performance architectures such as InfiniBand, Spectrum-X, and NVLink, although these setups are not automated and require custom configuration.

  • Follow the step-by-step deployment instructions using OneDeploy to build your AI Factory, with options for both on-premises installations and cloud-based deployments.

  • Optionally, you can validate the setup using the same methodology we apply during formal infrastructure acceptance. This validation covers direct vLLM execution for inference, SLURM integration for fine-tuning, and Kubernetes-based execution using NVIDIA Dynamo® for inference and NVIDIA KAI Scheduler® .

Hardware Requirements

AI Factories require high performance server hardware, including accelerated infrastructure such as GPUs and high-performance networking. The following details outline a minimal recommended hardware configuration:

  • CPU: 16 physical or logical CPU cores with hardware virtualization support.
  • Memory: 64 GB RAM. More memory is recommended when running multiple Kubernetes workers or large AI models.
  • Storage: At least 250 GB of fast SSD or NVMe storage, with sufficient additional capacity for VM images, container images, model weights, and datasets.
  • GPU: At least one NVIDIA data-center GPU with PCI passthrough support. Two GPUs are recommended for testing disaggregated inference workloads such as separate prefill and decode workers. See the Platform Notes for supported GPU models.
  • Networking: At least one physical Ethernet interface suitable for bridged OpenNebula networking. A 10 GbE or faster network is recommended for multi-node deployments.
  • Virtualization: CPU virtualization extensions and IOMMU enabled in the system firmware to support KVM and PCI passthrough.
  • Operating system: Ubuntu 24.04 is highly recommended. See the Platform Notes for a list of alternative operating systems supported by OpenNebula.
  • Bare-metal Cloud Instances: If you are using resources provided by a 3rd-party cloud provider, bare-metal instances are highly recommended. AI Factory deployments on virtual instances will result in heavily degraded performance and may not function at all.

Basic Outline

Configuring, deploying and validating a high-performance AI infrastructure using OpenNebula involves these steps:

  1. Familiarize yourself with Architecture and Specifications. We recommend consulting the guide on GPU PCI-passthrough for details relating to your GPU hardware and IOMMU.

  2. Deploy and configure your AI Factory with one of these alternatives:

  3. Integrate external infrastructure services if required:

  4. Perform Validation: As a prerequisite, you must have an AI Factory ready to be validated after completing the above installation procedures. These are the options to validate your AI Factory: