Ray AI
Appliance Description
Ray is an open-source framework for distributed computing and machine learning workloads. The Ray appliance leveragles Ray’s Serve library to enable efficient deployment of inference APIs, and includes the vLLM for fast inference and serving.
Main Features
The Ray appliance includes two frameworks for running LLMs:
- Hugging Face - Transformers, one of the most widely adopted frameworks for deploying large language models
- vLLM, the open source, high-perfomance engine for serving large language models with low latency and high throughput
- Configurable deployment options and behavior, controlled by contextualization parameters
Main References
- Ray in the OpenNebula one-apps project
- Full documentation for the Ray appliance
- Download the Ray appliance from the OpenNebula Marketplace
We value your feedback
Was this information helpful?
Glad to hear it
Sorry to hear that