Wallaroo Enterprise supports installation into a Kubernetes cluster that is isolated from the public internet. Organizations that need to install Wallaroo into an air gapped environment can contact their Wallaroo support representative to update their Wallaroo Enterprise license for air gap support.
The assumption for running machine learning models is that if you have a complex model with several parameters to evaluate and you have a short amount of time to get answers (measured in milliseconds), you will need the power of a GPU.
The problem is GPUs are expensive, have been hard to source (recently), and in some deployment scenarios, especially at the edge, are just not an option.
However, we’ve been able to surprise our clients by taking their most demanding NLP or even computer vision models, operationalizing them on standard CPUs, and generating faster inferences on less servers. That’s because we purpose built our runtime engine from the ground up in Rust specifically for machine learning.
Reach out to us if you’re ready to ditch GPUs to run even your boldest models.
The combination of 5G with AI promises new revenue streams to communication service providers (CSPs), such as private networks. But taking complex machine learning models trained in cloud or on-prem environments and deploying them at the edge is still relatively new. The dev environment can be so different from the live production environment that it could take months of reengineering before a single model can be successfully deployed at the edge. And once live, the data scientists who developed the model often have no view into the ongoing performance of their model until something goes wrong.
In this contributed article to RCR Wireless Media, Wallaroo’s Director of Architecture, Jason McCampbell, and Head of Product Marketing, Hector Leano, delve deeper into the observability and performance hurdles CSPs need to overcome in order to operationalize edge ML.