Why You Need A Single Pane Of Glass To Observe & Manage Your Model Operations
Going from a machine learning prototype to an actual AI-based product is too often thought of as just wrapping a ML model in a container and deploying it on to a server. At Wallaroo, we refer to the overall set of necessary processes and capabilities as production ML operations. Together they determine if an organization is able to both operate its ML in an industrial fashion as well as whether it can actually deliver ROI from its ML investments. In order to truly operationalize ML in production, AI teams need to consider the full model operations lifecycle beyond deployment, including:
- How do I know the model inference is meeting the cost, latency, or throughput requirements needed by the business or other downstream systems, and continues to do so?
- How do I easily make an update or change to a model in production without impacting the business?
- How do I test a new version of a model so I can roll it out into production with confidence?
- How do I quickly test and iterate a new model to get to the point it is producing the outcomes the business is hoping for?
- How do I monitor the ongoing performance of a model?
- How do I scale the production environment with more data or more use cases?
- How do I provide audit for compliance and fit into an overall governance scheme?
- How do I map the production model version to the trained model version and artifacts if I ever need to go back and investigate or make changes?
As you can see from this list of considerations, production AI has to meet various business and compliance requirements, fit within a larger ecosystem, have full lifecycle management, and have repeatable, scalable processes. Getting a model onto a server is just one small (and important) aspect of operationalization. And because ML operations are often treated as just deployment to a server, everything else is left to the data scientists or ML engineers to address on an adhoc basis. And that means they can’t focus on what they were hired to actually do, critical items fall to the wayside, and the team has a hard time getting new ideas launched because it’s spending too much time duct-taping the existing system. It also means as more models are added, you are linearly adding more headcount.
This means model operations don’t scale, and neither does the investment in AI, especially since these are some of the hardest jobs to hire for and retain. When it comes to scaling and industrializing AI, it is important to have robust model operations which include model deployment, tracking, validation, monitoring and observability. That’s what the Wallaroo Model Operations Center provides which makes it unique in its ability to make ML repeatable, scalable, and efficient.
The Wallaroo Model Operations Center: The One Hub To Manage ML models In Production
Previous to founding Wallaroo, our CEO helped build the high frequency trading desk of a major Wall Street Firm. His team needed to constantly test dozens of models against each other at a time and quickly detect when market conditions changed such that their live algorithms no longer generated alpha.
It was with this experience in mind that he set out to create a product that simplified model management. His vision was for a single view that allowed data scientists to quickly deploy and manage models with simple commands; and for data scientists to have a single hub where they could monitor the performance of all models, whether to compare performance of models against each other in production environment, or to ensure live models remained accurate even as the environment and data changed.
With Wallaroo we have delivered on that vision. As we highlighted in our recent customer wins, what enterprise data teams are looking for is simplicity. Our simple self-service tools, whether via the SDK, APIs, or the Wallaroo User Interface, provide all the major functionalities of model operations (deploy/undeploy, pipeline management, model validation, model monitoring, observability — see here for the full Wallaroo glossary).
As we work with more customers, we continue to expand the capabilities of the Wallaroo Operations Center. For example, be on the lookout for more announcements about out-of-the-box data science platform integrations and additional runtimes support because we want to make Wallaroo not just easy to use, but also to plug right into your existing data ecosystem without requiring complicated migrations or forcing data scientists and ML Engineers to stop using their favorite tools.
This comprehensive set of capabilities for production ML combined with our high-performance model serving engine purpose-built for ML is what delivers on our core mission to make MLOps in production scalable, repeatable and efficient.
Because if we can make that part simple and easy, then your AI initiatives can scale non-linearly.
