Introducing the Wallaroo Machine Learning Pipeline
At Wallaroo, our driving vision is to make AI deployment push-button easy and intuitive. If we design something and discover it’s not as intuitive as it should be, then we’re not afraid to give it another go. After a few iterations, we’ve streamlined the way data scientists interact with the platform to a core concept: the model pipeline.
A pipeline contains all the artifacts required for a specific ML process: not only all the trained models that are involved with that process but also any intermediate data processing that may be required for the models to run. The simplest pipeline is a single model; more complicated pipelines can include chained models, or multiple models being compared in an A/B test or other experiments.
In this article, we will cover Wallaroo pipeline features, and discuss the advantages of organizing data science processes via pipelines.
Pipeline Features
Intermediate Processing Steps
Real-world data often has to be prepared and massaged before it’s input to a model: this data processing and feature engineering is often the responsibility of the data scientist who is building the model. For example, you might have an ML process that takes in natural language text, like a question, and produces a human-understandable answer. This requires a preprocessing step that encodes the question text into a numerical representation that the model understands, and a postprocessing step that translates the model’s output back into text that the user can understand.
Through Wallaroo, the data scientist can deploy data preprocessing and postprocessing steps implemented in Python in the same way they deploy models, via our easy-to-use SDK:
By putting the processing steps and the model together in a Wallaroo pipeline, the data scientist can document the necessary data processing steps, and make sure they are deployed together.
Now suppose it’s time to update the question-answering model. If the new model uses the same text pre-and post-processing, then it’s easy just to update the model in the pipeline. If the new model uses different processing steps, then it’s still straightforward to create a new pipeline and deploy it in place of the old one!
Composability
Wallaroo pipelines also allow you to compose multiple models into a single flow, including any intermediate data processing that may be needed between the models. For example, you may want to analyze news articles by extracting the entities mentioned in the text, then doing sentiment analysis on that text, including the information about the extracted entities.
This might involve two models: an entity extraction model, and a sentiment analysis model. These models (and their associated data processing) might have been developed separately. With Wallaroo, you can compose these models into a single pipeline and deploy them.
A/B Testing and Shadow Deployments
Wallaroo pipelines also have explicit experimentation configurations that allow you to compare how different models perform in the real world. This might be as an A/B (or A/B/C…) test, where live input is streamed to the various models according to criteria that the user can specify. Or, it might be as a shadow deployment, where the primary models and the shadow models all see all the input, and their predictions are logged, but only the response from the primary model is returned to the system. Shadow deployments let data scientists “preview” how a model performs on real-world data before fully committing to its use in production.
Model Monitoring
In addition to easing the deployment process, Wallaroo pipelines also allow the data scientist to add some useful monitoring and alerting. For example, if you are running a credit-card fraud model over your credit card transactions, you may want to monitor and be alerted if an unusually high number of transactions get marked as “fraud” over a short period.
You may also want to monitor the inputs to the model, as unusual changes to model input may result in unexpected or undesirable model behavior. Wallaroo’s model monitoring can identify potentially anomalous input data as well as unusual model behavior, and send alerts when appropriate, so you can get to potential problems before it’s too late.
Facilitating the Deployment Process
An ML-based decision process can have a lot of moving parts, and putting these processes into production can be time-consuming. The Wallaroo platform and pipelines help data scientists organize and design complex ML and AI applications in a way that makes the deployment process simpler, faster, and more maintainable.
Interested in running a test to see how Wallaroo’s ML pipelines compare to your current deployment solutions? Reach out to us at deployML@wallaroo.ai so we can set up a technical assessment test.
