How to Shadow Deploy Models with Wallaroo

Prerequisites for Shadow Deployment

Step 1: Importing libraries

import wallaroofrom wallaroo.object import EntityNotFoundError

Step 2: Connecting to Wallaroo

wl = wallaroo.Client()

Step 3: Setting Variables

workspace_name = ‘ccfraud-comparison-demo’pipeline_name = ‘cc-shadow’pipeline_name_multi = ‘cc-shadow-multi’champion_model_name = ‘ccfraud-lstm’champion_model_file = ‘models/keras_ccfraud.onnx’shadow_model_01_name = ‘ccfraud-xgb’shadow_model_01_file = ‘models/xgboost_ccfraud.onnx’shadow_model_02_name = ‘ccfraud-rf’shadow_model_02_file = ‘models/modelA.onnx’sample_data_file = ‘./smoke_test.json’

Step 4: Creating or Connecting to a Workspace and a Pipeline

def get_workspace(name):wl = wallaroo.Client()workspace = Nonefor ws in wl.list_workspaces():if == name:workspace= wsif(workspace == None):workspace = wl.create_workspace(name)return workspacedef get_pipeline(name):wl = wallaroo.Client()try:pipeline = wl.pipelines_by_name(pipeline_name)[0]except EntityNotFoundError:pipeline = wl.build_pipeline(pipeline_name)return pipelineworkspace = get_workspace(workspace_name)wl.set_current_workspace(workspace)pipeline = get_pipeline(pipeline_name)pipeline

Step 5: Loading the Models

champion = wl.upload_model(champion_model_name, champion_model_file).configure()model2 = wl.upload_model(shadow_model_01_name, shadow_model_01_file).configure()model3 = wl.upload_model(shadow_model_02_name, shadow_model_02_file).configure()

Step 6: Creating Shadow Deployment

pipeline.add_shadow_deploy(champion, [model2, model3])pipeline.deploy()

Step 7: Running Test Inference

response = pipeline.infer_from_file(sample_data_file)response

Step 8: Viewing Pipeline Logs


Step 9: Viewing Shadow Deploy Pipeline Logs

logs = pipeline.logs_shadow_deploy()logs

Step 10: Undeploying the Pipeline




Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store



Wallaroo enables data scientists and ML engineers to deploy enterprise-level AI into production simpler, faster, and with incredible efficiency.