How to Shadow Deploy Models with Wallaroo

Prerequisites for Shadow Deployment

Step 1: Importing libraries

import wallaroofrom wallaroo.object import EntityNotFoundError

Step 2: Connecting to Wallaroo

wl = wallaroo.Client()

Step 3: Setting Variables

workspace_name = ‘ccfraud-comparison-demo’pipeline_name = ‘cc-shadow’pipeline_name_multi = ‘cc-shadow-multi’champion_model_name = ‘ccfraud-lstm’champion_model_file = ‘models/keras_ccfraud.onnx’shadow_model_01_name = ‘ccfraud-xgb’shadow_model_01_file = ‘models/xgboost_ccfraud.onnx’shadow_model_02_name = ‘ccfraud-rf’shadow_model_02_file = ‘models/modelA.onnx’sample_data_file = ‘./smoke_test.json’

Step 4: Creating or Connecting to a Workspace and a Pipeline

def get_workspace(name):wl = wallaroo.Client()workspace = Nonefor ws in wl.list_workspaces():if ws.name() == name:workspace= wsif(workspace == None):workspace = wl.create_workspace(name)return workspacedef get_pipeline(name):wl = wallaroo.Client()try:pipeline = wl.pipelines_by_name(pipeline_name)[0]except EntityNotFoundError:pipeline = wl.build_pipeline(pipeline_name)return pipelineworkspace = get_workspace(workspace_name)wl.set_current_workspace(workspace)pipeline = get_pipeline(pipeline_name)pipeline

Step 5: Loading the Models

champion = wl.upload_model(champion_model_name, champion_model_file).configure()model2 = wl.upload_model(shadow_model_01_name, shadow_model_01_file).configure()model3 = wl.upload_model(shadow_model_02_name, shadow_model_02_file).configure()

Step 6: Creating Shadow Deployment

pipeline.add_shadow_deploy(champion, [model2, model3])pipeline.deploy()

Step 7: Running Test Inference

response = pipeline.infer_from_file(sample_data_file)response

Step 8: Viewing Pipeline Logs

pipeline.logs()

Step 9: Viewing Shadow Deploy Pipeline Logs

logs = pipeline.logs_shadow_deploy()logs

Step 10: Undeploying the Pipeline

pipeline.undeploy()

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Wallaroo

Wallaroo

71 Followers

Wallaroo enables data scientists and ML engineers to deploy enterprise-level AI into production simpler, faster, and with incredible efficiency.