The Diminishing Importance Of Performance…A Note From Our CEO On Our Recent Win With RealPage

Does performance matter?

When we first started Wallaroo in 2017 the answer seemed obvious. Of course enterprise data teams are looking to continuously optimize performance. If we could only show a benchmark demonstrating how many more inferences per second or dollar Wallaroo’s purpose-built engine could deliver, we would immediately win the business. We weren’t just a little bit more efficient — we were exponentially faster and cheaper than either DiY solutions cobbled together from open source software or even from all-in-one MLOps tools.

So what have we actually learned since then:

Performance can only disqualify you, but will not be on its own enough for customers to switch.

Enterprise data ecosystems are complex. Different business units have different use cases and different needs, leading to different platforms and tools used by different users. Wringing out a few basis points of cost savings is not the primary concern for a CIO or CTO to upend their current data stack. The CFO will often ask them to find ways to cut infrastructure costs, but it is much easier to ask current vendors for discounts versus embarking on any major migrations. Even adding new point solutions can take months to onboard going through Procurement, Compliance, and Legal.

If not performance and cost, then what will actually make a CTO or Head of Data Science interested in talking to us?

In a word: simplicity.

Enterprises are investing millions in building out data science teams and in tooling for those teams to build machine learning models. Those are some very smart people that do a good job delivering a predictive model based on historical data. But then taking that model and turning it into something the business can use in real life conditions is extremely complex. Oftentimes teams are fine with highly manual and bespoke processes when only dealing with a handful of models — think of them almost like pets that are cared for individually. They could take weeks or months to reengineer a data scientist’s notebook into production-ready code, and then manually oversee each model to make sure it was still accurate as the environment changes.

The problem with bespoke processes is it’s difficult to onboard new people, and it’s even harder to maintain in the long run. Data scientists and ML engineers continue to be among the most difficult roles to fill. Enterprise data teams could not rely on either hiring all the head count they needed nor retaining those they did have.

We recently announced a win with RealPage, the world’s largest provider of property management software. They had been looking to scale the AI services they provided to their clients, but first they needed processes and tooling that are simple for onboarding, training, and using.

And once live, they needed an automated, simple way of tracking the ongoing accuracy of thousands of live models to detect anomalies and drift. Any undetected drift could have severe consequences not just for them but for their clients’ business.

This is why they have partnered with Wallaroo. Saving nearly two thirds on their AI compute costs is a nice bonus but not why they decided to make the change from what they were doing before. It was the simplicity of our user interface for deploying, testing, and managing models, coupled with our observability capabilities to ensure machine learning keeps generating value.

Be on the lookout for some more product news coming out. We will continue to invest in the performance of our purpose-built engine, but more than anything we will continue to make Wallaroo the simplest way to manage and observe machine learning.