Authors: Vanja Josifovski, Ivaylo Bahtchevanov

This article describes the motivation for building the Kumo platform and our vision for transforming how enterprises make decisions.

Current State of the Data-Driven Enterprise

Today, when enterprises say they are “data-driven,” they primarily rely on a backward-facing approach for making decisions. The role of data and analytics teams is to aggregate historical data to pull out insights and then use those learnings as a proxy for understanding the future.

Having a human interpret the data is necessary to form an opinion on the state of the future. This human-in-the-loop approach is very expensive and error-prone, further exacerbated by the cost of reactive decision-making. If you compare the enterprise to a car driving forward, then the driver would be looking entirely at the rear-view mirror, hoping to infer the road ahead while making the next steering decision.

A better approach for making decisions would be to build a set of models to predict all relevant future outcomes based on possible actions and determine the best decision. If this were easy, everyone would be doing it. Setting the foundation to perform thorough predictive decision-making effectively at scale is incredibly difficult for several reasons.

For one, the problem formulation often needs to be more intuitive and error-prone. The business problem is very different from the AI-defined objectives. There has to be a translation layer that requires significant experience and domain knowledge. If you define a sub-optimal objective, optimization can be more harmful than beneficial to the downstream decision-making process. Defining the ideal objective also requires trial and error; however, agile iteration is often not possible within the time frame needed to make a decision.

The next set of problems is a result of how delicate and brittle the remainder of the delicate process is. Once you have an objective function, you then need carefully curated and problem-specific features engineered from clean data. Data processing and feature engineering are prone to introducing noise, which is mistaken for signal and degrades model quality. Mistakes here can introduce information leakage, which renders the model useless. The evaluation process can also be misleading if the target validation is done on the uninterested cohort. The inference might be impossible if the target has missing feature values (which is often the case).

To build a predictive model, you need to build a data pipeline, apply domain-specific feature engineering, and build, validate, and deploy a problem-specific model - for each potential outcome. This approach is expensive and takes a long time to productionize - each additional prediction requires significant effort and resources. Consequently, the time it would take to properly model the right scenarios for just one specific problem has a much greater time horizon than the time needed to make the decision. To do this for many problems across a department or organization is even worse.

Introducing Kumo.AI

These challenges are the reason we built Kumo.AI. Our vision is to give enterprises a single platform to query the future for any problem they want to solve and ultimately to operate with their data as an autonomous vehicle would operate with inputs across all sensors and cameras.

Imagine being able to ask, “which product improvement is likely to yield the best outcomes?” or “which products should I recommend to each user” and receiving a sequence of actions that represent the best possible set of decisions based on future outcomes. Kumo.AI makes it possible for organizations to operate with their data autonomously - you point to your data, specify a business objective, and are pulled into an ideal future state.

Our goal is to not only have the driver understand the road ahead but also to take their hands off the steering and allow the car (enterprise) to drive along the best possible route autonomously. With Kumo.AI, our vision is to introduce the concept of the autonomous enterprise - characterized by an ability to proactively learn and adapt to changing situations impacting the business landscape, taking new inputs, and charting a clear course based on the desired final destination.

One might ask, what will be the role of human intelligence in the enterprise of the future?

In the case of self-driving cars, the destination is decided by the driver - with enterprises, strategic direction steers towards that destination. For an organization looking to understand the future, there’s a continuous spectrum between the near-term predictions, which require a high level of accuracy for very precise and tactical decisions, and the long-term strategic predictions, where a broader set of AI capabilities are needed to understand many complex variables over a longer horizon. The autonomous enterprise would enable both, but the largest shift would be in the latter - where leadership is empowered with capabilities not currently available in any data toolset.

So how does an enterprise get there?

Democratizing Predictions at Scale

The first hurdle is to make it easy to run predictions whenever you need them. Kumo.AI’s interface defines a new paradigm that lets the user write a task-driven declarative query that is automatically transformed into an ML model that predicts the future. Using this interface, users can write as many queries as they need - no additional overhead of pipelines, infrastructure, and model development for each subsequent query.

Examples of such queries include:

“Which users will churn at a given timeframe if I perform the following actions…”
“What does my LTV for each user look like if I perform the following actions…”
“Which accounts should the sales reps focus on to maximize revenue for the next three months”
“Which accounts should a specific sales rep focus on to maximize closing percentage?”

You can read more about the technology and design of Kumo. AI’s platform and how it enables almost instant predictions of the future here. Kumo eliminates the need for any feature or label engineering and the need to build infrastructure for data pipelines. The revolution is to bring true representation learning using deep learning over raw data to reach the final predictions. To learn more about the effectiveness of Kumo.AI’s models, you can read our blog on GNNs here.

Being able to query the future is equivalent to the driver of the car now looking forward rather than backward and making a decision using a combination of common sense, intuition, and the image of the road ahead.

This sounds more logical than the rear-view mirror approach - but we can take this a step further. Suppose the car can systematically capture each relevant potential scenario ahead. In that case, we can codify these learnings into a system that performs scenario planning and hones in on the best path every time. Think of this as a driver-assist with a forward-looking driver.

So what does this look like in practice?

Using Predictions to Enhance Decision-Making

If you can project a given scenario with a clear set of parameters, the next step forward is to run through the parameters to identify optimal ones. This is equivalent to running a what-if analysis, except rather than using spreadsheet formulas, you use cutting-edge AI models to perform the inference.

The user specifies a target business objective, runs through a series of queries defined within Kumo.AI UI and picks the action with the best outcome. For example, suppose the goal is to maximize retention. In that case, Kumo.AI allows users to specify queries to calculate the expected churn, LTV, and future purchases based on a set of actions (sales outreach, sending notifications, showing ads, curated content or product recommendations) and a set of target timeframes for each action, and calculate the expected uplift based on each set of actions at given times, and select the best path forward. Kumo. AI’s explainability will help identify the leading indicators of churn and the specific attributes/actions that separate the high spending from the low-spending customers - providing additional context to improve retention. The user can optimize their decision-making using an experimentation framework, balancing exploration/exploitation in a changing future data distribution - by changing the what-if conditions until an optimal future utility is found.

Under this paradigm, the role of data practitioners will change to enhance their impact and align their skills with what they do best. Analysts, who are best suited to ask questions on the data they have access to, can now ask questions about the future. Analysts can perform tasks once only capable by ML teams across all company functions. Given they already have access to data, they can unleash predictive capabilities without any infrastructure or pipelines and enhance the decision-making of the entire business unit. Meanwhile, existing machine learning and data science teams can streamline and focus their efforts on the top priorities of the organization.

From Driver-Assist to Fully Autonomous Enterprise

The next step is to codify this framework into a fully autonomous engine that pulls the car forward without human intervention. In an autonomous vehicle, the software takes all inputs and predictions, makes the best decision at a given point in time, identifies a probabilistic set of outcomes and future states which serve as the input to the next set of predictions, and charts a path forward according to each successive step.

Once Kumo.AI becomes an integral part of the data fabric of an organization, the predictions powered by the platform serve as part of a continuous discovery and execution flywheel - each decision results in an update over the data representing the problem space, which then serves as input for the next action. This enables teams to specify a business problem directly as an optimization problem. If the goal is to reduce churn over a particular set of customers, then Kumo.AI can kick off an iterative process that runs through a sequence of actions with corresponding subgoals. For any given problem space, there is typically a finite series of possible relevant actions that can be applied, so given the experimentation framework discussed in the previous step, we can fully automate an exploration/exploitation optimization by running through all possible decisions and corresponding scenarios and charting an optimal course.

Explainability in this self-driving mode becomes a powerful tool for understanding the key drivers and levers of your business. By running through all relevant scenarios and compiling insights on why certain actions were more successful than others, you have a front-row seat into the behavioral patterns of all of your users - the equivalent of running rapid-fire experimentation and AB testing without having actually to run any experiments on your users. For example, if you want to improve overall product retention, the system will show you outcomes for sending different notifications, providing customer outreach, showing ads or recommending new products, and a visual of which factors contributed to user churn or increased revenue across the different scenarios. At Kumo.AI, we have developed a proprietary explainability technology that relates the predictions to individual events or data points in the raw data. This is the ultimate and most precise way to explain why the model predicted certain outcomes.

Today, a “data-driven” enterprise refers to humans making decisions by using large-scale analytics. Using AI for strategic decisions is rare and only used in a small subset of important areas because it is extremely difficult to do well. This is the big opportunity that we see in the coming years. With the advent of new technologies such as graph neural networks, enterprises can connect predictive analytics across all of their processes and data sources and make useful predictions available across all functions. These revolutionary changes will result in a shift in organizational structure, enabling companies to adapt and respond quickly, create more value globally, and become leaders in their respective industries.

While these changes will not happen overnight, we at Kumo.AI are building the product that will allow forward-looking enterprises to start this journey today! If you’d like to join us on this journey and begin the transformation, request access to our platform here!

Cookies settings