Kumo AI + Databricks: Revolutionizing Predictive Modeling with GNNs and LLMs

By Min Shen and Hema Raghavan 

In today’s data-driven world, businesses need cutting-edge tools to handle vast datasets and extract actionable insights quickly and efficiently. Databricks is at the forefront of this transformation, empowering organizations to integrate complex AI models, manage diverse data types, and ensure real-time processing. It helps businesses maintain high model accuracy while managing large datasets, ultimately enhancing productivity and decision-making.

Kumo AI, through seamless integration with Databricks Unity Catalog, accelerates predictive model generation to produce accurate and actionable insights for various business applications.

Kumo AI: Revolutionizing Predictive Modeling

Kumo AI combines advanced technologies like graph neural networks (GNNs) and large language models (LLMs) to analyze and predict customer behaviors, lifetime value, and other key business metrics. Kumo works directly with raw data from Databricks without needing a fixed schema or complex feature engineering. By representing each row of data as a node and connecting primary-foreign keys, we create a graph structure that allows for efficient analysis and prediction.

Using GNNs, Kumo can learn from multiple Databricks tables stored within the Unity Catalog, producing highly accurate predictions. Kumo takes this further by leveraging LLMs such as DBRX, HuggingFace, and OpenAI to incorporate general-world knowledge into its models. The combination of GNN and LLMs delivers a more complete understanding of the data, enhancing the quality and precision of predictions.

In practical terms, Kumo offers out-of-the-box models that are typically up to 30% more accurate than baseline models. Domain experts can refine these models further to meet specific business needs, providing flexibility and customization. Within just a few hours, Kumo can generate batch predictions or embeddings that can be used for downstream tasks.

Customer Success with Kumo AI

A great example of Kumo’s impact comes from Databricks itself. Chris Klaczynski, Staff Data Scientist, and Anoop Muraleedharan, Senior Director for Marketing Technology and Analytics at Databricks, shared their experience using Kumo AI to improve lead scoring accuracy. The result? A 5.4x increase in prediction accuracy over their previous solution. With Kumo, Databricks’ sellers can focus on leads most likely to convert, driving efficiency and improving sales outcomes.

Predictive AI Workflow: Kumo + Databricks

Kumo operates directly on raw Databricks tables while utilizing Unity Catalog for data governance and security. This ensures that data never leaves the Databricks Data Intelligence Platform without being transformed, encoded, and deleted after use. This secure and scalable approach allows businesses to generate predictions reliably and at scale without compromising on data security.

By operating directly on Databricks Unity Catalog, Kumo harnesses features to create predictive models from both structured and unstructured data. This approach ensures accuracy and scalability, enabling organizations to work with terabyte-sized tables without having to sample or reduce their data.

Kumo learns from the latest data in real-time or on your chosen schedule, ensuring predictions are always based on up-to-date information. The ability to serve both real-time model inferences and batch predictions ensures businesses can get the insights they need, when they need them.

The Benefits of Kumo + Databricks Integration

Deliver More Predictive AI: Kumo makes predictive AI easier than ever. By connecting your Databricks Unity Catalog tables, you can generate numerous predictions across various business use cases. Kumo’s automated machine learning pipelines keep models updated with the latest data, ensuring predictions remain accurate over time.

Improve Model Performance: With Databricks Model Serving and support for LLMs, Kumo’s GNN models can learn from both structured and unstructured data. This boosts model accuracy, allowing data scientists to deliver even more precise predictions.

Secure Data Processing: Kumo prioritizes data security. By keeping all data within the Databricks Unity Catalog, Kumo ensures that no sensitive data leaves your environment. The use of advanced encoding methods protects your data and models throughout the machine learning lifecycle.

Unlock Insights with Deep Learning and Explainable AI

Traditional methods often fall short when trying to extract value from relational data stored in Databricks Unity Catalog. Kumo AI overcomes these limitations by using graph machine learning on raw data tables. Kumo’s GNN technology capitalizes on the relational structure of Databricks tables, incorporating additional context from LLM models supported by Databricks.

This combination of GNN and LLM models unlock the full potential of relational and unstructured data, significantly improving the performance and accuracy of predictive AI models. Moreover, Kumo provides explainable AI features that offer deep insights into how predictions are made, enabling businesses to understand the impact of different variables on the final output.

Conclusion

Kumo AI, in collaboration with Databricks, is transforming how businesses approach predictive modeling. Its unique combination of GNNs and LLMs allows for highly accurate predictions across vast datasets, without the need for complex feature engineering or schema design. By keeping data secure within the Databricks environment, Kumo ensures reliable and compliant data processing, making it a powerful tool for businesses looking to drive better insights and outcomes from their data.

With its scalability, security, and advanced AI capabilities, Kumo AI is shaping the future of predictive analytics, making it easier for businesses to extract actionable insights and improve decision-making at every level.

Start a free trial to join the private preview for Kumo + Databricks.