Using Graph Learning for Personalization

How GNNs Improve Performance and Structure of Recommender Systems

Authors: Dylan Sandfelder,  Ivaylo Bahtchevanov

Personalization systems are one of the most common applications of machine learning, essential for fueling growth in the modern economy. An intelligent and effective personalization engine ensures users have an optimal customer experience while maximizing overall revenue. However, common approaches used in practice today tend to fall short in their ability to generalize well, and often don’t take full advantage of the underlying structure of data. This blog explores different effective recommendation engines in practice and explains why Graph Neural Networks have become the best tools for personalization at scale. 

Applications of Personalization

Most recommendations, promotions, and advertisements people encounter on a daily basis are the result of many complex data pipelines that transform consumer behaviors into targeted predictions. Personalization systems are very effective at delivering tangible business value through predicting potential churn, forecasting product sales, analyzing user engagement and preferences, and more. Here are a few specific examples of existing personalization systems used by large companies today and how they integrate machine learning models into their pipelines.

Online Platforms and Marketplaces

Zillow, an online real estate platform connecting buyers and sellers of properties, provides a highly personalized and guided search for every user. With over 135 million listed properties, over 48.5% of all real estate web traffic, and with over 50 filters available to choose from, narrowing a search to the most relevant property can be very challenging. Zillow offers users a feature called search refinements – pre-selected buttons that suggest the most relevant categories for customers given the context, ranked by predicted relevance to the user. The personalization engine learns from user behavior in real-time and combines with observed preferences from the past to recommend the properties that will most likely be valuable to the users.

Retail and Ecommerce 

The global clothing brands like H&M and Zara heavily rely on recommendations in fashion. They have both an in-person retail experience as well as an online experience that they tailor to the different types of visitors. To create an ideal in-person experience, they use analytics to identify the right apparel for each retail location given the geographical demographics of local buyers, the latest fashion trends (which are in constant flux), unique preferences of the types of people the location attracts, and more. In 2018, H&M used AI to reverse one of their biggest sales slumps in retail history – they shifted their strategy from stocking all stores globally with the same merchandise to personalizing each store to its respective geographic location. They equip their merchandising team, a team of designers tasked with understanding what shoppers want to buy, with algorithms that analyze store receipts, returns, loyalty data, and behavior. Rather than taking a one-size fits all approach to design clothes and stocking stores, their team focused on tailoring inventory to the preferences of the types of shoppers in the area. 

With this approach, they predict overall demand across each category for the stores, which in turn optimizes their production, supply chain, and design pipeline to more efficiently get ahead of the target sales for the following period. Analytics will also determine how to best position and price each item within the store to maximize consumption for the target buyer.

To optimize the online ecommerce experience, they leverage factors like search history, recent purchase, and different interactions that would help identify new trends and behaviors. A user’s online activity helps create a user profile that can be used to recommend specific products that they are most likely to purchase. The end result is a unique homepage per user where they see only the most relevant products.

Media

Recommendations drive the majority of the overall viewership of YouTube – even more so than channel subscriptions and search. Having one of the largest and most popular collections of content, YouTube provides each user with a unique, highly curated and personalized catalog of videos in real-time. They collect over 80 billion signals on users – including factors such as watchtime, clicks, scrolls, shares/likes/dislikes, previous searches, and more – to dynamically update any given user’s recommended content to maximize the likelihood they’ll find value in the content and minimize the probability of seeing something they dislike.

Challenges and Limitations of Existing Approaches

Traditional approaches for building personalization engines can range from rules based approaches – which directly map attributes of items to user-specified preferences as filters – to combinations of content and collaborative filtering – which learn similarities between users and items –  to deep neural networks that learn latent preferences of users to rank the top candidates of items to recommend. All of these approaches are fundamentally and structurally limited in how well they can accurately perform and scale across a massive user base. Below are some of the major limitations on these approaches.

Models require fixed data input, which ignores the underlying relational structure in the data 

Traditional models, from rules-based systems to complex neural networks, require the data to be preprocessed and stored in static tabular form in order to train. This transformation enforces dimensional limitations on the representation of that data, which discards most of the structural, temporal, and contextual information. The model then operates on each input as a completely separate and independent entity – this disregards the complex interactions between entities which often contain meaningful signal towards the prediction problem. In order to maximize the signal leveraged from the complex structure of the data, a more flexible and nuanced representation is needed.

Models are extremely sensitive to new data

When a new entity joins the network, there is very little historical data on the entity’s previous behavior and interactions. The model has yet to meaningfully capture the entity’s profile and preferences, so the introduction of a new input could potentially create significant noise. Because the current approaches aren’t designed to capture the entirety of the relationships between entities in the data, new data points will always appear as outliers in this system. As a result, they are incredibly sensitive to changes in the input. This sensitivity could potentially create a chaotic user experience for any new users to a platform (and consequently any downstream users that then interact with them). Ideally, you want your model to take each new data point and treat it differently based only on the information known at a given point in time.

Models are rigid, with limited adaptability to new predictions or use cases

Each new query or prediction requires an entirely new data structure, pipeline, and model to power the prediction. Creating new data pipelines, setting up the infrastructure, and building new models every time you have a new problem to solve is expensive and time consuming. This limits the number of experiments a given company can perform, and forces teams to adopt more generic models over highly personalized ones.

Graphs – A Modern Approach to Personalization

Personalization problems are best understood and represented by graphs. Each problem can be broken down into a network of entities, which include users and the potential items they might buy or interact with, and the interactions of those users.

Graphs capture the relational properties that exist naturally in the data and represent a complex network of entities and links, where each entity exists in the context of its connected components. By capturing the structural information, graph models can make predictions with very little data on a given entity, which is very useful for minimizing false positives and tackling the imbalance label problem. Graph models require significantly less feature engineering than traditional ML because they are designed to operate on the raw data and learn the structural properties and interactions of the network automatically.

Why Graph Neural Networks

Graph Neural Networks (GNNs) address the issues mentioned above and are typically the most effective tools for building personalization and recommendation engines. GNNs are designed to operate on graphs the same way convolutional neural networks (CNNs) are built to operate on images. They learn node representations by repeatedly aggregating the features and embeddings of each node’s neighborhood through a process called message passing. GNNs improve on previous methods in the following ways:

GNNs use the entirety of the data’s underlying structure to generate embeddings

GNNs encode the entire graph – i.e. the entities and their diverse relationships – directly in embeddings. Doing so captures the full context of the network without making any assumptions on the structure. An embedding for any given entity doesn’t just contain information about the interactions and relations for that node – it also encodes information about connections with entities that are similar to it, as well as information on entities that are very disparate. By storing this information, the embedding maximizes signal for any given node. A traditional model would require extremely high dimensionality to capture the same information, which is extremely computationally and spatially intensive, and in most cases far from feasible.

Models generalize better to new data

As mentioned above, each embedding offers contextualized information of the entire network – this is particularly beneficial for making predictions on new data points coming into the network. When a new entity is added, a graph model is able to perform inference on it despite the lack of historical information.For example, when a new user joins a platform, the model doesn’t have any record of previous transactions or purchasing habits, but a GNN can take content-specific information that can relate the node to other entities on the platform and make reasonable inferences on the node. 

In the case of AirBnB, which uses personalization to match hosts with users looking for a place to stay, when a new host signs up, GNNs can still perform very effective recommendations for the newly registered host despite a lack of data on previous stays, searches, and interactions from that host. By understanding latent content on the new host as well as other connections on the platform, GNNs can quickly deduce information about the host based on other users on the platform.

GNNs can run many different queries on the same graph

In traditional machine learning, each new model requires partitioning and isolating the relevant data, preprocessing that data, feature engineering, tuning parameters, and training a new pipeline. Often, to answer a new question means rebuilding the end-to-end pipeline to suit the use case and relevant prediction. It is also essential to only include the relevant data so preprocessing also entails partitioning the data based on the model.

GNNs on the other hand, take in the entire data to construct the graph. There is no need to partition the data or build a new pipeline – inference is performed across the entire graph in a very computationally and spatially efficient manner. 

For example, if your first GNN model predicts which users are likely to churn off the platform, you can then make subsequent predictions on which actions those users will likely respond positively to and then predict which items they are likely to purchase – you can combine these predictions to create a retention strategy for identifying users who are likely to churn, identify the most relevant products to recommend, and then position the outreach in a way that will be most effective.

GNNs scale efficiently to massive data

GNN design utilizes something called a message passing framework in which each layer of the model runs through all graph node embeddings simultaneously. Large datasets can be ingested directly and then the graph can be trained on the entirety of the data. 

To learn more, you can read our deep dive on GNNs here.

Graph-Based Personalization in Practice

Companies with an established data science practice where data is at the heart of their product experience have started adapting GNNs as the core to their recommender systems. Here are some examples of companies that have improved performance from making this shift.

Spotify

Spotify is one of the most successful examples of recommendation and personalization in industry. Most people are intimately familiar with the song recommendations they provide each user in the form of radio channels, daily mixes, “this is..” artist-specific playlists, recommended songs playlists, and other forms of curated content that is unique to each individual listener.

In the 2019 ACM Conference on Recommender Systems (RecSys) in Copenhagen, Spotify revealed the impact of transitioning their core recommendation system to a graph learning approach. The representation of their data as a heterogenous graph and the use of GNNs helped them create and curate a knowledge graph that generated highly effective embeddings for content. These embeddings had a 360 understanding of both the local and the global connectivity of users and content. Rather than simply training a model optimized on instantaneous reward functions maximizing clicks, plays, and impressions, GNNs were able to learn the complex, nuanced “audio taste buds” of their users to provide highly curated playlists and suggested audio content, from music to podcasts. This new approach saw substantial performance gains in long-term user engagement and user experience.

What’s more, Spotify is using GNN-trained embeddings on the entire graph to go after new problems. Spotify expanded their offerings to include podcasts, and they were able to re-use the same embeddings trained on music to also create a curated podcast playlists for each user, despite not having any podcast-specific historical data. Similarly, they began providing personalized recommendations for concerts and live events. By using the same graph and a GNN-based pipeline, Spotify was able to rapidly experiment and deploy new features with minimal engineering resources and time.

Pinterest

One of the greatest and most fundamental values to Pinterest’s business model is the ability to make visual recommendations based on taste. They do this effectively by taking into account the context added by hundreds of millions of users, and then help people discover new ideas and products that best match their interests. When Pinterest hit a critical scale (~300M MAU and >100B content objects), they needed to augment their recommendation approach to take advantage of the massive amount of users and content on their platform. 

As a content discovery application, Pinterest offers something called a Pin – a visual bookmark used to capture online content (recipes, clothes, designs, products, etc.) onto personal boards, which reflect that user’s preferences. Pinterest modeled their problem as a graph capturing interactions between Pins and boards, incorporating graph learning models PinSage and PinnerSage in 2018. This update defined their state-of-the-art recommendation engine that scaled across their colossal user base. Treating their data as a graph allowed them to scale to over 10,000x larger datasets while performing recommendations on both visual aspects as well as topological similarities simultaneously. By embedding the entire graph, Pinterest created high-quality embeddings that borrow information from nearby nodes (ie Pins) – creating a more robust and accurate representation for each Pin and board. 

A/B testing demonstrated that their new approach improved their target metric – repining rate – by over 30%, as well as an increase in impressions of their other products by 25%.

Finally, this approach allowed them to build many revenue driving products on top of the graph. The Pin embeddings became essential to Pin recommendations for users, but they became the basis for other key services including Related Pins, Search, Shopping, and Ads.

Uber Eats

Food delivery industry is defined by fierce competition – customers are used to convenience and choice. With over 320,000 restaurant-partners in over 500 cities globally, Uber took a bet on personalization to gain a competitive edge – to make the user experience more seamless and easy to navigate, Uber shows users the dishes, restaurants, and cuisines they are most likely to choose upfront. The data science teams needed a robust recommendation system that can quickly and accurately boost restaurant and food choices to the top of each user’s home screen. 

The marketplace is modeled as a bipartite graph – where one graph relates users to the types of dishes they are inclined to order, while the other graph maps users to the restaurants they are likely to order from. 

The model GraphSAGE is trained across this graph to understand the taste buds and unique preferences of users and then learns contextual similarities between users, restaurants, and specific dishes. This approach predicts how often a user will order from a given place and what types of meals they are likely to order, achieving high performance and significantly improving Uber Eats’ order rate across all locations.

GNNs are particularly useful when a new user or a new restaurant joins the platform and there is little to no order history or transaction record. The model leverages the connectivity of the graph to infer potential matches based on a likely taste and preference of a given user, or the likely demographic a new restaurant is likely to attract based on similar places.

Applying Graph Solutions to Personalization with Kumo

We’ve seen how using graphs and GNNs can substantially improve the performance of personalization and recommender systems, but we have yet to discuss implementing them in practice. Creating the graph from raw data, finding a way to efficiently store and represent the graph, and building a GNN that can adequately learn from the data and make inferences from the graph representation is not a trivial task and requires considerable time and resources in the form of a large data science and engineering team. Often, the process requires steep involvement from domain or subject matter experts. 

For organizations with established data science functions and discipline, there exist open source tools such as PyTorch Geometric (PyG) which provide a standard set of APIs and interfaces for building GNNs using familiar design principles. 

For organizations who either don’t have considerable time and resources to dedicate, or who want to minimize the investment to start making predictions immediately, Kumo presents the fastest time-to-value solution in the market. Kumo streamlines the entire process and allows the user to directly write queries to make predictions after connecting to a raw data source. Once connected to your data, Kumo automates the entire end-to-end machine learning pipeline – from assembling the graph from raw data, to finding the best model for the specific task, to training and optimizing that model with the best possible set of parameters. The end user can make any number of predictions without any additional overhead or engineering effort. 

Whether you have a well-established data science function and ML pipelines already tackling personalization, or you are looking for new ways to build and develop an engine, Kumo can turbo-charge your platform with the best graph-based models and deliver the capabilities of an entire data science team directly via the interface.

If you want to get in touch with Kumo, visit our contact page here and learn more about how Kumo can help your business solve its data science challenges.