The embedding table is the model

Facebook open-sourced DLRM this month, a reference implementation of the Deep Learning Recommendation Model, along with a paper describing it. It’s interesting mostly because it shows what the architecture of a large production recommender looks like, which companies usually don’t talk about.

A recommender has to predict something like “will this user click on this item,” using two kinds of input. Dense features are numbers: the user’s age, how many times they’ve visited this week, the item’s price. Sparse features are IDs from huge sets: the user’s ID, the item’s ID, the ad’s ID, the categories the user has interacted with, the pages they follow. There are millions or billions of possible values for these.

DLRM handles each sparse feature with an embedding table. Every possible ID gets a learned vector, say 64 numbers. Dense features go through a small MLP to become a vector of the same size. Then the model computes dot products between every pair of these vectors, which is how it models interactions like “this kind of user and this kind of item,” concatenates those with the dense vector, and feeds the result through another MLP to get the click probability.

The interesting part is where the parameters are. The MLPs are small, maybe a few million parameters. The embedding tables are enormous. With hundreds of millions of users and items, each with a vector, you end up with tens or hundreds of gigabytes of embeddings. The paper talks about splitting tables across machines (model parallelism for the embeddings, data parallelism for the MLPs) because they don’t fit on one device. And the workload is limited by memory bandwidth, fetching the right rows from giant tables, not by compute, which is the opposite of vision models.

So in practice a large recommender is mostly a lookup table of learned vectors for every entity, with a small network on top that combines them. It memorizes a lot. Every popular item has its own well-trained vector that captures how people have interacted with it. That’s also its weakness. A brand new item or user has a random or empty vector, and the model knows nothing about it until it collects some interactions, which is the cold start problem.

I think this is where the next few years of recommendation research will go. How do you get the generalization of content understanding, what the item actually is, into a system that’s mostly built on memorizing IDs? If an item’s vector could be computed from its content instead of looked up, a new video or song would start with a sensible representation. And if items were represented in a shared space that doesn’t need a row for every ID, the table might stop being the whole model.