Book Deep Read · ML System Design Interview · Chapter 10
Machine Learning System Design Interview · Ch 10 · Ali Aminian & Alex Xu · 2023
You open a social app in the morning, and the first few things on the screen are not in time order. The people you follow posted hundreds of things yesterday; the platform picks a few dozen for you, and the order is computed for you alone, right then. This chapter is about how that order gets computed.
The lazy approach is to guess "will you tap on this?" and sort by that likelihood, high to low. Sounds fine.
Do exactly that, and a few months later your feed is wall-to-wall clickbait. The model asked "will you tap," and those headlines are precisely the ones with the highest tap rate and the deepest regret afterwards. You did tap. The model thinks it did a great job. The flaw is simple: a tap is not a liking.
It is hard because there is no right answer. Which post belongs at the top? Nobody can write that down. The same post is a welcome life update to an old classmate and noise to a stranger.
The only clue available is what you did: tapped, liked, commented, reshared, paused for a few seconds, or hit "not interested." Every one of those is real, and every one of them tells only a sliver: a like is casual, a comment means something genuinely landed, a reshare means you are willing to put your own name on it, and "not interested" means the post should never have appeared.
The chapter's move is to ask six or seven questions at once about the same post: Will they tap? Like? Comment? Reshare? How long will they linger? Will they get annoyed and swipe it away?
Each question gets its own little judge who returns a likelihood, and the likelihoods are each multiplied by a number and added up into that post's score. It works like gymnastics scoring: not one judge deciding, but several scoring separately and combining by weights fixed in advance. The "annoyed" judge carries a negative weight — it drags the score down.
The judges also share one understanding of "this person and this post," splitting apart only at the last step. That pays off concretely: resharing is rare enough that it cannot be learned on its own, but learning it alongside liking lets it ride along.
How the model trains is a technical question. What a comment is worth versus a like is a judgment call about values. Raise the weight on comments and creators start chasing "start a discussion" the next day; raise the weight on dwell time and thin, time-consuming content floats up.
The cost is just as direct: this set of weights has no correct answer. It can only be found through rounds of live experiments, and every change reshapes what every creator does.
Feed ranking is not one guess about "will you tap." It is several guesses at once — like, comment, reshare, dwell, annoyance — combined by weights fixed in advance. The model decides how accurate the guesses are; the weights decide what the product turns into.
Want the mechanisms, the notation and the diagrams? → Switch to the deep read
This chapter translates "how should a feed be ordered" into a trainable problem, and the answer is multi-objective ranking: instead of predicting a single click-through rate, train one prediction head per user reaction (click, like, comment, share, dwell time, hide / report), then combine those probabilities into a single ranking score by a weighted sum. You think it is teaching you to train a model; its real conclusion is that the model's job is accurate probabilities, and the weights' job is deciding what the product becomes — the first is engineering, the second is product judgment.
It is the second-to-last of the ten case studies in Part II. It builds on Ch6 video recommendation (the retrieval–ranking–rerank funnel) and Ch8 ad click prediction (a single CTR objective whose probability must be calibrated), then pushes one step further: Ch8 had one objective; this chapter has six or seven, and they must still collapse into one score. In the wild it maps to Facebook Feed, X's For You, and every content platform's home surface.
Start with scale. On a Facebook-sized platform, daily actives run into the billions; a single user's candidate pool — unseen posts from friends, groups and pages within the last few days — is on the order of hundreds to a few thousand, and every pull-to-refresh has to order them in tens of milliseconds.
But scale is not this chapter's difficulty; Ch6 already gave us the funnel. The real difficulty is that the objective function cannot be written down. No label anywhere says "this post belongs first." The only supervision is what users did, and every action is a biased proxy for value.
What happens if you ignore that? Look at what single-objective optimization produces: optimize clicks and clickbait plus engagement bait win; optimize dwell time and thin, padded content floats up; optimize likes and the screen fills with crowd-pleasing pablum. Worse, the whole thing is a closed loop: the model changes distribution, distribution changes creator behavior, and that new content comes back as training data. Hence the chapter exists — not to answer "how do we predict," but "what should we predict."
Step one is to decompose the fuzzy "will they enjoy this" into a set of events that can be labeled automatically from logs: explicit ones — like, comment, share, hide, report, "show me less of this"; implicit ones — click, dwell time, skip (swiped past within a few seconds). All are clean binary classifications except dwell time, which is regression (or bucketed).
One design motive is easy to miss: passive users. A large share of people never like, comment or reshare — they just scroll quietly. Train only on explicit reactions and the model has essentially no supervision for them. So dwell time and skip must join the task list to bring silent users' preferences in. That is the chapter's most practical addition to the standard list.
Now the densities: the positive rate for likes typically sits in the low single-digit percents, comments an order of magnitude below, shares another order below that, and hides / reports often only a few per ten thousand. Tasks inside one network can differ in density by three or four orders of magnitude — which is exactly why the next section shares a bottom.
Three families. Post side: the text encoded into a vector by a pretrained language model, images and video by a pretrained vision model; hashtags are numerous and churn fast, so practice favors lightweight representations like TF-IDF or feature hashing over dedicated trained embeddings (which neither learn well nor keep up with new tags); plus existing reaction counts (log-compressed) and the post's age. User side: demographics, language and region, plus context such as device and time of day.
The family that actually moves the metric is the third — author–user affinity: how often this person has liked or commented on this author before, how long the friendship has lasted, whether the author is marked a close friend, whether the post tags them. In a feed these features routinely beat architecture changes, for a plain reason: a feed is fundamentally about social relationships, not content retrieval.
The structure is straightforward: concatenate all features → several shared fully connected layers that learn a general "this person × this post" representation → one small head per task → each head emits a probability through a sigmoid, each with its own binary cross-entropy, and the total loss is the sum of the task losses.
Why share? Because sparse tasks like share and report learn almost nothing on their own; a shared bottom lets them borrow the representation that dense tasks such as click and like have paid for. That is the real reason for multi-task here — not saving machines.
The cost deserves honesty: when tasks pull in different directions you get negative transfer — the shared representation is dominated by the dense tasks and the sparse ones come out worse; and every task is welded together, so iterating on one head still means retraining the whole net. Industry later relaxed "full sharing" into "soft sharing," at the price of more parameters and training cost.
The final score is score = Σ wᵢ·pᵢ — multiply each reaction's predicted probability by what that reaction is deemed worth, and add. Three things matter.
The magnitude of the negative weight. Hide, report and "not interested" carry negative weights, and their absolute values are usually far larger than the positive ones: one report has to outweigh hundreds or thousands of likes, or negative feedback simply never registers in the sum — it is too sparse.
How the weights get set. There is no closed form. The practical path is: pick initial values by hand from product intent → run live A/B judged on long-horizon metrics (next-day and 7-day retention, satisfaction surveys) → iterate. Note the mismatch: training uses today's clicks and likes, while the verdict uses long-run retention — that weight vector is the layer that translates short-term signals into long-term value.
Whether calibration is required. Ranking cares about order, so unlike Ch8's ad scores (which get billed against) strict calibration is not mandatory. But the probabilities' scales must stay comparable, or the meaning of a fixed weight vector drifts quietly with every model refresh.
Online it is four stages: candidate generation (unseen + time window + relationship edges) → ranking (the multi-task model scores each post) → rerank → display.
The rerank layer enforces what a score cannot express: per-author caps, topic diversity, integrity demotions (misinformation, engagement bait), user-set blocks. Why not fold these into the model? Because they are rules and policy — they must be explainable, auditable, and changeable overnight. Buried inside model weights they are neither explainable nor changeable.
The soul of this chapter is entirely "what do we make the objective, and how do we combine them." The first table answers which signals can serve as training targets — watch the last column: optimizing any one of them alone breaks the product, which is the whole case for multi-objective.
| Signal | Density | What it stands for | How gameable | Alone, it produces |
|---|---|---|---|---|
| Click | high | curiosity; weakly tied to satisfaction | trivially | a screen full of clickbait |
| Dwell time | high | attention was occupied | easy (padding, slicing) | long, thin content rises |
| Like | medium | light agreement, very cheap | medium | crowd-pleasing pablum rises |
| Comment | low | something landed — or enraged | medium (outrage works) | divisive content rises |
| Share | very low | willing to endorse under your own name | hard | too sparse to cover the corpus |
| Hide / report | extremely low | an explicit negative verdict | hard (costly for the user) | usable only as a penalty term |
The second table is structural: the more the tasks conflict, the further toward soft sharing you should go — and the more it costs.
| Option | When to pick it | What it costs |
|---|---|---|
| One model per task | Few tasks, unrelated to each other, and separate teams shipping independently | Sparse tasks never learn; training and serving cost multiply by task count |
| Shared bottom + heads (this chapter) | Related tasks, sparse ones needing a lift, limited compute — the default starting point for nearly every feed | Negative transfer: dense tasks dominate the shared representation; tasks are coupled, so one head means retraining all |
| Soft sharing (e.g. MMoE) | Tasks visibly conflict ("watched to the end" and "liked" often disagree) and there is enough data | More parameters and training cost; the gates themselves become another thing to tune |
One last trade-off lives outside the model: how often to retune the weights. Too often and creators cannot keep up while the ecosystem oscillates; too rarely and the model slides toward whichever signal is easiest to game.
Saying "we'll use multi-task" is worth nothing in an interview. What is worth something is explaining three things: why share a bottom (to rescue sparse tasks, not to save machines), why the negative weight must be large (or sparse negative feedback gets drowned), and what decides the weights (offline metrics cannot; only long-horizon online metrics can). Same in an architecture review: the moment someone asks "what are we actually maximizing," the answer is that one weighted sum.
-20000 and negative feedback to -1000, public confirmation of the "one report must outweigh thousands of positives" design (defaults are zeroed in the open-source snapshot; live values live outside the repo).twitter/the-algorithm, GitHub 2023 ↗ · HomeGlobalParams.scala ↗1. In one line: feed ranking's answer is multi-objective ranking — one prediction head per reaction, then a weighted sum into a single score.
2. The real question is not "how do we predict" but "what should we predict": no label says which post belongs first; user actions are the only, biased, proxy.
3. Optimizing any single signal breaks things: clicks → clickbait, dwell → padded content, comments → divisive content.
4. Dwell time and skip must be on the task list, or passive users who never react give the model no signal at all.
5. The reason for shared bottom + heads is to rescue sparse tasks (shares and reports run three to four orders of magnitude below likes); the price is negative transfer and coupling. Escalate to MMoE-style soft sharing when tasks visibly conflict.
6. The feature family that moves the metric most is not content but author–user affinity — a feed is fundamentally about social relationships.
7. In score = Σ wᵢ·pᵢ, negative feedback needs an absolute weight far larger than the positives (X's open code allows report down to -20000), or the sparse negative signal is drowned.
8. That weight vector has no offline solution: hand-picked initial values plus live A/B on long-run retention and satisfaction. It is the layer translating short-term signals into long-term value — and it is the real product decision.