Sajal Halder

Hybrid fan-out feed design: fan-out on write vs fan-out on read

By Sajal Halder3 min read

  • System design
  • Redis
  • Cassandra
  • Java

A social feed looks simple: show me the latest posts from the people I follow. At scale it is one of the classic system design problems, because the same feature is either write-heavy or read-heavy depending on how you build it, and follower counts vary by orders of magnitude.

This article explains the hybrid approach used in the Hybrid Fanout Feed System project, which chooses between fan-out on write and fan-out on read for each author, and how PostgreSQL, Cassandra and Redis divide the work. It is a demonstration project for learning distributed systems design, so the performance figures here are design targets and not production measurements.

Two ways to build a feed

  • Fan-out on write (push)When someone posts, add a reference to that post to every follower's feed. Opening a feed is a single cheap lookup, but each post costs one write per follower.
  • Fan-out on read (pull)Store the post once. When someone opens their feed, fetch recent posts from everyone they follow and merge them. Writes are cheap and reads do the work.

The celebrity problem

Push works well until an account has millions of followers. Each of that account's posts becomes millions of cache writes, which delays delivery and loads the cache. Pull has the opposite weakness, because every reader pays a merge cost that grows with the number of accounts they follow.

Most accounts have a modest audience and a few have an enormous one, so neither strategy is right for everyone. The hybrid uses each where it is cheap.

The hybrid rule

The decision is made per author at the moment a post is created. Under 10,000 followers, the post is pushed. At 10,000 or more, it is stored once and pulled at read time. The follower count comes from PostgreSQL, a strategy factory picks the implementation, and celebrity status is kept up to date by database triggers so the rule lives in one place.

Simplified pseudocode of the decision, not the exact source
strategy = followerCount >= 10_000
    ? FanoutOnRead   // celebrity: store once, merge at read time
    : FanoutOnWrite; // regular account: push into follower feeds

Write path for regular accounts

  • Validate the post (280 characters at most) and save it to the posts table in Cassandra.
  • Read the author's follower IDs from PostgreSQL.
  • Add the post ID to each follower's Redis feed, a sorted set scored by timestamp, in batches of 100.
  • Return once the post is saved. The fan-out runs asynchronously.

Write path for celebrity accounts

  • Save the post to a celebrity posts table in Cassandra, partitioned by user ID and ordered by creation time.
  • Do no fan-out at all. That is the write amplification avoided.

Read path: merging both worlds

Reading a feed combines the two strategies in three steps.

  • 1. Cached postsRead the post IDs from the user's Redis feed with ZREVRANGE, then load each post's metadata from Redis. On a miss, fall back to Cassandra and cache the result.
  • 2. Celebrity postsLook up which celebrities the user follows in PostgreSQL, then query each one's recent posts from Cassandra using the created-at clustering key.
  • 3. MergeA feed merger combines both lists, sorts them newest first, applies the limit and returns a page with a has-more flag for infinite scrolling. Page size is capped at 100.

Choosing the stores

Each database is used for the access pattern it handles best.

  • PostgreSQLUsers and follow relationships, where ACID transactions and joins matter.
  • CassandraPosts and celebrity posts, chosen for high write throughput and time-series queries, with tables partitioned for efficient access.
  • RedisPre-computed feeds and post metadata. Entries expire after one hour by default, and the cache uses LRU eviction with a 512 MB limit.

Keeping fan-out off the request path

Fan-out for regular accounts is asynchronous, so the API can respond as soon as the post is stored, and followers are processed in configurable batches (100 by default) rather than one at a time. Together with multi-level caching and TTL-based expiry, this keeps a single post from monopolizing the system.

Targets, not measurements

The design sets these targets: feed generation under 100 ms at p95, feed retrieval under 150 ms at p95, post creation under 200 ms for regular accounts and under 50 ms for celebrity accounts. They are goals for a demonstration system. Reaching them would need load testing against real data volumes.

What a production version would still need to decide

  • What happens when an account crosses the follower threshold, since its earlier posts were delivered with the other strategy.
  • How unfollows and deleted posts are removed from feeds that were pre-computed.
  • How a Redis feed is rebuilt after it expires or is evicted.
  • How to scale the fan-out workers as follower counts grow.