HLDArena
HLD-409·Search & Indexing
Expert · 2,000 pts

Planet-Scale Web Search Inverted Index & Query Aggregator

Design Google / Bing’s real-time web search serving tier combining tiered result caches, term/document-partitioned inverted index leaves, and breaking-news fresh index ingestion.

Max p99
≤ 65 ms
Availability
≥ 99.6%
Budget
≤ $11,800

Functional Requirements

  • FR-1GET /search?q={query_hash} — Serves top-10 ranked web + fresh-news results under 65ms p99.
  • FR-2POST /index/fresh — Commits breaking-news crawled documents into the real-time memory index.
  • FR-3PATCH & DELETE /search/{query_hash} — Updates, revokes, or deletes query_hash entries with bounded TTL or explicit cache/index invalidation.
  • FR-4Must survive 8x slow index-leaf stragglers via hedged/circuit-broken timeouts and multi-tier query caches.

Workload Profile

Global Search Baseline92,000 Query RPS / 7,000 Fresh Index Write RPS
Breaking Global Event Peak240,000 Query RPS / 16,000 Fresh Write RPS
Active Working Set64 GB hot query result cache & fresh posting lists
Fault Scenario8x leaf storage tail-latency brownout in Wave 3

Consistency Model

Near-real-time (< 5s) fresh index visibility; aggressive query result caching absorbs trending queries.

Relevant practice for

Click an item in Inventory above, or press B to add a component, double-click to name, C to connect.

Design Write-up

Write under ### Architecture for Diagram to take that up directly

Compulsory section · if not written yet, Diagram reads from the start of your write-up.
Architecture: required·FRs mentioned: 0/4·0 boxes · 0 edges
0 words
Hidden Cases #4–#7 run automatically on Submit
Case 1: Smoke & API ContractInput Workload
Read RPS
2,000 rps
Write RPS
400 rps
Key Distribution
Uniform (Low Skew)
Chaos Fault
None (Healthy Cluster)
Expected Pass CriteriaASSERTIONS
• Tail Latency: p99 ≤ 130 ms
• Availability: ≥ 99.6%
Valid end-to-end Client → Compute → Storage topology