9.Splitting the Catalog Across Machines

The catalog survives machine failures now, and reads are fast. But write volume keeps climbing — more sellers, more listings, more updates — and every single one still has to go through the same one primary. Replication protected GreenMart's data. It never made a single primary write any faster.

Sarah needs a different kind of split now — not more copies of the whole thing, but the collection itself, divided.

This is the exact partitioning idea from early in the course, with MongoDB's own real mechanism behind it: sharding. A sharded collection is split across multiple machines — shards — each one holding only a slice of the documents, so no single primary has to absorb every write for the whole collection.

Which slice a document belongs to is decided by its shard key — one or more fields chosen when sharding is enabled, and genuinely hard to change later once a collection has grown large and live. MongoDB tracks ranges of shard key values as chunks, and a background process called the balancer keeps redistributing chunks so no shard ends up holding far more than the others.

Here's exactly what a bad shard key looks like in practice: a monotonically increasing field — a timestamp, or an auto-incrementing counter. Every new document's value is always higher than everything inserted before it, so every new write lands on whichever shard currently owns the highest range — one hot shard doing all the work while the others sit idle. sellerId avoids this because GreenMart's sellers themselves are spread out, not steadily increasing in one direction.

One more real, practical thing worth carrying forward from this whole Act: schema evolution stays cheap in the sense that adding a new field to new documents needs no migration — but old documents won't have it automatically, so application code has to handle a missing field gracefully. Flexibility doesn't erase that work; it just moves it from a database migration to a few lines of defensive code.

And the seller name and rating Sarah embedded into every listing, back in this Act's second chapter? That has a real name too: the Extended Reference Pattern — embedding just the few frequently-needed fields from a related document, instead of the whole thing, to avoid a lookup on the common read. Naming patterns like this — Extended Reference, or the Subset Pattern (embedding only the most recent slice of an otherwise-unbounded list, referencing the rest) — turns "this felt like the right call" into a repeatable move GreenMart can reach for on purpose next time. The anti-patterns are just the same lessons said backward: unbounded arrays embedded without a cap, documents that grow without a natural limit, and shard keys that increase in one direction — every one of them something this Act already watched go wrong once, concretely, before it got a name.