Data Gravity & Vendor Lock-In
Once petabytes of data and years of tooling accumulate around one cloud provider, moving it becomes structurally hard — not just expensive.
Overview
Data gravity is the tendency for compute, services, and new features to cluster around wherever large volumes of data already live — because moving that data is slow, costly (per the previous chapter's Egress FeesCharges a cloud provider bills for data leaving their network — often a hidden, easy-to-underestimate cost at scale.Learn more), and operationally risky. Over time, this same gravity makes migrating away from a cloud provider structurally harder, not just more expensive: vendor lock-in.
Why It Exists
Once petabytes of data live in one provider's storage, every new service built to use that data gets built where the data already is — it's simply the path of least resistance and lowest latency. Data gravity exists as a natural consequence of egress cost and latency, not a deliberate trap, but it produces the same practical effect as one: the more successful a system becomes on a given cloud provider, the more expensive and disruptive it becomes to ever leave.
Real World Example
A company starts with a modest dataset in AWS. Over five years, that dataset grows to 10 petabytes, with dozens of internal services, machine learning pipelines, and analytics tools all built directly against AWS-specific APIs (S3, Redshift, SageMaker) because that's where the data already lived. A proposal to move to a cheaper competing cloud now faces not just the egress cost of transferring 10 petabytes, but months of re-engineering every service that was built assuming AWS-specific behavior — the data's gravity made the whole platform effectively immovable.
How Lock-In Actually Accumulates
Data Gravity — the Root Cause
New services get built next to existing data because that's fastest and cheapest, not from any strategic decision to commit to a vendor — lock-in accumulates as a side effect of ordinary engineering decisions.
Proprietary APIs and Managed Services
Using a provider's managed, higher-level services (e.g. a proprietary database or ML platform) instead of portable, open-source equivalents deepens lock-in — the more built directly on vendor-specific APIs, the more must be rewritten to leave.
The Egress Cost Compounds the Data Cost
Beyond re-engineering effort, physically moving petabytes of data out incurs exactly the egress fees covered in the previous chapter, on top of whatever it costs the destination provider to receive and store it.
Mitigations: Abstraction Layers and Open Standards
Using cloud-agnostic tools (Kubernetes over a provider's proprietary container service, open table formats like Parquet/Iceberg over a proprietary data warehouse format) keeps a path to migration open, at some cost in convenience versus fully embracing one provider's tooling.
Diagram
Data gravity pulls new services toward existing data, compounding lock-in over time
Data accumulates in Provider A
years 1–2
New services built against Provider A's APIs
years 2–4 — path of least resistance
Migration cost: data + re-engineering
year 5 — now structurally hard
Common Mistakes
Treating vendor lock-in as a one-time decision made when first choosing a cloud provider
Why: Lock-in mostly accumulates gradually afterward, through years of ordinary engineering decisions to use whatever managed service is fastest to integrate with the data already in place.
Fix: Periodically evaluate how deeply new services depend on provider-specific APIs, not just the initial provider choice.
Adopting a multi-cloud strategy purely to avoid lock-in, without accounting for data gravity
Why: Spreading data across multiple providers to 'avoid lock-in' can add ongoing egress and complexity costs (from the previous chapter) that outweigh the flexibility gained, if the data never actually needs to move.
Fix: Weigh the real, demonstrated likelihood of needing to migrate against the ongoing cost of maintaining multi-cloud portability.
Assuming open-source-based managed services (e.g. a managed Postgres) carry the same lock-in risk as fully proprietary ones
Why: A managed version of an open-source technology is generally far more portable than a fully proprietary, provider-specific service, since the underlying technology and its data format are the same everywhere.
Fix: Distinguish between 'managed open-source' (lower lock-in) and 'proprietary' (higher lock-in) services when evaluating architecture decisions.
Interview Questions
What is data gravity, in plain terms?
It's the tendency for new services and compute to get built near wherever large volumes of data already live, because moving that data is slow and costly — over time, this naturally pulls more and more of a system toward a single location or provider.
How does data gravity lead to vendor lock-in, if no one ever makes a deliberate decision to lock in?
Each individual engineering decision — building a new service against the data that's already in one provider, using that provider's managed service because it's fastest to integrate — is locally reasonable. But collectively, these ordinary decisions accumulate into a system so deeply tied to one provider's data and APIs that leaving requires both migrating the data and re-engineering every service built against provider-specific behavior.
How would you architect a new system to limit vendor lock-in without sacrificing the velocity benefits of using a cloud provider's managed services?
Favor managed versions of open-source or open-standard technologies (managed Postgres, managed Kubernetes, open table formats like Iceberg/Parquet for a data lake) over fully proprietary services wherever the managed open-source option meets the requirement — this captures most of the operational convenience of a managed service while keeping the underlying data format and APIs portable. Reserve proprietary, fully vendor-specific services for cases where they provide a genuinely unique capability that justifies the lock-in, and revisit that trade-off periodically as the system and its data grow.
Production Best Practices
Do
✓Periodically evaluate how deeply new services depend on provider-specific APIs.
✓Favor managed open-source/open-standard services over proprietary ones where they meet the need.
✓Weigh multi-cloud portability costs against the real, demonstrated likelihood of needing to migrate.
Don't
✗Don't treat cloud provider choice as a one-time decision — lock-in accumulates continuously afterward.
✗Don't adopt multi-cloud purely to 'avoid lock-in' without weighing the ongoing egress/complexity cost.
✗Don't assume all managed services carry equal lock-in risk — open-source-based ones are far more portable.
Comparison
| Accumulates How | Migration Cost Driver | Mitigation | |
|---|---|---|---|
| Data Gravity | New services built near existing data | Re-engineering + data transfer | Open standards, periodic review |
| Vendor Lock-In | Deepening use of proprietary APIs | Rewriting provider-specific integrations | Managed open-source over proprietary |