Data Locality and Edge Computing

Light in fiber travels at roughly two-thirds the speed of light in a vacuum, due to the fiber's refractive index — about 200,000 km/s. Sydney to Virginia is a great-circle distance of roughly 16,000 km one way; a round trip at that speed alone takes about 160ms, before any server processing, TLS handshake, queueing, or actual computation happens. This is a propagation floor: a physical constant, not an engineering shortfall, and it cannot be reduced by a faster server, a better database, or more efficient code at the origin.

This matters because teams often try to solve a distance problem with a compute problem — optimizing the origin server's response time when the origin server was never the bottleneck. If the propagation floor alone exceeds the latency budget, the only lever left is reducing the distance the request has to travel — moving computation, data, or both physically closer to the user. This is the entire premise of edge computing: not that edge servers are faster than origin servers, but that they're closer.

The corollary matters just as much: for data or logic that genuinely must be globally authoritative (a real-time inventory count, an account balance), no amount of edge placement helps, because correctness requires reaching the one place that knows the true, current answer — the latency floor for that specific request is unavoidable, and the honest fix is budgeting for it, not architecting around it.

Edge caching (a CDN) serves a previously-computed, cacheable response from a point of presence near the user — it eliminates the round trip entirely, but only for responses that are the same for many users and can tolerate being slightly stale. It does nothing for GET /api/personalize?userId=u-501 — that response is unique per user and can't be cached against a shared key.

Edge functions (Cloudflare Workers, Lambda@Edge, and similar) run actual application logic at a point of presence close to the user, which helps with per-request computation — but if that function still has to call back to a centralized origin database for the data it needs, the propagation floor comes right back, just wrapped in a function instead of a direct API call. An edge function only removes the round trip if the data it needs is also local.

Edge data replication solves that: a subset of data relevant to a region is replicated out to regional or edge-adjacent stores, so an edge function can read locally without ever crossing the ocean. Combined with an edge function, this is the pattern that actually closes the gap for a request like this one — compute and data both move closer to the user, not just one of them.

Replicating data to the edge almost always means eventual consistency — the same trade-off covered in strong vs eventual consistency: a regional copy lags the source of truth by some window, and a personalization model retrained centrally takes time to propagate to every edge location. For recommendation data, that lag is usually an acceptable trade for latency; for a real-time inventory count gating checkout, it usually isn't, and that data has to stay origin-authoritative regardless of the latency cost.

Storage and operational cost multiply with every additional location data is replicated to — a dataset replicated to fifty edge regions costs roughly fifty times the storage of one, plus the ongoing replication pipeline itself. This cost has to be weighed against the actual latency benefit for the specific data in question, not applied as a blanket "replicate everything everywhere" policy.

Data residency is the trade-off most likely to be missed entirely, because it's a legal and compliance constraint, not a technical one: some data (particularly EU personal data under GDPR-style rules) is legally required to stay within specific jurisdictional boundaries. A replication pipeline built purely to optimize latency, without a corresponding residency-aware filter, can silently violate that requirement the moment it starts replicating regulated data to edge locations outside the permitted region — exactly the failure traced below.

Trigger. To hit the sub-100ms personalization target, the team builds an edge-replication pipeline that copies each user's profile and recommendation context to the nearest edge region — a purely latency-motivated design, built and shipped before any residency requirement existed for the product. Symptom. Months later, a compliance audit triggered by a new EU regulatory requirement finds EU users' personal profile data present on edge nodes in the US and APAC regions — data that was legally required to remain within EU infrastructure.

Root mechanism. The replication pipeline's design was "replicate every user's data to every edge region nearest their recent traffic," with no concept of jurisdiction attached to the data at all — it optimized purely for latency and had no residency dimension in its design, because residency wasn't a requirement when it was built and nobody revisited the pipeline when it became one. The pipeline was working exactly as designed; the design itself was the gap.

Evidence. Edge-node data audit logs show EU-origin user IDs present in the us-east and ap-southeast regional stores, with replication timestamps going back to the pipeline's original launch — this wasn't a recent bug, it had been true the entire time. Recovery: purge the non-compliant replicas immediately, notify legal/compliance to assess breach-notification obligations, and audit every other edge-replicated dataset for the same gap. Prevention: tag every dataset with its residency requirement at the source, enforce that tag as a hard filter in the replication pipeline itself (not a downstream check), and require a compliance review as a standing step before any new edge-locality initiative ships — not an afterthought triggered only by an external audit.

Start from what the specific response actually needs, not a blanket edge-everything or origin-everything policy. Fully shareable, cacheable responses go to a CDN. Per-request logic that can tolerate a real, bounded staleness window goes to an edge function paired with regionally-replicated data. Anything requiring a globally authoritative, immediately-consistent answer — a real inventory count, an account balance, anything fault tolerance vs high availability would call loss-intolerant — goes to the origin, and the latency cost is accepted honestly rather than engineered around.

Evaluate every edge-replication decision against three dimensions together, not latency alone: the consistency lag the data can tolerate, the storage/operational cost of replicating it to N locations, and — critically — its residency or compliance requirements. A design that only optimizes the first dimension is exactly how the failure above happened.

Treat data residency as a design-time filter baked into the replication pipeline from the first version, tagged at the data's source, not a policy applied after an audit finds the gap.