Fallacy Latency Is Zero

Core Idea

“Latency is zero” is the second of the Fallacies of Distributed Computing—the false assumption that network calls between services happen instantaneously. In reality, every network call introduces measurable latency (typically 1-100ms or more), which accumulates across service boundaries and fundamentally changes how distributed systems must be designed compared to in-process method calls.

What Is the “Latency Is Zero” Fallacy?

The fallacy assumes that calling a remote service is as fast as calling a local method. In-process calls complete in nanoseconds; network calls operate on a completely different timescale:

• Local method call within a single process: 1–10 nanoseconds
• Network call to a service in the same data center: 1–10 milliseconds — roughly one million times slower
• Cross-region call (e.g., US East Coast to Europe): 50–150 milliseconds
• Congested or overloaded network/service: seconds or timeout entirely

The fallacy becomes catastrophic when architects simply replace in-process calls with remote API calls without accounting for this difference. A web page making 20 downstream service calls at 50ms each introduces 1,000ms of network latency alone — before any business logic executes. Under load, threads block waiting for slow responses, connection pools exhaust, and the system becomes unresponsive despite adequate CPU and memory.

Addressing this fallacy requires deliberate strategies to minimize latency impact:

• Caching — reduces repeated calls to the same data
• Asynchronous communication — decouples services so they don’t block on responses
• Bulk APIs — batch multiple requests into fewer round trips
• Data locality — co-locate frequently communicating services in the same data center

Why This Matters

Latency directly impacts user experience: delays beyond 200–300ms significantly reduce conversion rates for web applications. It also affects throughput — 100ms of extra overhead per request means 10x fewer requests per second with the same resources.

This fallacy forces architects to make deliberate trade-offs when choosing distributed architectures. Independent deployability and scalability come at the cost of latency. Systems with tight inter-service coupling and high call frequency suffer exponentially from latency accumulation, making coarser-grained services or monolithic architectures a better fit in those cases.

Related Concepts

• Fallacies-of-Distributed-Computing — The complete set of eight fallacies this belongs to
• Fallacy-The-Network-Is-Reliable — Related fallacy about network failure assumptions
• Fallacy-Bandwidth-Is-Infinite — Related fallacy about data transfer capacity
• Monolithic-vs-Distributed-Architectures — The architectural decision this fallacy impacts
• Operational-Characteristics — Performance and latency are key operational characteristics
• Trade-Offs-and-Least-Worst-Architecture — Latency costs exemplify architectural trade-offs
• Microservices-Architecture-Style — Style most affected by latency accumulation
• Latency-Percentiles — How to measure latency correctly using P50/P95/P99 rather than averages
• Tail-Latency — Why high-percentile latency amplifies catastrophically across fan-out calls

Sources

• Richards, Mark and Neal Ford (2020). Fundamentals of Software Architecture: An Engineering Approach. O’Reilly Media. ISBN: 978-1-492-04345-4.

  • Chapter 9: Foundations
  • Discusses the Fallacies of Distributed Computing and their architectural implications
  • Available: https://www.oreilly.com/library/view/fundamentals-of-software/9781492043447/

• Deutsch, Peter (1994-1997). “The Eight Fallacies of Distributed Computing.” Originally articulated at Sun Microsystems.

  • Second fallacy in the original list
  • Identified through observing repeated performance issues in distributed systems
  • Widely referenced in distributed systems literature

Note

This content was drafted with assistance from AI tools for research, organization, and initial content generation. All final content has been reviewed, fact-checked, and edited by the author to ensure accuracy and alignment with the author’s intentions and perspective.