Manu's Vault

Data-in-Distributed-Architectures-Patterns

5 min readStatus:Seedling

Overview

Data management in distributed architectures is a trade-off analysis exercise. This framework synthesizes Data-Ownership-Patterns, competing data forces (Data-Disintegrators vs. Data-Integrators), and data access patterns—Interservice-Communication-Pattern, Column-Schema-Replication-Pattern, Replicated-Caching-Pattern, Data-Domain-Pattern—into a systematic approach. When services own their data, cross-service operations require Distributed-Transactions or eventual consistency; Contract design and Strict-vs-Loose-Contracts govern how services exchange and depend on data.

The key insight: successful data architecture requires understanding when to split data (disintegrators), when to keep data together (integrators), how to access data across boundaries (access patterns), and what consistency guarantees the system can offer.

Data Ownership and Boundaries

Data-Ownership-Patterns establish single-writer responsibility: each dataset has one authoritative service. Ownership boundaries determine who may modify data, who may read it, and how readers obtain it—through the owner’s API (interservice communication), replicated copies (column schema replication, replicated caching), or shared schema (data domain). Clear ownership enables independent service evolution and aligns with Bounded-Context boundaries from domain-driven design.

Ownership granularity: Table-level, schema-level, or database-per-service. Stronger isolation favors autonomy; shared schema simplifies transactions but increases coupling.

Data Forces: Disintegrators vs. Integrators

Data Disintegrators (Forces Favoring Separation)

Data-Disintegrators pressure toward separate databases per service:

  • Service change independence: Different schema evolution rates; independent migrations
  • Distinct scalability profiles: Read-heavy vs. write-heavy; different storage technologies
  • Fault isolation: Database failures contained to one domain
  • Security and compliance: Different sensitivity (PII, financial) and access controls
  • Technology diversity: Document, graph, time-series stores per domain
  • Team ownership: Domain-aligned teams own their data exclusively

Data Integrators (Forces Favoring Shared Data)

Data-Integrators pressure toward keeping data together:

  • ACID transaction requirements: Financial, inventory, orders need all-or-nothing guarantees
  • Tight data relationships: Referential integrity, foreign keys, natural aggregates
  • Complex query patterns: Joins, reports, analytics favor co-location
  • High consistency needs: Real-time systems intolerant of eventual consistency
  • Transactional workflows: Multi-step processes with coordinated state

Trade-off: Score disintegrators (autonomy, scale, isolation) vs. integrators (consistency, simplicity, performance). Strong disintegrators + weak integrators = good candidates for data separation. Strong integrators suggest shared schema or data domain pattern until boundaries can be relaxed.

Data Access Patterns

When data is split across services, consumers need strategies to access data they don’t own.

Interservice Communication (Ask the Owner)

Interservice-Communication-Pattern: Services request data from the owning service at runtime via API. Single source of truth, strong consistency at read time, but runtime coupling and network latency. Best when data changes frequently or strong consistency is required.

Column Schema Replication (Replicate Select Columns)

Column-Schema-Replication-Pattern: Replicate only the columns a consumer needs into its schema; synchronize via CDC or events. Local reads, no runtime dependency on owner for reads, but eventual consistency and replication lag. Best when read-heavy, stale data acceptable, or reducing load on owner.

Replicated Caching (Synchronized In-Memory Data)

Replicated-Caching-Pattern: Maintain synchronized in-memory caches across services. High read performance, fault tolerance across cache nodes, with synchronous or asynchronous replication trade-offs. Best for read-heavy, hot data with acceptable staleness.

Data Domain (Shared Schema Within Bounded Context)

Data-Domain-Pattern: Multiple services within the same bounded context share database schema (tables/schemas). ACID within the shared schema, no distributed transactions for local operations, but coordinated schema evolution and coupling. Best as incremental step in decomposition or when semantic cohesion and consistency outweigh autonomy.

Choosing a pattern: Match consistency requirements, read/write patterns, and ownership boundaries. Interservice communication enforces strongest ownership; data domain accepts coupling for transactional simplicity.

Consistency and Distributed Transactions

When operations span multiple services (and thus multiple data owners), local ACID no longer applies. Distributed-Transactions require saga-style compensation or two-phase commit. Contract design—Strict-vs-Loose-Contracts—affects how easily services can evolve and how failures propagate. Tighter contracts give consistency and clarity; looser contracts allow independent evolution with higher integration cost.

  • Strong consistency across services: Distributed transactions or synchronous coordination (high cost, coupling).
  • Eventual consistency: Accept temporary inconsistency; use sagas for compensation when needed.
  • Single-owner operations: Keep transactions within one service’s data; use access patterns for reads only.

Decision Framework

  1. Establish ownership boundaries: Align with bounded contexts; assign single-writer per dataset.
  2. Analyze data forces: Score disintegrators vs. integrators. Strong integrators → consider data domain or shared schema; strong disintegrators → separate databases and choose access pattern.
  3. Choose access pattern: Interservice communication (strong consistency, high coupling), column schema replication (eventual consistency, local reads), replicated caching (performance, staleness), or data domain (ACID, shared evolution).
  4. Design contracts: Define APIs and data shapes; decide strict vs. loose contracts for evolution and failure handling.
  5. Handle cross-service operations: Use sagas or accept eventual consistency; avoid distributed transactions unless necessary.

Sources

Note

This content was drafted with assistance from AI tools for research, organization, and initial content generation. All final content has been reviewed, fact-checked, and edited by the author to ensure accuracy and alignment with the author’s intentions and perspective.

Graph View

Backlinks