Core Idea
Data Integrators are architectural forces that create pressure to keep data schemas together rather than distributing them across services.
Definition
Data Integrators are architectural forces that create pressure to keep data schemas together rather than distributing them across services. These forces represent genuine business constraints—ACID transactions, tight relationships, complex joins, consistency guarantees—making data distribution costly or risky.
Key Characteristics
- ACID transaction requirements: Operations requiring strong consistency favor shared storage—financial transactions, inventory, and orders demand all-or-nothing guarantees that distributed transactions cannot reliably provide
- Tight data relationships: Entities with referential integrity resist separation—foreign key constraints only work within a single database (Customer → Orders → OrderItems)
- Complex query patterns: Operations requiring joins favor co-location—reports and analytics introduce latency and complexity when distributed
- High consistency needs: Real-time systems cannot tolerate eventual consistency—inventory, reservations, and financials require immediate consistency
- Shared domain semantics: Cohesive bounded contexts belong together—DDD keeps related concepts within aggregates
Examples
- Financial ledger: Double-entry bookkeeping requires perfect consistency and transactional integrity—splitting risks data corruption
- Real-time inventory: Multiple concurrent purchases need consistent stock counts—centralized storage prevents overselling
Why It Matters
Data integrators oppose Data-Disintegrators in trade-off analysis for service granularity. Architects evaluate the strength of integration forces against distribution benefits (scalability, deployment autonomy). Strong integrators suggest larger services; weak integrators support finer decomposition.
Distributed data patterns (saga, eventual consistency, CQRS) introduce significant complexity. Integrators identify scenarios where that complexity exceeds the benefit—financial systems and reservation platforms often have forces so strong that distribution becomes untenable. Bounded contexts align with natural integration forces, revealing optimal service boundaries.
Related Concepts
- ACID: Transactional properties that create strong data integration forces
- Data-Disintegrators: Opposing forces favoring data separation
- Service-Granularity: How drivers influence service and data boundaries
- Granularity-Integrators: Forces favoring larger services and shared data
- Data-Ownership-Patterns: Implementation when data is separated
- Distributed-Transactions: Challenge when operations span owners
- Saga-Pattern: Managing transactions across separated databases
- Eventual-Consistency: Alternative when strong consistency is relaxed
- CAP-Theorem: Fundamental trade-offs in distributed data
- Coupling: Data integrators create data coupling between entities
- Bounded-Context: Domain boundaries that align with natural data integration
- Architecture-Quantum: Data relationships define quantum boundaries
- Software Architecture - The Hard Parts - Ford, Richards, Sadalage & Dehghani - 2022: Source book discussing this concept
Sources
-
Ford, Neal; Richards, Mark; Sadalage, Pramod; Dehghani, Zhamak (2022). Software Architecture: The Hard Parts - Modern Trade-Off Analyses for Distributed Architectures. O’Reilly Media. ISBN: 978-1-492-08689-5. Chapter 5: Distributed Data Management. Available: https://www.oreilly.com/library/view/software-architecture-the/9781492086888/
-
Fowler, Martin; Dehghani, Zhamak (2019). “How to Move Beyond a Monolithic Data Lake to a Distributed Data Mesh.” Available: https://martinfowler.com/articles/data-monolith-to-mesh.html
-
Richardson, Chris (2025). “Pattern: Database per service.” Microservices.io. Available: https://microservices.io/patterns/data/database-per-service.html
-
Laigner, Rodrigo; Zhou, Yongluan; Salles, Marcos Antonio Vaz; Liu, Yijian (2021). “Data management in microservices: State of the practice, challenges, and research directions.” arXiv:2103.00170. Available: https://arxiv.org/pdf/2103.00170
-
Nookala, G. (2023). “Microservices and Data Architecture: Aligning Scalability with Data Flow.” International Journal of Digital Innovation. Available: https://researchworkx.com/index.php/ijdi/article/download/19/19
AI Assistance
This content was drafted with assistance from AI tools for research, organization, and initial content generation. All final content has been reviewed, fact-checked, and edited by the author to ensure accuracy and alignment with the author’s intentions and perspective.