Core Idea
Data Integrators are architectural forces that create pressure to keep data schemas together rather than distributing them across services.
Definition
Data Integrators are architectural forces that create pressure to keep data schemas together rather than distributing them across services. These forces represent genuine business constraints—ACID transactions, tight relationships, complex joins, consistency guarantees—making data distribution costly or risky.
Key Characteristics
-
ACID transaction requirements: Operations requiring strong consistency favor shared storage—financial transactions, inventory, orders demand all-or-nothing guarantees distributed transactions cannot provide
-
Tight data relationships: Entities with referential integrity resist separation—foreign key constraints only work within single databases (Customer → Orders → OrderItems)
-
Complex query patterns: Operations requiring joins favor co-location—reports, analytics introduce latency when distributed
-
High consistency needs: Real-time systems cannot tolerate eventual consistency—inventory, reservations, financials require immediate consistency
-
Shared domain semantics: Cohesive bounded contexts belong together—DDD keeps related concepts within aggregates
-
Transactional workflows: Multi-step processes requiring coordinated state favor unified storage—distribution introduces complexity
Examples
-
Order fulfillment: Atomic operations across Order, OrderItems, Payment, Inventory—splitting requires sagas; integration enables ACID
-
Customer aggregates: Customer with addresses and payment methods as natural aggregate—queries frequently join
-
Financial ledger: Double-entry bookkeeping requires perfect consistency—audit trails demand transactional integrity
-
Real-time inventory: Multiple purchases need consistent stock—centralized database prevents overselling
-
Analytics dashboards: Joining customers, orders, products—distributed requires API composition; integration simplifies SQL
Why It Matters
Data integrators oppose data disintegrators in trade-off analysis for service granularity. Architects evaluate integration force strength against distribution benefits (scalability, deployment, autonomy). Strong integrators suggest larger services; weak integrators support finer decomposition.
Distributed data patterns (saga, eventual consistency, CQRS) introduce complexity. Integrators identify scenarios where complexity exceeds benefits—financial systems, reservations often have such strong forces that distribution becomes untenable. Bounded contexts align with natural integration forces, revealing optimal service boundaries.
Related Concepts
- ACID: Transactional properties that create strong data integration forces
- Data-Disintegrators: Opposing forces favoring data separation
- Service-Granularity: How drivers influence service and data boundaries
- Granularity-Integrators: Forces favoring larger services and shared data
- Data-Ownership-Patterns: Implementation when data is separated
- Distributed-Transactions: Challenge when operations span owners
- Saga-Pattern: Managing transactions across separated databases
- Eventual-Consistency: Alternative when strong consistency is relaxed
- CAP-Theorem: Fundamental trade-offs in distributed data
- Coupling: Data integrators create data coupling between entities
- Bounded-Context: Domain boundaries that align with natural data integration
- Architecture-Quantum: Data relationships define quantum boundaries
- Ford-Richards-Sadalage-Dehghani-2022-Software-Architecture-The-Hard-Parts: Source book discussing this concept
Sources
-
Ford, Neal; Richards, Mark; Sadalage, Pramod; Dehghani, Zhamak (2022). Software Architecture: The Hard Parts - Modern Trade-Off Analyses for Distributed Architectures. O’Reilly Media. ISBN: 978-1-492-08689-5. Chapter 5: Distributed Data Management. Available: https://www.oreilly.com/library/view/software-architecture-the/9781492086888/
-
Fowler, Martin; Dehghani, Zhamak (2019). “How to Move Beyond a Monolithic Data Lake to a Distributed Data Mesh.” Available: https://martinfowler.com/articles/data-monolith-to-mesh.html
-
Richardson, Chris (2025). “Pattern: Database per service.” Microservices.io. Available: https://microservices.io/patterns/data/database-per-service.html
-
Laigner, Rodrigo; Zhou, Yongluan; Salles, Marcos Antonio Vaz; Liu, Yijian (2021). “Data management in microservices: State of the practice, challenges, and research directions.” arXiv:2103.00170. Available: https://arxiv.org/pdf/2103.00170
-
Nookala, G. (2023). “Microservices and Data Architecture: Aligning Scalability with Data Flow.” International Journal of Digital Innovation. Available: https://researchworkx.com/index.php/ijdi/article/download/19/19
AI Assistance
This content was drafted with assistance from AI tools for research, organization, and initial content generation. All final content has been reviewed, fact-checked, and edited by the author to ensure accuracy and alignment with the author’s intentions and perspective.