Rule-Beating
Core Idea
Rule-beating is a systems trap in which actors comply with the letter of a rule while violating its intent — because the system’s feedback rewards proxy adherence, not underlying purpose.
The Core Structure
- A rule is created to guide behaviour toward a desired outcome
- Rules are necessarily proxies: measurable, enforceable stand-ins for purposes that are harder to quantify
- Actors optimise for the proxy because that is what the feedback loop measures and rewards
- The proxy diverges from the purpose → compliance rises, outcomes degrade
The trap is structural — it arises wherever rules govern behaviour, incentives to game rules exist, and actors can distinguish the letter of a rule from its spirit.
Goodhart’s Law and Campbell’s Law
Two empirical generalisations capture the same dynamic:
- Goodhart’s Law: “When a measure becomes a target, it ceases to be a good measure.”
- Campbell’s Law: “The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures.”
Both describe the same feedback failure: the indicator was never the thing itself, and pressure to hit the indicator destroys the correlation between indicator and thing.
Rule-Beating vs. Seeking the Wrong Goal
| Trap | Actor’s knowledge | Mechanism |
|---|---|---|
| Rule-Beating | Knows the intent; games the proxy deliberately | Incentive structure rewards proxy over purpose |
| Seeking Wrong Goal | Sincerely pursues the proxy, unaware it diverges | Feedback system points at the wrong target |
Examples Across Domains
- Education: Schools optimise test scores rather than learning — score gains without knowledge gains
- Finance: Banks structure instruments to satisfy capital-ratio rules while carrying equivalent risk off-balance-sheet
- Manufacturing: Soviet factories met tonnage quotas with overweight, poor-quality goods
- Healthcare: Diagnostic upcoding improves reimbursement without improving care
Escape Routes
- Measure outcomes, not proxies: Redesign feedback loops to respond to the actual purpose
- Diverse, hard-to-game indicators: No single metric should dominate; use a portfolio harder to simultaneously game
- Acknowledge Bounded-Rationality: Change incentives rather than demanding actors override them
Related Concepts
- Systems-Thinking
- System-Purpose-and-Function
- Bounded-Rationality
- Mental-Models
- Thinking in Systems - Meadows - 2008
Sources
-
Meadows, Donella H. (2008). Thinking in Systems: A Primer. Chelsea Green Publishing. ISBN: 978-1-60358-055-7.
- Chapter 5, pp. 140–145: rule-beating as a system trap
-
Goodhart, Charles A. E. (1975). “Problems of Monetary Management: The U.K. Experience.” Papers in Monetary Economics, Vol. I. Reserve Bank of Australia.
- Original articulation of Goodhart’s Law
-
Campbell, Donald T. (1979). “Assessing the Impact of Planned Social Change.” Evaluation and Program Planning, Vol. 2, No. 1, pp. 67–90. DOI: 10.1016/0149-7189(79)90048-X
-
Muller, Jerry Z. (2018). The Tyranny of Metrics. Princeton University Press. ISBN: 978-0-691-17472-5.
- Comprehensive documentation of rule-beating across education, medicine, policing, and finance
-
Strathern, Marilyn (1997). “‘Improving Ratings’: Audit in the British University System.” European Review, Vol. 5, No. 3, pp. 305–321. DOI: 10.1002/(SICI)1234-981X(199707)5:3<305::AID-EURO184>3.0.CO;2-4
Note
This content was drafted with assistance from AI tools for research, organization, and initial content generation. All final content has been reviewed, fact-checked, and edited by the author to ensure accuracy and alignment with the author’s intentions and perspective.