Manu's Vault

Rule-Beating

4 min readStatus:Seedling

Rule-Beating

Rule-beating is a systems trap in which actors comply with the letter of a rule while violating its intent — because the system’s feedback rewards proxy adherence, not underlying purpose.

The Core Structure

  • A rule is created to guide behaviour toward a desired outcome
  • Rules are necessarily proxies: measurable, enforceable stand-ins for purposes that are harder to quantify
  • Actors optimise for the proxy because that is what the feedback loop measures and rewards
  • The proxy diverges from the purpose → compliance rises, outcomes degrade

The trap is structural. It arises wherever rules govern behaviour, incentives to game rules exist, and actors can distinguish the letter of a rule from its spirit.

Goodhart’s Law and Campbell’s Law

Two empirical generalisations capture the same underlying dynamic from different fields:

  • Goodhart’s Law: “When a measure becomes a target, it ceases to be a good measure.” A metric that was descriptively useful becomes corrupted once it is directly optimised.
  • Campbell’s Law: “The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is meant to monitor.”

Both laws describe the same feedback failure: the indicator was never the thing itself, and pressure to hit the indicator destroys the correlation between indicator and thing.

Rule-Beating vs. Seeking the Wrong Goal

These traps are structurally distinct:

TrapActor’s knowledgeMechanism
Rule-BeatingKnows the intent; games the proxy deliberatelyIncentive structure rewards proxy over purpose
Seeking Wrong GoalSincerely pursues the proxy, unaware it divergesFeedback system points at the wrong target

Rule-beating involves intentional gaming; Seeking Wrong Goal (a related but distinct trap) involves sincere misalignment. The first is a problem of incentives; the second is a problem of measurement design. Both require fixing what the system is actually optimising for.

Examples Across Domains

  • Education: Schools optimise test scores rather than learning; teaching to the test produces score gains without knowledge gains
  • Finance: Banks structure instruments to satisfy capital-ratio rules while carrying equivalent risk off-balance-sheet
  • Manufacturing: Soviet factories met tonnage quotas with overweight, poor-quality goods; nail quotas were met with one enormous nail
  • Healthcare: Billing codes are optimised; patient outcomes are not — diagnostic upcoding improves reimbursement without improving care

Escape Routes

  • Measure outcomes, not proxies: Redesign feedback loops to respond to the actual purpose where feasible
  • Diverse, hard-to-game indicators: No single metric should dominate; use a portfolio that is harder to simultaneously game
  • Purpose-based rules: Where possible, articulate intent in rules rather than only measurable proxies
  • Bounded-Rationality: Acknowledge that actors respond to local incentives; change incentives rather than demanding actors override them

Sources

  • Meadows, Donella H. (2008). Thinking in Systems: A Primer. Chelsea Green Publishing. ISBN: 978-1-60358-055-7.

    • Chapter 5, pp. 140–145: Rule-beating as a system trap — rules as proxies, actors gaming proxies, structural origin of the phenomenon

  • Goodhart, Charles A. E. (1975). “Problems of Monetary Management: The U.K. Experience.” Papers in Monetary Economics, Vol. I. Reserve Bank of Australia.

    • Original articulation of what became known as Goodhart’s Law; monetary policy context but principle generalises widely

  • Campbell, Donald T. (1979). “Assessing the Impact of Planned Social Change.” Evaluation and Program Planning, Vol. 2, No. 1, pp. 67–90. DOI: 10.1016/0149-7189(79)90048-X

    • Campbell’s Law: quantitative social indicators corrupt when used as decision targets; empirical evidence from social programme evaluation

  • Muller, Jerry Z. (2018). The Tyranny of Metrics. Princeton University Press. ISBN: 978-0-691-17472-5.

    • Comprehensive documentation of rule-beating across education, medicine, policing, finance, and military; argues for blending quantitative metrics with qualitative judgement

  • Strathern, Marilyn (1997). “‘Improving Ratings’: Audit in the British University System.” European Review, Vol. 5, No. 3, pp. 305–321. DOI: 10.1002/(SICI)1234-981X(199707)5:3<305::AID-EURO184>3.0.CO;2-4

    • Coined the phrase “Goodhart’s Law” in its modern social-science form; case study in UK higher education audit and metric gaming

Note

This content was drafted with assistance from AI tools for research, organization, and initial content generation. All final content has been reviewed, fact-checked, and edited by the author to ensure accuracy and alignment with the author’s intentions and perspective.

Graph View

Backlinks