Manu's Vault

Humans-On-The-Loop

5 min readStatus:Seedling

Humans-on-the-loop is a model of human-AI collaboration where humans govern the system that directs AI agents rather than directly reviewing or executing individual agent outputs. It sits between two failure modes on the autonomy spectrum: humans micromanaging every artifact the agent produces (“in the loop”) and humans abdicating oversight entirely (“outside the loop”). The term is widely used in agentic AI and software engineering literature to describe the optimal positioning for humans as AI coding agents become capable of generating code faster than humans can inspect it.

The Three Positioning Models

Humans in the loop — humans act as gatekeepers inside the innermost execution cycle, manually inspecting and approving agent outputs before they proceed:

  • Preserves tight quality control and human expertise in the critical path
  • Creates a severe throughput bottleneck — agents generate code faster than humans can review it
  • Reduces net productivity gains: time saved by generation is consumed by review
  • Does not scale; remains viable only for very low-volume, high-stakes tasks

Humans outside the loop — also called “vibe coding,” humans specify desired outcomes and agents handle all implementation autonomously:

  • Removes the human bottleneck and allows agents to iterate at machine speed
  • Code quality degrades progressively — messy code causes agents to spiral and slow down even for subsequent AI-driven tasks
  • Lacks verification of correctness and behaviour; defects accumulate invisibly
  • Viable only for genuinely throwaway prototypes where quality is irrelevant

Humans on the loop — humans build and continuously improve the harness (specifications, quality checks, constraints, workflow guidance) that controls agent behaviour:

  • Humans define the how-loop; agents run it
  • When output is unsatisfactory, humans improve the system producing it — not the output itself
  • Avoids both the bottleneck of “in the loop” and the quality collapse of “outside the loop”
  • Creates a compounding improvement dynamic: a better harness produces better outputs for all future runs

Why “On the Loop” Is Optimal

The core asymmetry is one of scale and speed:

  • AI agents can produce code at rates that make per-output human review economically and temporally infeasible
  • Delegating all autonomy leads to entropy — models operating without constraints accumulate errors and structural degradation that compound over time
  • “On the loop” resolves the asymmetry by shifting human attention upstream, to the systemic inputs (harness quality) rather than downstream outputs (individual code artifacts)
  • This matches findings from the human oversight literature: “human-on-the-loop” architectures allow AI to operate at machine speed while humans retain strategic governance and intervention capability (Amershi et al. 2019; Wu et al. 2022)

The Role Shift: From Executor to Harness Governor

Humans on the loop perform a fundamentally different kind of work:

  • Signal interpretation — treating agent failures as diagnostic signals about harness gaps, not individual errors to fix manually
  • Harness investment — adding specifications, guardrails, linters, tests, and documentation that encode quality standards durably
  • System monitoring — watching patterns across many agent runs rather than auditing single outputs
  • Intervention at the right altitude — stepping in to adjust systemic constraints, not to patch individual artifacts

This role shift requires Harness-Engineering as its operational counterpart: the practice of designing and evolving the harness that makes “on the loop” governance tractable.

  • Harness-Engineering — the practice of building the harness that “on the loop” humans govern

Sources

  • Boeckeler, Birgitta (2026). “Humans and Agents in Software Engineering Loops.” martinfowler.com. Retrieved March 2026. https://martinfowler.com/articles/exploring-gen-ai/humans-and-agents.html

    • Primary source defining the three positioning models (in, on, outside the loop) in a software engineering context and the role of the harness in the “on the loop” approach

  • Boeckeler, Birgitta (2026). “Harness Engineering.” martinfowler.com. Retrieved March 2026. https://martinfowler.com/articles/exploring-gen-ai/harness-engineering.html

    • Supporting context on what the harness is and the iterative feedback loop through which humans improve it

  • Wu, Xingjiao, Luwei Xiao, Yixuan Sun, Junhang Zhang, Tianlong Ma, and Liang He (2022). “A survey of human-in-the-loop for machine learning.” Future Generation Computer Systems, Vol. 135, pp. 364–381. DOI: https://doi.org/10.1016/j.future.2022.05.014

    • Comprehensive academic survey establishing the definition and taxonomy of human-in-the-loop approaches across ML domains; provides scholarly grounding for the positioning spectrum

  • Amershi, Saleema, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi Iqbal, Paul N. Bennett, Kori Inkpen, Jaime Teevan, Ruth Kikin-Gil, and Eric Horvitz (2019). “Software engineering for machine learning: A case study.” Proceedings of the 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP 2019), pp. 291–300. IEEE. DOI: https://doi.org/10.1109/ICSE-SEIP.2019.00042

    • Empirical study from Microsoft identifying human oversight design as a key challenge in ML-integrated systems; establishes context for why on-the-loop governance patterns emerged in practice

  • Mosqueira-Rey, Eduardo, Elena Hernández-Pereira, David Alonso-Ríos, José Bobes-Bascarán, and Ángel Fernández-Leal (2023). “Human-in-the-loop machine learning: a state of the art.” Artificial Intelligence Review, Vol. 56, pp. 3005–3054. Springer. DOI: https://doi.org/10.1007/s10462-022-10246-w

    • State-of-the-art review covering human oversight positioning models and the trade-offs between tight human control and autonomous operation; directly relevant to the in/on/outside-loop spectrum

Note

This content was drafted with assistance from AI tools for research, organization, and initial content generation. All final content has been reviewed, fact-checked, and edited by the author to ensure accuracy and alignment with the author’s intentions and perspective.

Graph View

Backlinks