Governed Agentic Empirical Research

Governing AI-assisted empirical software engineering research

Research on what must be recorded and checked when AI agents participate in empirical software engineering research: which decisions are delegated, what evidence supports them, and how the process stays reviewable.

The research digest

The public digest maps what has been published on LLMs and agents at each stage of empirical software engineering research — question design, literature reviews, repository mining, qualitative and quantitative analysis, artifact generation and reporting — and what those sources say about rigor, validity, reproducibility and the human role. It is anchored on Gomes et al. (2026) and backed by a source explorer and dated method note.

AI and agents across the empirical-SE lifecycleThe synthesis, the 219-record source explorer, the Consensus searches and the literature-gap table.219 sources · 6 October 2026

The tooling

The work here runs on two plugins I maintain: one governs how agents and people change a repository, the other scaffolds and checks a paper. These pages explain what each does, where a human decides, and the study being set up to measure how much of the research process the tooling actually covers.

How agentic-governance worksThe workflow, the L0 to L3 levels, the four AI roles, the L0 fast track, and every point where a human decides.v0.9.1 · explainer What agentic-research doesSkills, agents and checks for a paper repository, how it composes with governance, and the gaps found so far.v1.8.0 · explainer Does the tooling cover the work?A planned completeness benchmark against the research and paper-writing lifecycle.planned · no results yet

How this relates to the other projects

This project is the methodological layer on top of work that is reported elsewhere on this site. Each project below supplies one strand of the argument; none of them is the whole research programme.

Agentic harnesses andAgent memory (theState of the Art in Agentic SE project) are themselves governed research passes: schema-constrained source records, discovery separated from an adversarially prompted verification pass, provenance fields on every record. They are the method's literature-review stage in practice — one stage of the lifecycle, not the whole process.
Research.AI (the agentic-kg project under the knowledge-graphs tenet) supplies the sharpest case for human methodological judgement: reconciling a gold-label schema where the labelling choice itself could distort measured precision and recall, so the evaluation would measure the researchers' decision rather than the system's behaviour.
A time-boxed empirical study run in August 2026 under the framework's tooling (agentic-governance, agentic-research) is one of this project's two worked cases: typed claims, pre-registered thresholds, a human re-derivation ledger and an adversarial reviewer gating what could be reported. Changes that came out of it flowed back into both tools.

Status

The research digest on AI across the empirical-SE lifecycle is public (219 sources, 6 October 2026). Explainers for the two plugins were added on 9 November 2026, alongside the design of a completeness benchmark. Current writing is private.