AI and Agents across the Empirical-SE Research Lifecycle

What has been published on LLMs and AI agents at each stage of empirical software engineering research — question design, literature reviews, repository mining, qualitative and quantitative analysis, artifact generation, reporting — and what those sources say about rigor, validity, reproducibility and the human role. A research digest as of6 October 2026, gathered for research on AI across the empirical-SE lifecycle.

Read the synthesisEight sections: the lifecycle map, rigor stage by stage, the reporting guidance that already exists, human-in-the-loop and provenance work, agentic-science systems, the gap, the sources that complicate the thesis, notes for the research write-up.Explore the 219 sourcesFilter by lifecycle stage, year, evidence type, AI role, discovery channel and verification status. Every claim in the synthesis links here.Consensus searchesThe eight consensus.app queries, one per lifecycle stage, with the ranked results and the channel agent's synthesis of each.Literature-gap tableThe plan's Task 1 view: for each key source, what it establishes, where it falls short, and whether the research cites it or positions against it.

The one-paragraph answer

AI is already a participant in empirical SE research, concentrated in data processing and analysis and used mostly to automate rather than to support decisions; reporting of that use is often incomplete. The field's guidance has moved past "disclose the model and the prompt": the 2025 ESE guidelines ask for session traces and human validation, 2026 guidance says approval and rejection decisions in agentic studies should be reported, and stage-specific frameworks now govern literature screening and repository mining. Gated, provenance-first agentic research exists, mostly as 2026 preprints outside SE. What no standard yet specifies is the research process itself as a governed object: which decisions were delegated to AI and under what authority, which intermediate artifacts they produced, and which human checks accepted them, recorded as reviewable provenance across the whole lifecycle. The store also shows that more traces do not make oversight more effective on their own, which is the argument for human gates that are semantic, placed and criterioned rather than merely logged.

Dataset at a glance

Records	219 — 196 academic, 13 industry, 9 standards-body
By year	2026: 91 · 2025: 65 · 2024: 37 · 2023: 17
Empirical or benchmark evidence	105 records · 28 guidelines or standards
AI acting as an autonomous or multi-agent system	89 records
Surfaced by Consensus	71 records (37 found only there)
Independently re-verified	80 of 219 re-fetched from primary pages: 53 confirmed, 26 minor metadata fixes applied, 1 wrong, 0 not found

Method

The anchor was Gomes et al. (2026), the systematic review of LLM-assisted empirical SE, recorded first and chased through its references and citing work. Two channels then ran in parallel: eight Consensus searches, one per lifecycle stage, and four discovery agents covering two stages each through web and arXiv search, publisher pages, Crossref, OpenAlex and Semantic Scholar. Records were written only from pages actually fetched; 265 raw records were deduplicated by identifier and title to 219. A separate agent that had written no records, prompted to find errors rather than confirm them, re-fetched a stratified 37% sample — every low-confidence record, every Consensus-only record in the sample, the anchor — and its corrections were applied in place. The synthesis was written by a further agent that read only the records and the channel memos. No record here is citable in the research write-up until it has been verified again through the citation tooling.

Caveats: 139 records remain unchecked; the field is preprint-heavy (104 of 219) and very recent (91 from 2026); one record was found to be wrong and is not used in the synthesis; no LLM-use policy specific to ESEM, EMSE or TOSEM was found.