ReproCheck

Audit batches

Method

How to read these findings

1. Nature of the evidence. Every finding quotes the paper and cites the released code by file and line; the key evidence of each batch is listed with its command or source in Appendix A. In the Top-10 batch, items marked “found by the audit agent” were not individually re-verified.
2. Static review. Findings come from reading papers and released code. No downloaded code is executed and no experiment is re-run, so “no issues found” means the static checks pass — not that the numbers reproduce.
3. What Tier S means. The released code itself lets test labels or test metrics choose the model, epoch or hyper-parameters, and that path produces the reported numbers. Unlike Tier A (something claimed is missing), Tier S is logic that is present in the code and can be pinned down line by line.
4. Motive cannot be proven. Comparing code with the paper can only show that “claims and artifacts contradict each other”; it cannot directly prove intent. “Suspected fabrication / data conflation” here means the contradictions reach that level of systematicity and severity. The final determination (whether there was fabrication, whether a paper should be retracted) requires author responses, institutional review and independent replication.
5. What “retraction-level risk” means. The paper’s core experimental claims cannot be supported by any public artifact, or systematically contradict them — that is, its key reported numbers have no verification path. In research-integrity processes, this is the situation that warrants a formal investigation.

Severity tiers

Ranking rule. Tier first (S → A → B → C → D → no issues found) → within a tier, by citations (Semantic Scholar counts, which have broader coverage; OpenAlex counts are listed alongside and used when Semantic Scholar has no record; “–” marks a missing count) → venue tier. The full method of each batch is described in the Appendix.