# We Scanned 15 AI-Written Python Files: What Stuck

BrassCoders scanned 15 AI-generated Python files and emitted a finding in every one: 53 findings total, 16 of them critical or high. On triage, 9 of the 15 carried a real security or performance issue, and the rest were style notes or false positives. The split between those two numbers is the whole point of how BrassCoders is built.

The headline "every file had a finding" is true and nearly meaningless on its own. The number that matters is what survived a careful read.

## How The Corpus Was Built

BrassCoders' corpus is 15 Python files generated by Claude Opus 4.8 from realistic coding prompts, each carrying a PROVENANCE docstring naming the prompt and model, written at normal effort with nothing planted or scrubbed. The prompts cover web handlers, data processing, subprocess calls, auth, and config.

The scoring was pre-registered before the scan: count the files for which `brasscoders --offline scan` emits at least one finding, then break the findings down by category, with no file added or removed after the first run. It's a single-model corpus of 15 files, which is a stated limit, not a hidden one. The point isn't a population-level rate; it's a reproducible look at what a deterministic scanner surfaces on un-planted AI code.

## What BrassCoders Found

BrassCoders emitted 53 findings across the 15 files, 16 of them critical or high severity, with at least one finding in every file. The critical and high set clustered in the security categories.

The real, confirmed issues: SQL injection assembled with string formatting in `user_lookup.py` and `bulk_insert.py`; command injection through `shell=True` in `run_command.py` and `os.system` in `thumbnail.py`; hardcoded credentials in `token_check.py` and `email_sender.py`; an unsafe `yaml.load` in `config_loader.py`; a `requests` call with no timeout in `api_client.py`; Flask `debug=True` in `user_lookup.py`; and one genuine O(N^2) string-concatenation loop in `csv_report.py`. Nine of the fifteen files carried at least one of these.

## After Triage: What Was Real

BrassCoders' value showed up at triage: of the 16 critical-and-high findings, the security structural ones held up, while several performance and style flags did not, leaving 9 of 15 files (60%) with a confirmed issue. A raw finding is a question, not a verdict.

This is the division of labor the tool is designed around. [BrassCoders reports the pattern](https://github.com/CopperSunDev/brasscoders); a human or an LLM like Claude Code decides whether it's real. The confirmed-issue rate of 60% sits in the same range as external measurements of AI-code defects, which run from Veracode's [45% OWASP-vulnerability rate](https://www.veracode.com/blog/genai-code-security-report/) to CodeRabbit's [1.7x more issues per PR](https://www.coderabbit.ai/blog/state-of-ai-vs-human-code-generation-report). Those are different methodologies on far larger samples, cited here for context, not as this corpus's result.

## What The False Positives Teach

BrassCoders flagged four O(N^2) string concatenations, but only one was real: the other three were loops incrementing an integer counter, not building a string. A deterministic reporter that never inferred context produced exactly the noise the triage layer exists to remove.

The misfires are instructive. In `api_client.py`, `log_parser.py`, and `retry.py`, the flagged line was `page += 1` or a counter increment, not string building, so the performance warning was wrong. In `file_dedupe.py`, MD5 was flagged as a weak security hash when it was being used for content deduplication, where it's fine. And in `api_client.py`, the scanner flagged a false string-concat while missing the real performance bug one line up: `items.insert(0, item)` in a loop, an O(N^2) list prepend. A scanner that guessed at intent would hide these tradeoffs; one that reports patterns lets the triage layer catch both the false alarm and the real miss.

## Reproduce It

BrassCoders ships this corpus, its manifest, and the pre-registered scoring in the OSS repo, so the static findings reproduce exactly. Clone it, install the tool, and scan the files directory.

```bash
git clone https://github.com/CopperSunDev/brasscoders
pip install brasscoders
brasscoders --offline scan brasscoders/docs/benchmarks/ai-code-findings-corpus/files
```

The takeaway is the two-number story: a deterministic scan flags a lot, and the triage layer decides what's real. Run the scan on every commit; spend your reading on the findings that survive it.