Fields are extracted by agents and are not human-audited
How a judgment field is decided
Deterministic facts — a DOI, a PubMed identifier, a publication date, a registry number, a listed price — come from APIs and parsers and are not judgments. Judgment fields go through multiple independent extraction passes over the same source document.
Where the passes disagree, the field publishes as uncertain. It is never resolved by picking a winner, taking a majority, or trusting the more confident pass. Disagreement is information about the source — usually that the document does not actually say — and discarding it to produce a clean value would be manufacturing a fact.
An uncertain field is marked at the field on the page where it appears, not aggregated away into a footnote.
Disagreement rate, per field
Read from the corpus on this request · 3,548 extractions| Field | Extractions | Disagreed | Rate |
|---|---|---|---|
| study_type Whether a paper reports an RCT, an observational study, animal work, or in vitro work. Gates the grade outright — a misread here moves a pair between D and B. | 297 | 21 | 7.1% |
| sample_size Largest human n. Frequently absent from abstracts and paywalled in full text. | 525 | 8 | 1.5% |
| blinded Whether the study was blinded. A bias flag only where a record states it was not; absence deducts nothing and certifies nothing. | 525 | 1 | 0.2% |
| preregistered Whether the trial was preregistered. Same rule: stated absence of preregistration is a flag, silence is neither a flag nor a pass. | 525 | 3 | 0.6% |
| industry_funded Whether the funding source is a commercial party with an interest in the result. A flag only where commercial funding is stated. | 525 | 15 | 2.9% |
| direction_of_effect Whether the reported effect went the way the outcome describes. Feeds directional consistency. | 525 | 29 | 5.5% |
| outcome_id Which outcome in the fixed taxonomy the study measured. The highest-disagreement field, and the one that decides which page a study appears on at all. | 45 | 11 | 24.4% |
| peptide_administered Whether the peptide was actually administered, as opposed to mentioned, reviewed, or measured as an endogenous level. | 193 | 5 | 2.6% |
| measures_outcome Whether the study measured the outcome as an endpoint, rather than mentioning it in discussion. | 381 | 46 | 12.1% |
| stance Whether a mention was supportive, neutral, or cautionary. Read from a transcript where one exists. | 7 | 0 | 0.0% |
| All fields | 3,548 | 139 | 3.9% |
A low disagreement rate is not an accuracy figure. It measures how often independent passes reached the same answer, and passes can agree on a wrong one — particularly where a document is ambiguous in a consistent way. Read it as a floor on uncertainty, never as a ceiling on error.
The highest rate in the table is generally outcome mapping, and that is the expected shape: deciding which outcome in a fixed taxonomy a paper measured is a genuinely interpretive act, where deciding whether a trial was blinded usually is not.
How often the bias fields are filled in at all
Over all 5,525 evidence recordsThe three fields that feed the bias deduction are the emptiest columns in the corpus, and the deduction rule only makes sense read next to how empty they are. A flag is counted only where a record states one. Silence deducts nothing — not because silence is reassuring, but because deducting for it would subtract the same constant from every peptide, which measures our extraction coverage rather than anyone’s bias.
| Field | Flag stated | Stated the opposite | Not stated | Silent |
|---|---|---|---|---|
| industry_funded | 451 | 322 | 4,752 | 86.0% |
| preregistered | 136 | 664 | 4,725 | 85.5% |
| blinded | 276 | 438 | 4,811 | 87.1% |
So a “Bias risk +0” on a peptide page means no record on that pair said anything that would deduct. It does not mean the studies disclosed their funding and were clean. Read it as the absence of a finding, which is what it is, and read the size of the “silent” column as the reason the component moves so rarely.
The gap closes as extraction coverage grows, and it closes in one direction only: these fields can add flags and can never remove one, so a grade carrying no bias deduction today can fall later and cannot rise.
Why almost every mention is unclassified
Stance — whether a voice was supportive, neutral, or cautionary about a peptide — is read from a transcript. For the video mentions that make up most of this corpus, there is no transcript: YouTube’s caption download endpoint requires the video owner’s OAuth credentials and cannot be called with an API key for videos we do not own. The undocumented alternative endpoint is outside YouTube’s terms and we do not use it.
What remains is the video title, which is almost never enough to tell a warning from a recommendation. “Dr X on BPC-157” reads as endorsement whether the speaker was recommending it or warning against it, and mischaracterising a named clinician is both a credibility failure and a legal exposure. Those mentions stay unclassified.
Consequently roughly 99% of mentions in this corpus are unclassified. Every stance distribution on the site shows that as its own share of the total rather than renormalising it away, because a bar showing the remaining 1% at full width would present near-total ignorance as a finding.
The PRD treats an unclassified rate above 20% as a signal that the classifier needs work. Here it is not the classifier — it is the absence of the input the classifier reads, and it will not improve until a transcript source we can lawfully use is in place.
What is recorded on every row
- Evidence records5,525
- Records an agent has read945
- Agent-extracted field values3,541
- Field extractions, all methods3,541
An agent has read 17.1% of the corpus. 945 of 5,525 evidence records carry at least one agent-extracted field. The remaining 4,580 hold only what the collectors’ APIs supplied — an identifier, a title, a date, and whatever study type the publication-type mapping could determine. Every judgment field on those records is unfilled, and the grades computed over them are floors rather than estimates.
Every ingested row carries its source URL, the timestamp it was fetched, the extraction method (API, parser, or agent), a confidence, the identifier of the collector that produced it, and its source tag. That method names which process wrote this row, not whether a model has since read the document behind it: an evidence record created by the PubMed collector stays api for its whole life, and the agent’s work on it lands in field_extractions with its own method, confidence and provider. With no human verification step, that provenance is the entire audit trail — it is also what lets a badly behaved collector’s output be removed by collector identifier without disturbing the rest of the corpus.
Source documents are retained in full so that any field can be re-verified, and re-extracted with a better method, without re-crawling. Retained documents are never served publicly and no public route reads them.