Right the first time? Data published with the study at https://www.vallit.net/research/right-the-first-time summary.json Every measured result in the study, computed by scripts/research-export-blind.mjs in the vallit-website repository from the coverage benchmark's raw reports and each case's case.json (their SHA-256 is listed under "sources"). "first" is the one score a set got, on checks frozen before it was written; "after" is the same set after the fixes it led to. Intervals are 95 % Wilson score intervals. "notEvaluated" names the cases a check declined to run because the case was written without the owner's OK it needs; they are counted as the run scored them. cases.csv Every case of the blind sets, one row each: the set as the benchmark names it, the case id, the check, whether it is a website or a repository case, what the label expects ("finding" or "clean"), and the verdict of the first run and after the fixes ("found", "missed", "clean" or "wrong", a safe case the check reported). score.mjs Recounts the totals and each set from cases.csv: node score.mjs cases.csv The cases describe made-up apps. Their addresses are left out of every file, because a made-up name can belong to a real domain. The cases themselves and the checks are not public.