The Pwnie-1 model card has a cardboard back
A transparent look at the questions a model card should answer.
Meet Pwnie-1, the name on our evolving model notes. Our first question: what should a model card actually tell you?
It should state what was measured, who was in the test set, what the failure cases look like, and whether the chart starts at zero. We added one more field: ‘Things we would be nervous to demo live.’ It is currently the longest section.
Before anyone announces a breakthrough, we read the footnotes.