Phase 1
Human research
Results locked
Before you start
Set aside about 50 minutes, and do this phase without AI — no chatbots, and don't lean on AI-generated search summaries.
You'll research the ecosystem around a dataset — what the news, the federal government, and social media are saying about it — and rate its risk against the rubric. You are not trying to become an expert in the dataset itself.
1. Pick your dataset
Fill in the dataset's title, publisher, description, and URL below. Phase 2 builds its prompts straight from these fields, so you won't be asked for them again.
The data set you choose (or is assigned to you) will be yours for both phases of this exercise.
The data set you choose (or is assigned to you) will be yours for the rest of this exercise.
2. Pick your categories
The rubric has six categories. Your moderator will assign a triad for you to assess throughout the exercise.
3. Rules of engagement
Do
- Look anywhere (other than the data index): government websites, social media, news, trade press, paywalled material.
- Do your best to cover all your assigned categories. If you run out of time, please use "Couldn't assess."
- Paste the URLs you used into the evidence box for each category.
Do not
- Look at the data index.
- Use AI chatbots or search engines to help you find sources or information for this first phase.
4. Your rubric and worksheet
Assess each of your categories at one of four levels, and record your rating and evidence as you go. Full six-category rubric on the rubric page.
Everything you type below is saved in this browser.
5. Submit your assessment
When you're done (or out of time), send us the worksheet above. There's no separate form — everything you typed goes with it. "Couldn't assess" is a legitimate answer for any category.
The automated benchmark
Before you started, we ran an agent over the same dataset with the same rubric and output format, and no human steering. It records its reasoning as it works.
Its ratings sit next to yours on the results page.
What happens next
You're done. Thank you.
You'll now run the same assessment through an agent of your own choosing, coaching its method but never its answers. Everything above is submitted at the end of Phase 2, in one go.
Stop
Do not press "Continue to Phase 2" without the okay from the session lead.