Phase 2
The prompting session
You will now run the same assessment alongside an AI agent, coaching it without telling it the conclusions to draw.
1. Set up your agent
- Any agent is allowed — ChatGPT, Claude, Gemini, or another.
- Turn memory off.
- Turn thinking mode on.
- Make sure the agent can search the web. This work depends on live sources, so an agent with no internet access will invent its evidence. Turn on web search, browsing, or research mode, and if you're unsure whether it's working, ask the agent to name the URLs it used.
- Start a fresh conversation for this exercise only.
ChatGPT
Click "Think" in the prompt input field to enable thinking mode. For memory, click Profile > Settings > Personalization, then scroll down to "Memory" and turn off "Enable memory."
Claude
Thinking mode is on by default for all Opus models. Ensure effort is set to "high." For memory, click Profile > Settings > Capabilities and turn off "Search and reference chats" and "Generate memory from chat history."
Anything else
Look for a reasoning, thinking, or research mode and turn it on. If you can't find one, ask a moderator for help.
2. Ground rules
Do
- Interact like you're training the agent.
- Stay neutral: ask for changes to its method, not its result.
- Keep the "I like… / please try…" feedback structure.
- Deviate small amounts from the prompts if another line of queries renders better results, but note that you did so.
- Please feel free to ask the agent to consider more precise information, such as the specific unit division or years.
Do not
- Deviate too far from the prompts as provided — you don't want to turn this into "guess what I'm thinking, AI…"
- Nudge the bot toward a particular risk level. It will happily change its answer to please you without improving its analysis.
- Move between agents or try starting new conversations to illicit different responses.
3. Step One — dataset research
Paste this into your agent's chat box:
4. Step Two — risk assessment
This prompt includes only the rubric categories selected in Phase 1.
5. What the agent added
Your Phase 1 rating and evidence are shown for each category — record where the agent landed, and what it surfaced that your own research hadn't. Leave a box empty if it added nothing.
6. Close out and submit
Paste this in last. It makes the agent identify itself, so we know which model produced the assessment. Keep its answer in your transcript.
If your agent can't produce a link, export the transcript instead.
The automated benchmark
Before your session, we ran an agent over the same dataset with the same rubric and output format, and no human steering.
You can see how your results compare to the background agent's on the results page.
7. Compare
Your results page shows your ratings, the background agent's, and America's Data Index's side by side. Check out the Data Index pages for more information on why the dataset is rated the way it is.