RPG Notetaker Bench
RPG Notetaker Bench
A comprehensive comparison of how different AI models process and organize tabletop RPG session transcripts. This benchmark analyzes a 2.5-hour D&D session transcript processed by six different AI models, comparing their approaches to note-taking, organization, and information extraction.
The Session: “Dicing With Death #225”
Session Details:
- Date: June 26, 2025
- Duration: 2h 37m
- Participants: Neal (GM), Ryan (Phoenix the Warlock)
- System: D&D
- Campaign: Dragon politics and intrigue in Arcadia
AI Model Comparison
Each model was tasked with converting the raw transcript into organized session notes. Click through the different versions to see how each AI approached the task:
Original Transcript
Size: 116KB, 5,522 lines
Format: Raw audio transcript with timestamps
The unprocessed transcript directly from the recording, including all the natural speech patterns, interruptions, and cross-talk that occur during live play.
GPT-4.1 Processing
Size: 10KB, 205 lines
Approach: Comprehensive scene-by-scene breakdown with detailed NPC ledgers
Creates highly structured notes with scene summaries, character tracking, and forward-looking session preparation notes.
GPT-4o Processing
Size: 3.1KB, 72 lines
Approach: Concise summary format
Focuses on the essential plot points and key moments, creating a streamlined overview of the session.
GPT-4o Mini Processing
Size: 4.2KB, 88 lines
Approach: Balanced summary with key highlights
Provides a middle-ground approach between comprehensive detail and concise summarization.
O1-Mini Processing
Size: 7.5KB, 203 lines
Approach: Analytical breakdown with strategic insights
Emphasizes the tactical and strategic elements of the session, with detailed combat analysis.
O3-Mini Processing
Size: 12KB, 147 lines
Approach: Formatted sections with processing metadata
Creates visually distinct sections with clear headers and includes processing statistics.
O4-Mini Processing
Size: 6.4KB, 124 lines
Approach: Narrative-focused organization
Emphasizes story flow and character development over mechanical details.