imjasonh/nethackbot
1
0
Fork 0
mirror of https://github.com/imjasonh/nethackbot synced 2026-07-07 00:12:30 +00:00
Code Activity
nethackbot/plan.md
Jason Hall fdd17b6774
Create plan.md
2026-03-11 11:43:02 -04:00

62 KiB
Raw Permalink Blame History

NetHack LLM Bot — Design Document & Engineering Plan

Project: nethack-llm — An LLM-powered agent that plays NetHack locally via PTY
Language: Go
LLM Backend: Anthropic Claude API
Execution: Local NetHack process driven via pseudoterminal
Goal: Full ascension run
Primary Output: Structured game recordings for LLM-driven eval/critique
Status: Draft v2.0

1. Project Overview

1.1 What We're Building

A Go application that launches NetHack locally via a pseudoterminal (PTY), maintains a virtual terminal to parse the game screen, extracts structured game state, and uses Claude to make decisions — from basic movement and combat through inventory management, survival, and long-term ascension strategy.

The bot produces rich, structured game recordings as its primary artifact. These recordings capture every game state, every LLM decision (with full reasoning), and every outcome — forming a dataset that a separate LLM "critic" agent can analyze to identify mistakes, suggest strategy improvements, and systematically improve future runs.

1.1.1 The Improvement Loop

The system is designed around a continuous improvement cycle:

  ┌──────────┐     ┌───────────┐     ┌───────────┐     ┌──────────┐
  │  Play    │────▶│  Record   │────▶│  Critique  │────▶│  Update  │
  │  (Bot)   │     │  (Log)    │     │  (Eval LLM)│     │  (Config)│
  └──────────┘     └───────────┘     └───────────┘     └──────────┘
       ▲                                                      │
       └──────────────────────────────────────────────────────┘
  1. Play: The bot plays NetHack, making LLM-driven decisions
  2. Record: Every turn is logged with full context — game state, prompt sent, LLM reasoning, action taken, outcome
  3. Critique: A separate LLM agent reviews the recording, identifies bad decisions, missed opportunities, and strategic errors
  4. Update: Critique findings feed back into prompt templates, knowledge base, and strategy rules for the next run

1.2 Why This Is Hard

NetHack is one of the hardest games in existence. An ascension run requires:

  • Thousands of decisions across 50+ dungeon levels
  • Resource management — food, HP, items, gold, spell power
  • Risk assessment — knowing when to fight, flee, or use consumables
  • Long-term planning — quest completion, artifact acquisition, elemental planes
  • Obscure knowledge — hundreds of special interactions, prayer timing, sacrifice rules, genocide targets, polypiling, etc.
  • Adaptation — every run is procedurally generated; no memorized routes

An LLM must handle all of this with a finite context window and no built-in game tree search.

1.3 Success Criteria

Milestone Definition
M0 — Launch & Record Launch local NetHack via PTY, parse screen, produce a valid game recording
M1 — Explore Navigate dungeon levels, open doors, use stairs, avoid walking into walls
M2 — Survive Fight monsters, eat food, manage HP, don't starve or die to trivial threats
M3 — Critique Loop Eval agent reviews recordings, produces actionable feedback, feedback improves next run
M4 — Progress Reach Minetown, complete Sokoban, identify items, manage inventory
M5 — Quest Reach quest, complete it, retrieve artifact
M6 — Endgame Reach the Castle, obtain the Amulet, navigate elemental planes
M7 — Ascend Offer the Amulet on the Astral Plane altar

Realistically, M0–M3 is the v1 target, with M4 as a stretch goal. The critique loop (M3) is a force multiplier — each run improves the next.

2. Architecture

2.1 High-Level Diagram

┌───────────────────────────────────────────────────────────────┐
│                        nethack-llm                            │
│                                                               │
│  ┌──────────┐   ┌───────────┐   ┌──────────────────┐         │
│  │   PTY    │──▶│  VT100    │──▶│  Game State      │         │
│  │  Driver  │   │  Terminal  │   │  Parser          │         │
│  │          │◀──│  Emulator  │   │                  │         │
│  └──────────┘   └───────────┘   └────────┬─────────┘         │
│       ▲                                  │                    │
│       │                                  ▼                    │
│  ┌──────────┐   ┌───────────┐   ┌──────────────────┐         │
│  │  Action  │◀──│  Decision │◀──│  LLM Client      │         │
│  │  Executor│   │  Engine   │   │  (Claude API)    │         │
│  └──────────┘   └───────────┘   └──────────────────┘         │
│                      ▲                                        │
│                      │                                        │
│               ┌──────┴───────┐                                │
│               │   Memory /   │                                │
│               │   History    │                                │
│               └──────────────┘                                │
└───────────────────────┬───────────────────────────────────────┘
                        │  writes
                        ▼
              ┌──────────────────┐
              │  Game Recording  │
              │  (JSONL + TTYRec)│
              └────────┬─────────┘
                       │  feeds
                       ▼
              ┌──────────────────┐     ┌───────────────────┐
              │  Eval / Critique │────▶│  Updated Prompts  │
              │  Agent (offline) │     │  & Knowledge Base │
              └──────────────────┘     └───────────────────┘
         │                           ▲
         ▼                           │
  ┌──────────────┐            ┌──────────────┐
  │  Local       │            │  Critique    │
  │  NetHack     │            │  Reports     │
  └──────────────┘            └──────────────┘

2.2 Module Breakdown

Module Package Responsibility
PTY Driver pkg/pty Launch NetHack subprocess, manage pseudoterminal, raw I/O
VT100 Terminal pkg/terminal ANSI/VT100 escape sequence processing, 80×24 char buffer with color
Game State Parser pkg/gamestate Extract map, player stats, messages, inventory, prompts from terminal buffer
LLM Client pkg/llm Claude API integration, prompt construction, response parsing
Decision Engine pkg/brain Orchestrates the game loop, manages turn flow, handles multi-step actions
Memory pkg/memory Rolling game history, run summary, knowledge base
Action Executor pkg/action Translates high-level actions ("move north") to keystrokes, handles multi-key sequences
Recording pkg/recording Full TTY recording, structured decision log (JSONL), game state snapshots
Eval / Critique cmd/eval Offline tool: feeds recordings to a critic LLM, produces reports and improvement suggestions
Config pkg/config Game settings, API keys, tuning parameters, .nethackrc generation

3. Module Design

3.1 PTY Driver (pkg/pty)

Purpose: Launch NetHack as a local subprocess and communicate via pseudoterminal.

Running locally via PTY instead of telnet gives us significant advantages: zero network latency, no connection drops, no server rate limits, full control over the NetHack binary and configuration, deterministic seeding (if desired), and the ability to run games as fast as the LLM can respond.

Key responsibilities:

  • Allocate a pseudoterminal (PTY) with os/exec + creack/pty
  • Set terminal size to 80×24 via TIOCSWINSZ ioctl
  • Launch the NetHack binary with appropriate environment (TERM=xterm-256color, NETHACKOPTIONS, custom .nethackrc)
  • Provide a clean byte-stream interface (read/write) to the terminal emulator layer
  • Detect process exit (game over / quit) via wait status
  • Optionally tee raw PTY output to a ttyrec file for visual replay

Interface:

type PTYDriver interface {
    Start(ctx context.Context) error       // Launch nethack subprocess
    Read(buf []byte) (int, error)          // Read output bytes from PTY
    Write(data []byte) (int, error)        // Write input bytes to PTY
    Resize(cols, rows int) error           // SIGWINCH
    Wait() (*os.ProcessState, error)       // Wait for process exit
    Close() error                          // Kill process, close PTY
}

NetHack launch configuration:

type GameConfig struct {
    Binary      string   // Path to nethack binary, e.g. "/usr/bin/nethack"
    NethackRC   string   // Path to .nethackrc file (we generate this)
    PlaybackDir string   // Where to store playground directory
    Seed        int64    // Random seed (0 = random). NH 3.7+ supports this.
    Cols        int      // Terminal width (80)
    Rows        int      // Terminal height (24)
}

.nethackrc generation:

We control the NetHack configuration to make parsing easier and more reliable. The bot generates a .nethackrc tailored for machine consumption:

# Generated by nethack-llm — optimized for bot parsing
OPTIONS=color
OPTIONS=showexp
OPTIONS=showscore
OPTIONS=time
OPTIONS=autopickup
OPTIONS=pickup_types:$
OPTIONS=!autopickup      # We handle pickup decisions via LLM
OPTIONS=!rest_on_space   # Space only for --More--
OPTIONS=hilite_pet       # Distinguish pets
OPTIONS=lit_corridor     # Show lit areas
OPTIONS=standout         # Use reverse video for --More--
OPTIONS=DECgraphics      # Consistent line-drawing characters
OPTIONS=!tombstone       # Skip tombstone animation on death
OPTIONS=disclose:yi ya yv yg yc yo  # Auto-yes all death disclosures
OPTIONS=suppress_alert:3.4.3
OPTIONS=msg_window:reversed
OPTIONS=msghistory:20
OPTIONS=statushilites:10

Key options for bot parsing: showexp and time give us XP and turn count on the status line. disclose options auto-reveal inventory/stats on death for post-mortem analysis. DECgraphics gives us consistent wall/door characters across terminals.

Advantages over telnet:

Aspect Telnet (NAO) Local PTY
Latency 50-200ms network RTT <1ms
Stability Connection drops Always stable
Speed Limited by server/network As fast as LLM responds
Configuration Limited to server options Full control of .nethackrc
Reproducibility Random seeds per server Can set seed for replay
Cost per run Same Same (LLM cost dominates)
Setup Account required Just install nethack binary
Rate limits Possible server policies None

Go libraries:

  • creack/pty — the standard Go library for PTY allocation. Mature, well-maintained.
  • os/exec — stdlib, for launching the subprocess
  • golang.org/x/sys/unix — for TIOCSWINSZ ioctl if needed beyond what creack/pty provides

3.2 VT100 Terminal Emulator (pkg/terminal)

Purpose: Process the raw byte stream from the PTY into a 2D character grid with attributes.

This is the most technically complex low-level module. NetHack uses VT100/ANSI escape sequences extensively for cursor positioning, color, and screen management.

Terminal buffer structure:

type Cell struct {
    Char  rune
    FG    Color  // foreground color (ANSI 0-15 or 256-color)
    BG    Color  // background color
    Bold  bool
    Attrs uint8  // underline, reverse, etc.
}

type Terminal struct {
    Width, Height int          // 80×24
    Cells         [][]Cell     // [row][col]
    CursorRow     int
    CursorCol     int
    // scrollback, alternate screen buffer, etc.
}

Escape sequences we must handle:

Sequence Meaning Priority
ESC[H or ESC[;H Cursor home Critical
ESC[{r};{c}H Cursor position Critical
ESC[{n}A/B/C/D Cursor up/down/forward/back Critical
ESC[J / ESC[2J Clear screen / clear all Critical
ESC[K Clear to end of line Critical
ESC[{n}m SGR (color/attribute) Critical
ESC[?1049h/l Alternate screen buffer Medium
ESC[?25h/l Show/hide cursor Low
\r, \n, \b, \t Carriage return, newline, backspace, tab Critical
ESC[{n};{m}r Set scrolling region Medium
ESC[S / ESC[T Scroll up/down Medium

SGR (color) handling:

NetHack uses colors to distinguish monster types, item types, and dungeon features. The color of a character on the map often carries critical information (e.g., a red d is a hell hound, a brown d is a jackal).

ESC[0m      — Reset
ESC[1m      — Bold (often = bright color)
ESC[7m      — Reverse video
ESC[30-37m  — Foreground color
ESC[40-47m  — Background color
ESC[90-97m  — Bright foreground

Design considerations:

  • Process bytes incrementally as they arrive from the PTY
  • Use a state machine for escape sequence parsing (ground, escape, CSI parameter, CSI intermediate, etc.)
  • Expose ProcessByte(b byte) and GetScreen() [][]Cell interfaces
  • Consider using an existing Go VT100 library like danielgatis/go-vte as a starting point, but we likely need custom implementation for the game state extraction hooks

Go libraries to evaluate:

  • danielgatis/go-vte — VT parser in Go
  • creack/pty — PTY handling (used by the PTY driver layer)
  • Custom implementation — most likely path since we need tight integration with game state parsing

3.3 Game State Parser (pkg/gamestate)

Purpose: Transform the raw terminal grid into structured game data.

NetHack's screen layout is well-defined:

Row 0:          Message line (game messages, prompts)
Rows 1-21:      Map area (the dungeon)
Row 22:         Status line 1 (name, stats, alignment)
Row 23:         Status line 2 (HP, power, AC, level, gold, etc.)

Core game state structure:

type GameState struct {
    // Map
    Map          [21][80]MapCell  // rows 1-21
    PlayerPos    Position         // location of '@' on the map
    
    // Messages
    TopMessage   string           // row 0 text
    MorePrompt   bool             // "--More--" detected
    
    // Status bars
    Stats        PlayerStats
    
    // Derived state
    VisibleMonsters []Monster
    NearbyItems     []Item
    DungeonFeatures []Feature      // stairs, doors, altars, fountains
    
    // Prompt detection
    PromptType   PromptType       // None, Direction, YesNo, Inventory, Menu, etc.
    PromptText   string
    
    // Screen mode
    ScreenMode   ScreenMode       // Normal, Inventory, Menu, TextDisplay
}

type PlayerStats struct {
    Name       string
    Race       string
    Role       string
    Alignment  string
    STR        string  // can be "18/07" format
    DEX, CON, INT, WIS, CHA int
    HP, HPMax  int
    PW, PWMax  int
    AC         int
    XPLevel    int
    XP         int
    Gold       int
    Dungeon    string  // "Dungeons of Doom", "Mines", "Sokoban", etc.
    DLevel     int
    Turns      int
    Hunger     string  // "Satiated", "", "Hungry", "Weak", "Fainting"
    Encumbrance string // "", "Burdened", "Stressed", etc.
    Conditions []string // "Conf", "Blind", "Stun", "Hallu", etc.
}

type MapCell struct {
    Char    rune
    FG      Color
    BG      Color
    Type    CellType   // Wall, Floor, Door, Corridor, Stairs, Monster, Item, etc.
    Walked  bool       // have we stepped here before?
}

Parsing challenges:

  1. "--More--" detection: When there are multiple messages, NetHack shows "--More--" and waits for spacebar. The bot must detect this and send space before trying to act.

  2. Prompt detection: Many game actions trigger prompts:

    • "In what direction?" — direction prompt
    • "What do you want to eat? [a-zA-Z?*]" — inventory selection
    • "Really attack the peaceful shopkeeper? [yn]" — yes/no
    • Menu screens (inventory, identify, enhance, etc.)

  3. Overlay screens: Inventory, spell lists, and help screens overlay the map. We need to detect when we're in a menu vs the normal map view.

  4. Symbol identification: NetHack uses the same character for different things depending on color. A complete mapping of character + color → game entity is needed. For example:

    • d (brown) = jackal, d (red) = hell hound pup, d (yellow) = dog
    • . (gray) = floor, . (cyan) = ice, . (green) = swamp

  5. Status line parsing: The bottom two lines have a specific format but can vary by configuration. NAO uses the default format. Regex-based parsing of HP, AC, stats, conditions, etc.

3.4 LLM Client (pkg/llm)

Purpose: Interface with Claude API, construct prompts, parse responses.

API integration:

type LLMClient struct {
    apiKey     string
    model      string  // "claude-sonnet-4-20250514" for speed, opus for harder decisions
    httpClient *http.Client
}

Prompt structure:

The prompt is the soul of this project. It needs to convey:

  1. System prompt: NetHack rules, symbol reference, strategy guidelines
  2. Current game state: Map, stats, messages, nearby entities
  3. Recent history: Last N turns of actions and outcomes
  4. Run context: What we've accomplished, current goals, inventory summary
  5. Action request: What should we do next?

System prompt design (abbreviated):

You are an expert NetHack player controlling a character via single-keystroke
commands. You are playing on nethack.alt.org.

GAME KNOWLEDGE:
- You are the '@' symbol on the map
- Movement: y/k/u/h/l/b/j/n (vi-keys: upleft/up/upright/left/right/downleft/down/downright)
- Common commands: s (search), . (wait), i (inventory), e (eat), q (quaff), r (read), z (zap), w (wield), W (wear), T (takeoff), d (drop), , (pickup), < (upstairs), > (downstairs), # (extended command), P (puton), R (remove), a (apply)
- Pray with #pray (emergency heal when below 1/7 HP, max once per ~700 turns)

MAP SYMBOLS:
- . floor  # corridor  | and - walls  + door  < upstairs  > downstairs
- @ you  d dog/canine  : (eye/sphere)  etc.
- Colors distinguish monster types (red d = dangerous hell hound, brown d = jackal)

CURRENT PRIORITIES:
1. Don't die (manage HP, avoid dangerous fights)
2. Don't starve (eat corpses and food rations)
3. Explore and descend
4. Collect useful items
...

Respond with EXACTLY ONE of these formats:
ACTION: <keystroke>
SEQUENCE: <keystroke1> <keystroke2> ...
THINK: <reasoning>
ACTION: <keystroke>
QUIT: <explanation of why you are stuck and cannot make progress>

Response parsing:

The LLM response must be parsed into actionable keystrokes. The response format is constrained to make parsing reliable. We extract:

  • The ACTION or SEQUENCE line → translate to keystrokes and send
  • The QUIT line → gracefully quit the game, record the explanation
  • Translate named keys (space, enter, escape) to actual bytes
  • Handle direction-based actions that require two keystrokes (e.g., o then h to open a door to the west)

The QUIT response is important: if the bot determines it is stuck (e.g., trapped with no escape, confused by an unrecognizable game state, or in an unwinnable position), it should quit cleanly rather than thrash. The quit explanation is recorded and becomes valuable eval data — it tells us exactly what situations the bot can't handle yet.

Model selection strategy:

  • Use claude-sonnet-4-20250514 for routine decisions (movement, simple combat) — faster and cheaper
  • Optionally escalate to claude-opus-4-20250514 for complex decisions (inventory management, wish decisions, endgame navigation)
  • This can be controlled by a "difficulty estimator" — if multiple monsters visible, low HP, or in endgame, use the stronger model

Token budget considerations:

  • System prompt: ~2,000 tokens (game knowledge, symbol reference)
  • Current screen state: ~500–1,000 tokens (map grid, stats, messages)
  • History: ~1,000–2,000 tokens (last 10-20 turns)
  • Run context: ~500 tokens (inventory summary, accomplishments)
  • Total per turn: ~4,000–5,500 input tokens
  • At ~$3/M input tokens (Sonnet), that's ~$0.015 per turn
  • A typical ascension run is ~30,000–80,000 turns
  • Estimated cost per run: $450–$1,200 with Sonnet, much more with Opus

This cost is significant. Optimizations include: batching multiple trivial moves (long corridors), caching repeated states, and only calling the LLM when the game state actually changes meaningfully.

3.5 Decision Engine / Brain (pkg/brain)

Purpose: Orchestrate the game loop and manage turn-by-turn flow.

Main loop:

func (b *Brain) Run(ctx context.Context) error {
    for {
        // 1. Read PTY until stable (no new bytes for N ms)
        b.terminal.WaitForStable(50 * time.Millisecond) // faster than telnet — local PTY
        
        // 2. Parse game state from terminal buffer
        state := b.parser.Parse(b.terminal.GetScreen())
        
        // 3. Record game state
        b.recorder.RecordState(state.Turns, state)
        
        // 4. Check for death / game over
        if state.IsGameOver() {
            b.recorder.EndRun(b.buildRunResult(state, "death"))
            return nil
        }
        
        // 5. Handle mechanical prompts without LLM
        if action := b.handleMechanical(state); action != "" {
            b.recorder.RecordMechanical(state.Turns, action.Trigger, action.Keys)
            b.execute(action.Keys)
            continue
        }
        
        // 6. Check for stuck detection (pre-LLM)
        if b.stuckDetector.IsHardStuck() {
            reason := b.stuckDetector.Reason()
            b.recorder.RecordQuit(state.Turns, "stuck_detected", reason)
            b.quitGame()
            b.recorder.EndRun(b.buildRunResult(state, "quit_stuck"))
            return nil
        }
        
        // 7. Build context and query LLM
        prompt := b.buildPrompt(state)
        response, err := b.llm.Query(ctx, prompt)
        
        // 8. Parse response — may be action, sequence, or quit
        decision := b.parseResponse(response)
        b.recorder.RecordDecision(state.Turns, decision)
        
        // 9. Handle QUIT response from LLM
        if decision.IsQuit {
            b.recorder.RecordQuit(state.Turns, "llm_quit", decision.QuitReason)
            b.quitGame()
            b.recorder.EndRun(b.buildRunResult(state, "quit_llm"))
            return nil
        }
        
        // 10. Execute action(s)
        for _, key := range decision.Actions {
            b.execute(key)
            time.Sleep(20 * time.Millisecond) // Brief pause between keystrokes
        }
        
        // 11. Wait for outcome, record it
        b.terminal.WaitForStable(50 * time.Millisecond)
        outcome := b.assessOutcome(state)
        b.recorder.RecordOutcome(state.Turns, outcome)
        
        // 12. Update memory and stuck detector
        b.memory.RecordTurn(state, decision, outcome)
        b.stuckDetector.RecordTurn(state, decision)
    }
}

// quitGame sends the quit sequence to NetHack and confirms
func (b *Brain) quitGame() {
    b.execute('#')
    b.terminal.WaitForStable(50 * time.Millisecond)
    b.executeString("quit\n")
    b.terminal.WaitForStable(50 * time.Millisecond)
    b.execute('y') // "Really quit?" → yes
    // NetHack will then show score and exit
}

Stuck detection (brain/loop_detect.go):

The stuck detector runs independently of the LLM, catching degenerate behavior:

type StuckDetector struct {
    positionHistory []Position  // ring buffer of recent positions
    actionHistory   []string    // ring buffer of recent actions
    windowSize      int         // configurable, default 20
    threshold       int         // repeated positions before "stuck", default 5
}

func (s *StuckDetector) IsHardStuck() bool {
    // Hard stuck conditions (quit without asking LLM):
    // 1. Same position visited 5+ times in last 20 turns
    // 2. Exact same action sequence repeated 3+ times
    // 3. No HP/XP/DLevel change in 100+ turns
    // (These are invariants — the bot is clearly not making progress)
}

func (s *StuckDetector) IsSoftStuck() bool {
    // Soft stuck conditions (flag in prompt, let LLM decide):
    // 1. Same position visited 3+ times in last 10 turns
    // 2. No new map tiles explored in 30+ turns
    // (These get injected into the LLM prompt as a warning)
}

When soft-stuck is detected, the prompt includes: "WARNING: You appear to be stuck. You have visited this position 3 times recently. If you cannot make progress, respond with QUIT and explain why." This gives the LLM a chance to try something creative before we force-quit.

Mechanical handlers (bypass LLM for speed/cost):

Some situations don't need LLM reasoning:

  • "--More--" → send space
  • "Really quit?" → send y (only reached when the bot has decided to quit)
  • "Do you want your possessions identified?" → send y (death screen)
  • Corridor movement with only one obvious direction → send direction key
  • "There is a _ here; pick it up?" → can be LLM or rule-based

Screen stability detection:

After sending a keystroke, NetHack redraws the screen via the PTY. We need to wait until the screen is "stable" before parsing — meaning no new bytes have arrived for a threshold period. With a local PTY this is much faster than over telnet (typically <10ms for NetHack to finish drawing), so we use a shorter timeout.

func (t *Terminal) WaitForStable(timeout time.Duration) {
    // Read bytes until no new data for `timeout` duration
    // With local PTY, 50ms is generous — NetHack redraws in <10ms
}

3.6 Memory System (pkg/memory)

Purpose: Maintain context across turns within the LLM's context window limits.

Components:

  1. Turn History (rolling window):

    • Last 15-20 turns: full map state + action + result
    • Compressed to essential info: "Turn 1042: Moved east, saw goblin, killed it"

  2. Run Summary (persistent, updated periodically):

    • Current character: role, race, level, alignment
    • Key inventory (weapons, armor, rings, amulets, important tools)
    • Dungeon progress: levels visited, branches explored
    • Major events: quest status, artifact found, etc.
    • Current strategy/goal

  3. Knowledge Base (static reference):

    • Symbol reference sheet (what each character+color means)
    • Common strategies (Sokoban solutions, sacrifice rules, etc.)
    • Dangerous monsters by level (what to avoid)

  4. Spatial Memory:

    • Remembered map of visited levels (since the terminal only shows current level)
    • Location of important features: altars, shops, stashes, stairs

Context window management:

Claude's context window is large but not infinite. The memory system must prioritize what to include:

Priority 1: Current screen + messages (always included)
Priority 2: Recent turn history (last 5 turns always, up to 20 if space)
Priority 3: Run summary (always included, kept concise)
Priority 4: Relevant knowledge (included based on current situation)
Priority 5: Spatial memory for current branch (if space permits)

The run summary should be regenerated every ~50 turns by asking the LLM to summarize the current run state, replacing the old summary.

3.7 Action Executor (pkg/action)

Purpose: Translate high-level actions to byte sequences and send them.

Action types:

Action Keys Notes
Move direction h/j/k/l/y/u/b/n vi-keys
Move until blocked H/J/K/L/Y/U/B/N shift = run
Wait . Single turn wait
Search s Search for hidden doors/passages
Pickup , Grab item from floor
Eat e then inventory letter Two-step
Quaff q then inventory letter Two-step
Read r then inventory letter Two-step
Zap z then inventory letter then direction Three-step
Open door o then direction Two-step
Go upstairs < Must be on <
Go downstairs > Must be on >
Inventory i Display only
Extended # then command name then enter Variable length
Pray #pray\n Extended command

Multi-step action handling:

Many actions are multi-keystroke. The executor needs to handle intermediate prompts:

func (e *Executor) Eat(inventoryLetter rune) {
    e.send('e')
    e.waitForPrompt()  // "What do you want to eat?"
    e.send(inventoryLetter)
    // May get follow-up: "There is a foo corpse here; eat it? [ynq]"
}

3.8 Recording System (pkg/recording)

Purpose: Produce the primary output artifact of every run — a rich, structured recording that captures everything needed for offline analysis and LLM-driven critique.

The recording system is not just logging — it's the foundation of the improvement loop. Every recording must be self-contained: a critic agent should be able to fully understand what happened, why the bot made each decision, and what the consequences were, without needing to replay the game.

Recording format — two complementary files per run:

File 1: TTYRec (visual replay)

Standard ttyrec format — raw byte stream with timestamps. Can be replayed with ttyplay, termplay, or any ttyrec-compatible tool. Useful for human review and visual debugging.

Filename: recordings/{run_id}.ttyrec
Format:   [sec:4][usec:4][len:4][data:len] repeated

File 2: Decision Log (JSONL — machine-readable)

One JSON object per line, capturing every meaningful event. This is the primary input for the eval agent.

Filename: recordings/{run_id}.jsonl

Event types in the decision log:

// === Run metadata (first line) ===
{
  "type": "run_start",
  "run_id": "2026-03-11T14-30-00_val-dwa-neu",
  "timestamp": "2026-03-11T14:30:00Z",
  "nethack_version": "3.6.7",
  "seed": 1741234567,
  "character": {"role": "Valkyrie", "race": "Dwarf", "alignment": "Neutral", "gender": "Female"},
  "config_hash": "a1b2c3d4",  // Hash of .nethackrc + prompt templates for reproducibility
  "prompt_version": "v3"       // Which iteration of prompts was used
}

// === Game state snapshot (every turn where LLM is consulted) ===
{
  "type": "state",
  "turn": 1042,
  "timestamp": "2026-03-11T14:35:12Z",
  "map_ascii": "... (21×80 raw map) ...",
  "player": {"pos": [34, 12], "hp": 45, "hp_max": 62, "pw": 12, "pw_max": 30,
             "ac": 2, "xl": 8, "dlvl": 5, "dungeon": "Dungeons of Doom",
             "str": "18/04", "dex": 14, "con": 18, "int": 7, "wis": 10, "cha": 10,
             "hunger": "", "encumbrance": "", "conditions": [], "gold": 312, "turns": 1042},
  "messages": ["The goblin hits you!", "You kill the goblin!"],
  "visible_monsters": [{"char": "o", "color": "red", "pos": [36, 11], "name": "orc"}],
  "nearby_items": [{"char": ")", "color": "cyan", "pos": [34, 13], "desc": "weapon"}],
  "features": [{"type": "stairs_down", "pos": [40, 8]}]
}

// === LLM decision (the core record — one per LLM call) ===
{
  "type": "decision",
  "turn": 1042,
  "timestamp": "2026-03-11T14:35:12Z",
  "tier": "tactical",
  "model": "claude-sonnet-4-20250514",
  "prompt_tokens": 4200,
  "completion_tokens": 85,
  "latency_ms": 1240,
  "prompt_summary": "Standard tactical — 1 monster visible, moderate HP",
  "full_prompt_hash": "e5f6a7b8",  // Can retrieve full prompt from prompt cache
  "reasoning": "There's an orc 2 tiles NE. At HP 45/62 and XL 8, I should engage. Moving toward it via 'u' (up-right). My Excalibur should handle it easily.",
  "action": "u",
  "alternatives_considered": ["k (move north to avoid)", "s (search, waste of time)"],
  "confidence": "high"
}

// === Mechanical action (no LLM involved) ===
{
  "type": "mechanical",
  "turn": 1043,
  "timestamp": "2026-03-11T14:35:13Z",
  "trigger": "more_prompt",
  "action": " ",
  "description": "Dismissed --More-- prompt"
}

// === Outcome / consequence tracking ===
{
  "type": "outcome",
  "turn": 1043,
  "timestamp": "2026-03-11T14:35:13Z",
  "hp_delta": -8,
  "messages": ["The orc hits you!"],
  "player_died": false,
  "notable": false
}

// === Periodic summary (every N turns, for eval context) ===
{
  "type": "summary",
  "turn": 1050,
  "timestamp": "2026-03-11T14:36:00Z",
  "run_summary": "Valkyrie DL5, XL8, HP62. Have Excalibur. Cleared mines to ML3. Heading toward Sokoban.",
  "inventory_snapshot": ["Excalibur (wielded)", "+1 small shield (worn)", "17 food rations", "..."],
  "levels_visited": ["D:1-5", "Mines:1-3"],
  "key_events_since_last": ["Found Excalibur at fountain on D:4", "Killed floating eye with ranged attack"]
}

// === Run end (last line) ===
{
  "type": "run_end",
  "turn": 2847,
  "timestamp": "2026-03-11T15:12:00Z",
  "cause": "killed_by",      // "killed_by" | "quit_stuck" | "quit_llm" | "ascended" | "escaped"
  "death_reason": "killed by a blue dragon",    // null if quit
  "quit_reason": null,                           // null if died; explanation string if quit
  "final_score": 14230,
  "deepest_level": 12,
  "total_decisions": 847,
  "total_mechanical": 2000,
  "llm_cost_usd": 12.40,
  "duration_minutes": 42,
  "end_context": {
    "hp_at_end": 0,
    "hp_before_event": 34,
    "monster": "blue dragon",               // null if quit
    "could_have_prayed": true,
    "had_escape_item": false,
    "last_reasoning": "Tried to melee the blue dragon. Should have used Elbereth or retreated.",
    "stuck_details": null                   // populated if quit_stuck: position loop count, actions repeated, etc.
  }
}

// === Example: bot quit because it was stuck ===
{
  "type": "run_end",
  "turn": 412,
  "timestamp": "2026-03-11T14:42:00Z",
  "cause": "quit_llm",
  "death_reason": null,
  "quit_reason": "I am trapped in a room with locked doors on all sides. I have no lock picks, no kick ability (wounded leg), and no spells. I've searched all walls 5 times with no hidden exits. There is no way to make progress.",
  "final_score": 1230,
  "deepest_level": 3,
  "total_decisions": 120,
  "total_mechanical": 292,
  "llm_cost_usd": 1.80,
  "duration_minutes": 6,
  "end_context": {
    "hp_at_end": 34,
    "hp_before_event": 34,
    "monster": null,
    "could_have_prayed": true,
    "had_escape_item": false,
    "last_reasoning": "All exits are locked. I have no tools to open them. Quitting.",
    "stuck_details": {"position_repeats": 8, "turns_without_progress": 45}
  }
}

Full prompt caching:

To keep the JSONL manageable, full prompts are stored separately:

recordings/{run_id}/prompts/{hash}.txt

The decision log references prompts by hash. This avoids duplicating the system prompt (which is mostly static) thousands of times while keeping full reproducibility.

Recording writer interface:

type Recorder interface {
    StartRun(config RunConfig) error
    RecordState(turn int, state *GameState) error
    RecordDecision(turn int, decision *Decision) error
    RecordMechanical(turn int, trigger string, action string) error
    RecordOutcome(turn int, outcome *Outcome) error
    RecordQuit(turn int, cause string, reason string) error
    RecordSummary(turn int, summary *RunSummary) error
    EndRun(result *RunResult) error
    Close() error
}

3.9 Eval / Critique Agent (cmd/eval)

Purpose: Offline tool that reviews game recordings and produces actionable improvement feedback.

This is a separate binary (nethack-llm-eval) that reads JSONL recordings and uses Claude to analyze decision quality. It's the "coach" that reviews game tape.

Eval modes:

Mode 1 — Death / Quit Autopsy

Triggered automatically after every run. Analyzes differently based on how the run ended:

For deaths — examines the final 20-30 turns leading up to death:

Input:  Last 30 decision records + end_context from run_end
Output: Death autopsy report

Questions the eval answers:
- What killed the bot?
- When did the situation become dangerous? Which turn?
- What was the last decision that could have prevented death?
- Was the right information available in the prompt at that point?
- What rule or heuristic would have saved the bot?

For quits — examines how the bot got stuck and whether the quit was justified:

Input:  Last 50 decision records + quit_reason + stuck_details
Output: Quit analysis report

Questions the eval answers:
- Was the bot actually stuck, or did it give up too early?
- Was there a creative solution it missed? (kick door, pray, dig, polymorph, etc.)
- How did it get into this situation? Which earlier decision led here?
- Is this a common stuck pattern? (If so, add a rule to the knowledge base)
- Should the prompt include escape strategies for this type of situation?

Mode 2 — Strategic Review

Analyzes an entire run at the summary/milestone level:

Input:  All summary records + key decisions (inventory, branch choice, etc.)
Output: Strategic review report

Questions:
- Did the bot explore efficiently? (turns per level)
- Were branch decisions correct? (mines vs sokoban ordering)
- Was inventory management reasonable? (dropped good items? hoarded junk?)
- Were there missed opportunities? (altar not used, fountain not dipped, etc.)
- How does this run compare to typical human ascension runs at this stage?

Mode 3 — Decision Spot-Check

Samples N random decisions from the run and evaluates each one:

Input:  N randomly sampled decision records with surrounding context
Output: Per-decision grades (good / acceptable / suboptimal / bad) with explanations

This catches:
- Systematic errors the bot makes repeatedly
- Situations where the bot's reasoning is sound but conclusion is wrong
- Prompt engineering gaps (bot didn't have needed info)

Mode 4 — Comparative Analysis

Compares two or more runs (e.g., before and after a prompt change):

Input:  Two or more JSONL recordings
Output: Comparison report — what improved, what regressed, statistical differences

Eval output format:

// recordings/{run_id}.eval.json
{
  "run_id": "2026-03-11T14-30-00_val-dwa-neu",
  "eval_timestamp": "2026-03-11T16:00:00Z",
  "eval_model": "claude-opus-4-20250514",
  
  "death_autopsy": {
    "root_cause": "Engaged blue dragon in melee without cold resistance",
    "critical_turn": 2841,
    "could_have_survived": true,
    "missed_action": "Should have written Elbereth or used upstairs at turn 2841",
    "knowledge_gap": "Bot prompt doesn't emphasize elemental dragon breath dangers",
    "recommended_rule": "NEVER melee a dragon without the matching resistance. Flee or use Elbereth."
  },
  
  "strategic_review": {
    "exploration_efficiency": "adequate — 45 turns/level average",
    "branch_ordering": "good — mines before sokoban",
    "inventory_management": "poor — carried 3 identified junk scrolls for 200+ turns",
    "missed_opportunities": ["Didn't dip for Excalibur until DL:6 despite finding fountain on DL:4"]
  },
  
  "spot_check_results": {
    "sample_size": 50,
    "grades": {"good": 32, "acceptable": 11, "suboptimal": 5, "bad": 2},
    "common_issues": [
      "Repeatedly searches walls that have already been searched",
      "Doesn't prioritize eating when hunger is 'Hungry' (waits until 'Weak')"
    ]
  },
  
  "recommended_prompt_changes": [
    {
      "section": "system/combat_rules",
      "change": "Add: 'Never melee dragons without matching resistance. Blue=cold, Red=fire, etc.'",
      "priority": "critical",
      "evidence_turns": [2841, 2843, 2845]
    },
    {
      "section": "system/hunger_rules", 
      "change": "Change eating threshold from 'Weak' to 'Hungry'",
      "priority": "high",
      "evidence_turns": [312, 567, 891]
    }
  ]
}

Eval pipeline invocation:

# Run death autopsy on most recent run
nethack-llm-eval --mode=autopsy recordings/latest.jsonl

# Full review
nethack-llm-eval --mode=full recordings/2026-03-11T14-30-00.jsonl

# Compare two runs
nethack-llm-eval --mode=compare recordings/run1.jsonl recordings/run2.jsonl

# Spot-check with custom sample size
nethack-llm-eval --mode=spot-check --samples=100 recordings/latest.jsonl

4. Prompt Engineering Strategy

This is where the project succeeds or fails. The LLM's ability to play NetHack well depends entirely on how we frame the game state and guide its reasoning.

4.1 State Representation

Option A — Raw ASCII map:

Send the actual 80×24 screen (or the 80×21 map portion) as text. This is simple and preserves all information, but uses more tokens and requires the LLM to interpret raw symbols.

                  --------
                  |......|
                  |......+########
                  |..@...|       #
                  --------       #
                                 #
                          ########
                          #
                   -------+------
                   |.............|
                   |.............|
                   ---------------

Option B — Structured representation:

Parse the map into a structured format with annotated entities:

Player at (21, 3). Current room: 8×4, exits: east door (open).
Visible entities:
  - Goblin (g) at (24, 5), 2 tiles SE
  - Food ration at (20, 3), 1 tile W
Dungeon features:
  - Open door at (25, 3)
  - Corridor leading east from door
  - Staircase down at (19, 10)

Recommendation: Use both. Send the raw map (the LLM can read ASCII maps) plus structured annotations for key entities. This gives the LLM spatial reasoning ability while ensuring it doesn't miss important details.

4.2 Decision Tiers

Not all decisions are equal. We should tier them:

Tier 1 — Reflexive (no LLM needed):

  • Press space on "--More--"
  • Move through a corridor with only one exit
  • Pick up gold automatically

Tier 2 — Tactical (fast model, simple prompt):

  • Which direction to explore
  • Whether to fight or flee a single monster
  • Whether to pick up a visible item
  • Basic movement decisions

Tier 3 — Strategic (full prompt, possibly stronger model):

  • Inventory management (what to keep, drop, use)
  • Whether to descend or keep exploring current level
  • Handling shops, altars, fountains
  • Eating decisions (corpse safety, timing)
  • Equipment optimization

Tier 4 — Critical (strongest model, extended reasoning):

  • Wish decisions
  • Genocide targets
  • Near-death crisis management
  • Endgame navigation
  • Quest decisions

4.3 Prompt Templates

Minimal tactical prompt:

[NetHack Turn {turn} | DL:{dlvl} | HP:{hp}/{hpmax} | {hunger}]

MAP:
{raw_map_grid}

MSG: {last_message}
NEARBY: {annotated_entities}
STATS: STR:{str} DEX:{dex} CON:{con} INT:{int} WIS:{wis} CHA:{cha} AC:{ac} XL:{xl}

Last 5 actions: {recent_history}

What single keystroke should I press? Respond: THINK: <brief reasoning> then ACTION: <key>

Full strategic prompt:

[System: detailed NetHack knowledge base, symbol reference, strategy guide]

=== CURRENT STATE (Turn {turn}) ===
{full map with annotations}

=== PLAYER STATUS ===
{detailed stats, conditions, encumbrance}

=== INVENTORY SUMMARY ===
{categorized inventory}

=== RECENT EVENTS (last 15 turns) ===
{turn-by-turn history}

=== RUN CONTEXT ===
{run summary: progress, goals, strategy}

=== DECISION NEEDED ===
{specific situation description and what needs to be decided}

4.4 Known LLM Weaknesses to Mitigate

  1. Spatial reasoning: LLMs struggle with exact grid coordinates. Mitigate with clear annotations ("goblin is 2 tiles east") and possibly a local neighborhood view (5×5 around player).

  2. Memory across turns: The LLM has no persistent memory. Mitigate with the memory system feeding context each turn.

  3. Obscure game knowledge: Claude knows NetHack but may confuse details. Mitigate with a knowledge base in the system prompt for critical rules (prayer cooldown, intrinsic resistances, safe corpses, etc.).

  4. Repetitive behavior: LLMs can get stuck in loops (go east, go west, go east...). Mitigate with loop detection in the brain module — if the same position is visited 3+ times in 10 turns, flag it in the prompt.

  5. Risk calibration: LLMs may be too aggressive or too cautious. Mitigate with explicit HP thresholds in the prompt ("if HP < 30%, consider retreating or praying").

5. Engineering Plan

5.1 Project Structure

nethack-llm/
├── cmd/
│   ├── nethack-llm/
│   │   └── main.go              # Entry point — play bot
│   └── nethack-llm-eval/
│       └── main.go              # Entry point — eval/critique tool
├── pkg/
│   ├── pty/
│   │   ├── driver.go            # PTY allocation, subprocess launch, I/O
│   │   ├── driver_test.go
│   │   └── nethackrc.go         # .nethackrc generation
│   ├── terminal/
│   │   ├── terminal.go          # 80×24 cell buffer
│   │   ├── parser.go            # VT100 escape sequence state machine
│   │   ├── parser_test.go       # Test with known NetHack screen dumps
│   │   ├── cell.go              # Cell type, color definitions
│   │   └── testdata/            # Captured screen sequences for testing
│   ├── gamestate/
│   │   ├── state.go             # GameState struct
│   │   ├── parser.go            # Screen → GameState extraction
│   │   ├── status.go            # Status line parsing
│   │   ├── map.go               # Map cell classification
│   │   ├── message.go           # Message line + prompt detection
│   │   ├── symbols.go           # Character+color → entity mapping
│   │   └── parser_test.go
│   ├── llm/
│   │   ├── client.go            # Claude API HTTP client
│   │   ├── prompt.go            # Prompt template construction
│   │   ├── response.go          # Response parsing (ACTION/SEQUENCE extraction)
│   │   └── client_test.go
│   ├── brain/
│   │   ├── brain.go             # Main game loop
│   │   ├── mechanical.go        # Mechanical handlers (--More--, etc.)
│   │   ├── tiers.go             # Decision tier classification
│   │   ├── loop_detect.go       # Stuck/loop detection
│   │   └── quit.go              # Graceful quit handling (send #quit, confirm)
│   ├── memory/
│   │   ├── history.go           # Turn history ring buffer
│   │   ├── summary.go           # Run summary management
│   │   ├── spatial.go           # Per-level map memory
│   │   └── knowledge.go         # Static game knowledge
│   ├── action/
│   │   ├── executor.go          # Keystroke sending via PTY
│   │   ├── commands.go          # Named command → keystroke mapping
│   │   └── multi_step.go        # Multi-keystroke action handling
│   ├── recording/
│   │   ├── recorder.go          # Recording coordinator
│   │   ├── ttyrec.go            # TTYRec format writer
│   │   ├── jsonl.go             # JSONL decision log writer
│   │   ├── events.go            # Event type definitions
│   │   ├── prompt_cache.go      # Full prompt dedup/storage by hash
│   │   └── reader.go            # Read recordings (used by eval)
│   ├── eval/
│   │   ├── autopsy.go           # Death analysis
│   │   ├── strategic.go         # Full-run strategic review
│   │   ├── spot_check.go        # Random decision sampling + grading
│   │   ├── compare.go           # Multi-run comparison
│   │   └── report.go            # Report generation (JSON + human-readable)
│   └── config/
│       └── config.go            # Configuration loading
├── data/
│   ├── symbols.json             # Monster/item symbol database
│   ├── knowledge.md             # NetHack knowledge base for prompts
│   └── prompts/                 # Prompt templates
│       ├── system.txt           # Main system prompt
│       ├── tactical.txt         # Tier 2 decisions
│       ├── strategic.txt        # Tier 3 decisions
│       ├── eval_autopsy.txt     # Eval agent: death analysis prompt
│       ├── eval_strategic.txt   # Eval agent: strategic review prompt
│       └── eval_spotcheck.txt   # Eval agent: decision grading prompt
├── recordings/                  # Output directory for game recordings
│   └── .gitkeep
├── go.mod
├── go.sum
├── Makefile
└── README.md

5.2 Sprint Plan

Sprint 1 (Week 1-2): Foundation — PTY + Terminal + Recording

Task Est. Hours Description
Project scaffolding 2 Go module init, directory structure, Makefile
PTY driver 6 Subprocess launch, PTY allocation, byte stream I/O via creack/pty
.nethackrc generator 3 Generate bot-optimized config, manage NETHACKOPTIONS env
VT100 state machine 16 Escape sequence parser, cell buffer, cursor handling
TTYRec writer 3 Record raw byte stream in ttyrec format
JSONL writer skeleton 3 Recording infrastructure, event types, run start/end
Integration test 4 Launch NetHack, dump parsed screen, verify round-trip
Total 37

Deliverable: Launch local NetHack, parse the screen, produce a ttyrec + skeleton JSONL recording.

Sprint 2 (Week 3-4): Game State Parser

Task Est. Hours Description
Status line parser 6 Regex-based HP/stats/conditions extraction
Message line parser 4 Top message, --More-- detection
Prompt detection 8 Direction, YN, inventory, menu prompts
Map cell classification 8 Symbol + color → entity type mapping
Monster/item database 6 Populate symbols.json with NetHack data
Overlay screen detection 4 Inventory/menu vs normal map
Game state tests 6 Test against captured screen data
Total 42

Deliverable: Full structured game state extraction from live game screen.

Sprint 3 (Week 5-6): LLM Integration + Basic Bot + Recording

Task Est. Hours Description
Claude API client 6 HTTP client, auth, retry logic
Prompt templates 10 System prompt, tactical prompt, state formatting
Response parser 4 Extract ACTION/SEQUENCE from LLM response
Action executor 6 Keystroke sending via PTY, multi-step actions
Mechanical handlers 4 --More--, corridor following, trivial prompts
Main game loop 8 Brain module, turn flow, stability detection
Full recording integration 6 Wire recorder into game loop — state, decision, outcome events
Prompt cache 3 Hash-based dedup of full prompts for recording storage
Total 47

Deliverable: Bot plays NetHack autonomously, exploring and fighting. Produces complete JSONL recordings with every decision and its reasoning. Likely dies frequently but records everything.

Sprint 4 (Week 7-8): Memory + Survival

Task Est. Hours Description
Turn history 6 Rolling window of recent turns
Run summary 8 Periodic LLM-generated summary
Spatial memory 10 Per-level explored map storage
Knowledge base 8 Critical NetHack rules for prompt injection
Loop detection 4 Detect and flag repetitive behavior
Hunger management 4 Prompt tuning for eat timing
Total 40

Deliverable: Bot survives significantly longer, manages food, remembers where it's been.

Sprint 5 (Week 9-10): Decision Quality + Inventory

Task Est. Hours Description
Decision tier system 6 Classify situations by complexity
Model selection 4 Route to Sonnet vs Opus by tier
Inventory management prompts 10 Equipment optimization, drop decisions
Strategic prompt template 8 Full context for complex decisions
Prayer/emergency handling 4 Crisis detection and response
Cost optimization 6 Skip LLM for trivial moves, batch corridors
Total 38

Deliverable: Bot makes smarter decisions, manages inventory, handles emergencies.

Sprint 6 (Week 11-12): Eval Agent + Improvement Loop

Task Est. Hours Description
Recording reader 4 Parse JSONL recordings for eval consumption
Death autopsy eval 8 Analyze final turns, identify root cause, suggest rules
Strategic review eval 8 Full-run analysis, efficiency metrics, missed opportunities
Spot-check eval 6 Random decision sampling, grading, pattern detection
Eval report generation 4 JSON + human-readable output
Prompt iteration pipeline 6 Apply eval recommendations → update prompts → re-run → compare
Documentation 4 README, setup guide, recording format spec, eval usage
Total 40

Deliverable: Complete improvement loop — bot plays, eval agent critiques, findings feed back into prompts. Comparative analysis shows measurable improvement across runs.

5.3 Total Estimate

Phase Weeks Hours
Sprint 1: Foundation (PTY + Terminal + Recording) 2 37
Sprint 2: Game State 2 42
Sprint 3: LLM + Bot + Recording 2 47
Sprint 4: Memory + Survival 2 40
Sprint 5: Decisions + Inventory 2 38
Sprint 6: Eval Agent + Improvement Loop 2 40
Total 12 weeks 244 hours

6. Key Technical Risks

6.1 VT100 Emulation Fidelity

Risk: Incomplete escape sequence handling causes garbled screen parsing.
Mitigation: Record raw PTY byte streams from real game sessions and build a comprehensive test suite. Start with the most common sequences and add more as issues arise. Local execution makes it easy to capture test data — just tee the PTY output. Consider using go-vte as a base and extending it.

6.2 Screen Stability / Timing

Risk: We parse the screen mid-update, getting a half-drawn frame.
Mitigation: Wait for a quiescent period (no new bytes for 50ms) before parsing. With local PTY this is much less of an issue than over telnet — NetHack typically finishes screen redraws in <10ms locally. The 50ms timeout is generous and reliable.

6.3 Prompt Quality

Risk: The LLM makes poor decisions, gets stuck in loops, or misunderstands the game state.
Mitigation: Iterative prompt engineering based on replay analysis. Build a feedback loop: play → record → analyze deaths → improve prompts → repeat. The logging/replay system is essential infrastructure for this.

6.4 Cost

Risk: At $0.015/turn with potentially 50K+ turns per run, costs add up fast.
Mitigation: Aggressive use of mechanical handlers, corridor batching, and tier-based model selection. Most turns in NetHack are "walk down this corridor" — these don't need an LLM call. Target <30% of turns needing LLM involvement.

6.5 Eval Agent Quality

Risk: The eval/critique agent produces vague or unhelpful feedback that doesn't translate to actionable prompt improvements.
Mitigation: Design eval prompts to demand specificity — every critique must cite exact turn numbers, quote the bot's reasoning, and propose a concrete rule change. Grade eval output quality manually for the first several runs. Iterate on eval prompts just as aggressively as gameplay prompts.

6.6 Recording Volume

Risk: JSONL recordings become very large for long runs (50K+ turns × detailed state snapshots).
Mitigation: Only record full state snapshots on turns where the LLM is consulted. Mechanical turns get minimal records. Prompt dedup via hash-based cache avoids storing the (mostly static) system prompt thousands of times. Target <50MB per run for the JSONL, which is manageable.

6.6 LLM Hallucination on Game Rules

Risk: Claude misremembers NetHack rules (e.g., which corpses are safe to eat, prayer cooldown timing).
Mitigation: Include critical rules explicitly in the knowledge base injected into the system prompt. Don't rely on the LLM's training knowledge for safety-critical game decisions.

7. Configuration

# config.yaml
game:
  binary: "/usr/bin/nethack"       # Path to nethack executable
  nethackrc: "auto"                # "auto" = generate bot-optimized, or path to custom
  seed: 0                          # 0 = random, any other value = deterministic (NH 3.7+)
  character:
    role: "Valkyrie"
    race: "Dwarf"
    alignment: "Neutral"
    gender: "Female"

terminal:
  width: 80
  height: 24
  stability_timeout_ms: 50         # Faster than telnet — local PTY

llm:
  api_key: "${ANTHROPIC_API_KEY}"
  default_model: "claude-sonnet-4-20250514"
  strategic_model: "claude-opus-4-20250514"
  max_tokens: 300
  temperature: 0.3       # Lower = more consistent play
  retry_attempts: 3
  retry_delay_ms: 1000

brain:
  mechanical_handlers: true
  corridor_batching: true
  loop_detection_window: 10
  loop_threshold: 3
  decision_delay_ms: 20   # Pause between keystrokes (fast — local)

memory:
  history_window: 20       # Turns of full history
  summary_interval: 50     # Regenerate summary every N turns
  spatial_memory: true

recording:
  dir: "./recordings"
  ttyrec: true             # Visual replay file
  jsonl: true              # Structured decision log
  prompt_cache: true       # Dedup full prompts by hash
  full_state_on_mechanical: false  # Skip full state snapshots for mechanical turns

eval:
  model: "claude-opus-4-20250514"  # Use strongest model for critique
  autopsy_context_turns: 30        # How many turns before death to analyze
  spot_check_sample_size: 50       # Decisions to sample per run
  log_level: "info"

8. Dependencies

Dependency Purpose Go Module
PTY allocation Launch NetHack subprocess github.com/creack/pty
HTTP client Claude API net/http (stdlib)
JSON API serialization, recording encoding/json (stdlib)
YAML Configuration gopkg.in/yaml.v3
CLI flags Command-line interface github.com/spf13/cobra or flag (stdlib)
Logging Structured logging log/slog (stdlib, Go 1.21+)
Hashing Prompt dedup in recordings crypto/sha256 (stdlib)
Testing Unit + integration tests testing (stdlib)

Intentionally minimal. Go's standard library covers most needs. The only required external dependency is creack/pty for pseudoterminal allocation and a YAML parser for config.

9. Future Considerations (Post-v1)

NAO/remote mode: Add telnet support to play on nethack.alt.org or hardfought.org for competitive play and tournaments. The PTY driver interface is abstract enough that a telnet driver can implement the same interface.

Multi-agent architecture: Use separate LLM "specialists" for different aspects — a tactical combat agent, an inventory manager, a navigation planner — coordinated by a meta-agent.

Automated improvement pipeline: Chain play → eval → prompt update → play into a fully automated loop. Run N games, eval all recordings, generate consolidated prompt improvements, apply them, run N more games, measure improvement. This is the end goal of the recording architecture.

Fine-tuning: Collect successful game transcripts and fine-tune a smaller model specifically for NetHack decision-making. The JSONL recordings are already in the right format for training data extraction.

Monte Carlo Tree Search hybrid: For critical decisions (wishes, genocide), generate multiple candidate actions and simulate outcomes before committing.

Web dashboard: Real-time visualization of the bot playing, with game state, LLM reasoning, and statistics displayed in a browser. Recordings can also be replayed in the dashboard.

Eval leaderboard: Track performance metrics across runs (deepest level, turns survived, score, XP) and correlate with prompt versions to measure improvement over time.

Deterministic replay: Using NH 3.7+ seed support, replay interesting runs with different prompts to see if a strategy change would have helped at a specific point.

10. Resolved Decisions

  1. NetHack version: Use whatever is installed on the system (latest). The bot detects the version at startup and logs it in the run metadata.

  2. Character selection: Configurable, defaulting to Valkyrie (Dwarven, Neutral) for v1. The eval pipeline is scoped per-character-type — prompt templates and knowledge base entries may differ across roles. Future roles will need their own eval baselines.

  3. NetHack installation: Assumed pre-installed. The bot expects a nethack binary on PATH (or configured via game.binary in config). No Docker or install scripts in v1.

  4. Eval agent model: Configurable, defaulting to Opus. Opus produces higher-quality critique and we run eval infrequently compared to gameplay, so the cost is acceptable.

  5. Checkpoint/resume: We rely on NetHack's built-in save system. If the bot process crashes, NetHack has already saved state. On restart, the bot detects an existing save file and resumes. We do not build custom memory checkpointing — the recording system captures everything needed, and the memory module can reconstruct from the current game screen.

  6. Parallel runs: Future work. The local PTY architecture makes this trivial (launch N processes), but we want the single-run improvement loop working well first.

  7. Stuck handling: The bot can quit. If the LLM determines it is stuck and cannot make progress, it responds with QUIT: <explanation>. Additionally, a hardcoded stuck detector force-quits if the bot is looping (same position 5+ times in 20 turns, or no progress in 100+ turns). Both quit paths record a detailed explanation that feeds into the eval system. This is strictly bot-only — no human-in-the-loop.