igrr/agent_dev_utils

readme

# agent_dev_utils

AI agent debugging utilities for ESP-IDF projects.

## Overview

This ESP-IDF component provides tools for AI coding agents to analyze application crashes through structured JSON output. It packages test execution, coredump parsing, and crash analysis into easy-to-use `idf.py` commands.

## Features

- Structured crash data instead of unstructured logs
- Automatically shows code around crash location
- Run and analyze multiple test files
- Clear backtrace with file:line information
- Compare what was expected vs what happened
- Save results for later analysis
- QEMU support: Run tests in emulator without physical hardware

## Installation

Add this component to your ESP-IDF project:

```bash
idf.py add-dependency igrr/agent_dev_utils
```

## Quick Start

Run tests on your ESP32 hardware:

```bash
idf.py run-project
```

View AI agent debugging instructions:

```bash
idf.py agent-info
```

## Commands

### `idf.py run-project`

Runs pytest-embedded tests on physical hardware and outputs structured JSON for crash analysis.

**Basic usage**:
```bash
idf.py run-project
```

**With options**:
```bash
idf.py run-project --target esp32s3 --baud 115200 --output-file results.json
```

**QEMU support**:
```bash
idf.py run-project --qemu
```

**Options**:
- `--target <chip>`: Target chip (esp32, esp32s3, etc.) [default: esp32]
- `--baud <rate>`: Serial baud rate [default: 921600]
- `--skip-build`: Skip the build step
- `--output-file <path>`: Save JSON to file
- `--test-pattern <glob>`: Test file pattern [default: test_*.py]
- `--case <filter>` / `-k <filter>`: Filter test cases by name (passed to pytest -k)
- `--verbose`: Show pytest output
- `--qemu`: Run tests in QEMU instead of on physical hardware

### `idf.py agent-info`

Displays comprehensive markdown instructions for AI agents on analyzing ESP-IDF crashes.

```bash
idf.py agent-info
```

## JSON Output Examples

### Crash with Debugging Hints

When a crash occurs, the output includes automatic debugging hints based on the exception type:

```json
{
  "status": "crashed",
  "task_name": "main",
  "exception": {
    "code": "0x1c",
    "name": "LoadProhibited",
    "faulting_address": "0x00000044",
    "hints": "Likely causes:\n  - NULL pointer dereference\n  - Accessing structure member through NULL pointer\n  - Use-after-free (accessing freed memory)\n  - Uninitialized pointer\n\nDebugging tips:\n  - Check EXCVADDR register for the faulting address\n  - Address near 0x0 indicates NULL pointer dereference\n  - Address looks like garbage suggests corrupted/uninitialized pointer\n  - Review recent memory allocations and frees\n\nFaulting address: 0x00000044\n  -> Address near zero suggests NULL pointer dereference"
  },
  "stack_trace": [
    {
      "frame": 0,
      "function": "app_main",
      "file": "main/app.c",
      "line": 42
    }
  ],
  "source_context": "    40 | void app_main(void) {\n    41 |     int *ptr = NULL;\n>>> 42 |     *ptr = 123;\n    43 | }",
  "test_expected": "Expected output: Hello world!",
  "test_actual": "Crashed: Guru Meditation Error: Core 0 panic'ed (LoadProhibited)"
}
```

### Success with Unity Test Results

When firmware uses the Unity test framework, individual test results are parsed:

```json
{
  "status": "success",
  "test_file": "test_example.py",
  "test_expected": "Test passed",
  "test_actual": "Test passed",
  "duration_seconds": 12.5,
  "unity_results": {
    "total": 5,
    "passed": 4,
    "failed": 0,
    "ignored": 1,
    "tests": [
      {"name": "test_addition", "status": "PASS", "file": "test/test_math.c", "line": 10},
      {"name": "test_subtraction", "status": "PASS", "file": "test/test_math.c", "line": 20},
      {"name": "test_multiplication", "status": "PASS", "file": "test/test_math.c", "line": 30},
      {"name": "test_division", "status": "PASS", "file": "test/test_math.c", "line": 40},
      {"name": "test_floating_point", "status": "IGNORE", "file": "test/test_math.c", "line": 50, "message": "Not implemented"}
    ]
  }
}
```

### Multi-Test Suite Output

When running multiple test files, results are aggregated:

```json
{
  "project_path": "/home/user/my_project",
  "target": "esp32",
  "total_tests": 3,
  "passed": 2,
  "failed": 0,
  "crashed": 1,
  "tests": [
    {
      "test_file": "test_basic.py",
      "status": "success",
      "duration_seconds": 8.2
    },
    {
      "test_file": "test_network.py",
      "status": "success",
      "duration_seconds": 15.7,
      "unity_results": {"total": 10, "passed": 10, "failed": 0, "ignored": 0}
    },
    {
      "test_file": "test_stress.py",
      "status": "crashed",
      "exception": {
        "name": "StoreProhibited",
        "faulting_address": "0x00000000",
        "hints": "Likely causes:\n  - Writing through NULL pointer\n  ..."
      },
      "stack_trace": [{"frame": 0, "function": "stress_test", "file": "main/stress.c", "line": 100}]
    }
  ]
}
```

### Simple Success Output

```json
{
  "status": "success",
  "test_expected": "Test passed",
  "test_actual": "Test passed"
}
```

## Use Cases

### AI Agent Analysis

AI coding agents can parse the JSON to:
1. Identify root cause from `exception.name` and `exception.faulting_address`
2. Locate the bug using `stack_trace[].file` and `stack_trace[].line`
3. See the code via `source_context` with >>> marking the crash line
4. Understand state by examining `registers` for variable values
5. Check resources via `threads[].stack_used` for stack overflow

### Example Agent Workflow

```python
import json
import subprocess

# Run tests
result = subprocess.run(['idf.py', 'run-project'], capture_output=True, text=True)
data = json.loads(result.stdout)

if data['status'] == 'crashed':
    exc = data['exception']
    print(f"Crash: {exc['name']} at {exc['faulting_address']}")

    # Find crash location
    for frame in data['stack_trace']:
        if frame['file'] and frame['line']:
            print(f"Bug in {frame['function']}() at {frame['file']}:{frame['line']}")
            break

    # Show source
    print(data['source_context'])
```

## QEMU Mode

When using `--qemu`, tests run in QEMU emulator instead of physical hardware. This is useful for CI/CD pipelines or when hardware is unavailable.

### How QEMU Coredump Decoding Works

1. ESP-IDF outputs base64-encoded coredumps to serial when a crash occurs
2. The tool extracts the base64 coredump from the DUT log
3. The base64 data is decoded to binary format
4. `esp-coredump` tool processes the binary with the ELF file
5. The decoded coredump is saved as `coredump.log`
6. The same parsing logic applies, producing identical JSON output

### QEMU vs Hardware

| Feature | Physical Hardware | QEMU |
|---------|------------------|------|
| Coredump source | pytest-embedded decoding | Base64 extraction + esp-coredump |
| Serial device needed | Yes | No |
| Structured output | Identical | Identical |
| Performance | Depends on hardware | Faster (emulated) |

## Documentation

For detailed AI agent debugging instructions:
```bash
idf.py agent-info
```

Or see [docs/agent_instructions.md](docs/agent_instructions.md).

## Requirements

- ESP-IDF v5.5 or later
- pytest-embedded-idf (`./install.sh --enable-ci` or `pip install pytest-embedded-idf`)
- Physical ESP32 hardware or QEMU
  - For QEMU: `pytest-embedded-qemu` plugin (`pip install pytest-embedded-qemu`)
- esp-coredump tool (included with ESP-IDF) for QEMU coredump decoding


## Contributing

Contributions welcome! This component focuses on making ESP-IDF crash analysis accessible to AI coding agents.

## License

MIT License - see [LICENSE](LICENSE) file.

## Links

- [ESP-IDF Documentation](https://docs.espressif.com/projects/esp-idf/)
- [pytest-embedded](https://github.com/espressif/pytest-embedded)
- [Component Registry](https://components.espressif.com/)