# IMX708 snapshot
Grabs one IMX708 frame, hardware-JPEG encodes it, and sends it down the USB
serial link. Pins preset for the Waveshare ESP32-P4-WIFI6.
**No microSD card is needed.** The card path still exists behind
`IMAGE_OUT_SD` (default off) and writes a 24-bit BMP to `/sdcard/imx708.bmp`;
with it off the example never touches the SD hardware. See
[Getting frames off the board](#getting-frames-off-the-board).
Frames are 1920x1080 by default. Four smaller modes are available —
1280x720, 1024x768, 800x600 and 640x480 — chosen with
`CAMERA_IMX708_MIPI_IF_FORMAT_INDEX_DEFAULT` (indices 0 to 4, largest to
smallest). This example reads its geometry from `VIDIOC_G_FMT`, so it needs no
change to follow: all five have been captured on hardware and CRC-checked.
`CAPTURE_MODE_INDEX` in `main/imx708_snapshot_main.c` selects the mode through
the driver API instead — `imx708_format_by_index()` and `VIDIOC_S_SENSOR_FMT`,
with `-1` (the default) keeping the build-time mode and `0..4` picking one. It
is a `#define`, so it still needs a rebuild and flash; what it shows is the call
an application would make. The call has to run before the first
`VIDIOC_STREAMON` — see that define's comment for why a later switch strands
auto-exposure and autofocus.
Two switches next to it drive that same call at run time, by cycling the video
stack per mode so each one gets a pipeline built for its own geometry:
- **`MODE_CONSOLE`** (default on) puts a `MODESEL> ` prompt on the console.
Press `0`–`4` to capture that mode, `q` to finish — no Enter, no rebuild, no
reboot. This is the demonstration that resolution is an application's choice
at run time; the `#define`s above exist only because nothing else fed the
index in. The same prompt also sets the flips (`h`, `v`, `n` to clear) and
the exposure (`-` and `+` for a stop either way, `a` back to auto), the
frame rate (`f`), the sensor test pattern (`t`) and the MIPI link frequency
(`l`), each one keystroke, each applied to the next capture.
- **`MODE_CYCLE`** walks the whole mode table off one boot, largest to
smallest. Five frames of one scene at one lens position is the only honest
way to compare modes — separate runs differ by exposure and framing as much
as by geometry.
```bash
python tools/capture.py --seconds 300 --interactive --out manual # type the modes
python tools/capture.py --flash --seconds 150 --keys "4,2,0,q" --out console_test
```
`--interactive` forwards your keystrokes and echoes the board back, stepping
over the binary payloads so images still extract while you watch the log;
`--keys` does the same from a script, one keystroke per prompt.
### Exposure bias
`-` and `+` move the exposure one stop from whatever AE settles on; `a` hands
it back to AE. The bias is relative on purpose — "two stops under what the
meter said" means the same thing in two different rooms, and an absolute line
count does not.
It is for the frame that gets measured rather than looked at. A clipped
highlight has no colour left in it, so a question about how the pipeline renders
highlights can only be answered on a frame where nothing clipped, and AE will
not hand you one. The bias goes into the file name (`imx708_1920x1080_ev-2.jpg`),
so a ladder does not overwrite itself:
```bash
# auto, then -2, -4 and -6 stops, one scene, one lens position
python tools/capture.py --flash --seconds 250 --keys "0,d,d,0,d,d,0,d,d,0,q" --out evsweep
```
`d` and `b` are aliases for `-` and `+`, because argparse reads a `--keys`
value beginning with a dash as the next option.
The log reports what the sensor actually took, read back rather than echoed:
```
exposure bias -4 stop: AE had 2494 lines @ 16.000x, now 2220 lines @ 1.123x
```
Gain falls to its floor before integration time shortens — the rule is the
lowest gain whose exposure still fits inside the mode's frame — so a
deliberately dark frame is also the least noisy one available at that light.
A bias the hardware cannot reach is clamped and says so, naming the range that
ran out.
### Frame rate
`f` cycles the sensor's frame length: the mode's own (2672 lines, 28 fps), then
4990 (15 fps), 7485 (10 fps), 1336 (56 fps, the sensor's floor) and 149694
(a 2 s frame, on the sensor's long-exposure shift), round again. It goes through `imx708_set_frame_length()`, applied *after*
`VIDIOC_STREAMON` because the control is live and a running stream is the case
worth proving. The frame length goes into the file name (`_fl4990`), and the
log reports the rate actually delivered, timed across DQBUFs, beside the
exposure ceiling V4L2 now advertises:
```
frame length 4990 lines (range 1336..65535), V4L2 exposure 4..4942 step 2
measured 15.000 fps (45 frames in 3000 ms)
```
```bash
# 28, 15, 10 and 56 fps at 640x480
python tools/capture.py --flash --seconds 240 --keys "4,f,4,f,4,f,4,q" --out fps
```
Measured 2026-09-25: every rate lands on `74847 / lines` to three decimals, and
every mode is clean at 56 fps **except 640x480, which tears from 48 fps up**
(clean at 40) — frames arrive on time but assembled from shifted pieces, with
nothing logged. 800x600 and larger are clean at 56, so it is neither the
sensor nor readout blanking; it is somewhere downstream in the P4 path, and not
yet explained.
A longer frame here does not brighten the picture: AE's exposure ceiling is
read once, when `esp_video_init()` builds the pipeline, so only a manually set
exposure can use the extra room.
### Test patterns and link frequency
`t` cycles the sensor's test pattern: off, colour bars, solid colour,
fade-to-grey bars, PN9. PN9 is sent raw rather than as JPEG — it is noise, and
it overruns the hardware JPEG encoder — and a raw PN9 frame is byte-identical
from one boot to the next, so its CRC is a check on the whole link.
`l` cycles the MIPI link frequency, 450, 447 and 453 MHz, applied before
`VIDIOC_STREAMON` because the driver refuses it on a running stream. Frame
timing is identical at all three, so PN9 at each should give the same CRC:
```bash
# PN9 at 800x600, at 450, 447 and 453 MHz
python tools/capture.py --flash --seconds 150 --keys "t,t,t,t,3,l,3,l,3,q" --out linkfreq_pn9
```
Both go into the file name when not at their defaults, as `_bars`, `_solid`,
`_fade` or `_pn9` and `_lf447` or `_lf453` — that run lands
`imx708_800x600_pn9`, `imx708_800x600_pn9_lf447` and `imx708_800x600_pn9_lf453`.
The full set of suffixes, in order, is
`imx708_<w>x<h>[_h][_v][_ev±N][_fl<lines>][_<pattern>][_lf<MHz>]`.
**Copy-pasteable commands for each mode** are in
[`docs/cli-cookbook.md`](../../../../docs/cli-cookbook.md#resolution-modes),
with the measured frame sizes. If you installed this from the component
registry rather than the repo, the component README's *Choosing a resolution*
section covers `menuconfig` and the scripted equivalent.
Every mode is a centred digital crop of the same 2×2-binned 2304×1296 readout,
so they all run at 28 fps with the same exposure range, and a smaller one costs
field of view rather than buying frame rate (frame rate is its own control -
see above). Nothing is scaled — the sensor's
scaling block is read-only, so 640x480 is a narrow window on the middle of the
scene, not the scene shrunk.
Why 1920 and not the sensor's native 2304: there is a datapath width ceiling
between 1920 and 2048 px on the P4, above which the scene runs out and the edge
columns duplicate each other exactly 2048 px apart. Do not "restore" the full
width.
For the same thing served over WiFi instead of the cable, see
[`imx708_wifi_snapshot`](https://github.com/mushBrainDave/esp32-p4-imx-camera/tree/main/examples/imx708_wifi_snapshot),
in the project repository.
## Before you run
Aim the camera at something with detail and light. That is all.
## Build, flash and receive — one command
```bash
python ../../../../tools/capture.py --flash
```
That flashes, resets the board, captures the stream and writes the image out.
**Do not use `idf.py monitor` here**: the port has a single owner, so a monitor
left running makes `idf.py flash` fail with `Access is denied`, and a monitor
mangles binary as it prints it, so it cannot recover the frame anyway.
After boot the example starts streaming, prints
`aim the camera — settling for 6 s...`, then grabs a frame and sends it. The
settle window is what gives auto-exposure, white balance and autofocus time to
converge. The run ends at `==== done ... ====` and the JPEG lands in the output
directory.
## What you're validating
- **The image is a real, changing photo of the scene** → the whole pipeline is
genuinely live end-to-end. This is the visual proof.
- With `IMAGE_OUT_SD` on, the board SD pins used are CLK 43, CMD 44, D0 39,
D1 40, D2 41, D3 42 (4-bit), with SD power via **on-chip LDO channel 4** —
set `host.pwr_ctrl_handle` or the card never powers up.
## ISP tuning (IPA)
The sensor's ISP tuning lives in
`components/esp_cam_sensor_imx/sensors/imx708/cfg/imx708_default.json`, keyed
`"IMX708"` to match `IMX708_SENSOR_NAME`. It turns on six IPA units: `agc`
(auto-exposure), `awb` (white balance), `acc` (saturation + colour-correction
matrix), `aen` (gamma, sharpen, contrast), `adn` (denoise), `ian` (metering
weights) and `af` (autofocus).
Registration happens in this example's **project** `CMakeLists.txt`, between
`include(project.cmake)` and `project()`. It cannot go in the component's
CMakeLists: esp_ipa reads `ESP_IPA_JSON_CONFIG_FILE_PATH` while its own
CMakeLists is processed, which is already done by the time a component's runs.
Get this wrong and there is no error — just
`failed to get configuration to initialize ISP controller` in the log, and no
AE or AWB.
### Things you may need to change
- **Gain is a menu control, not a number.** esp_video's AE reads gain via
`VIDIOC_QUERYMENU` and sets an *index* into `imx708_total_gain_val_map`. If a
future sensor here declares `ESP_CAM_SENSOR_GAIN` as
`ESP_CAM_SENSOR_PARAM_TYPE_NUMBER`, the query fails and the AE silently drives
exposure only — which looks exactly like a too-dark, too-grainy picture.
- **`agc.anti_flicker.ac_freq` is set to 60**, for 60 Hz mains. Change it to 50
if yours is 50 Hz, or you may see banding under artificial light. Note that
moving it 50 -> 60 here made **no visible difference**: at 60 Hz the light
ripples at 120 Hz (8.33 ms), `anti_flicker: full` snaps exposure to integer
multiples of that, and in a dim room AE is already pinned near the 35 ms
ceiling, so the constraint is nearly non-binding. Banding shows up at *short*
exposures, so retest in a bright scene before concluding the setting is
inert.
- **The CCM is a seed, not a calibration.** Real values come from shooting a
colour chart under known illuminants and solving for the matrix. The rows sum
to 1.0 so neutrals stay neutral, and it is deliberately gentler than
Espressif's OV5647 matrix because a **NoIR** module has no IR-cut filter —
infrared contaminates all three channels and aggressive saturation amplifies
that error.
- **The `af` scan band needs calibrating.** `af.min_pos` / `af.max_pos` (380 and
940) come from libcamera's Camera Module 3 tuning data, not from your module.
See *Autofocus* below.
### The build warning that was not harmless
This section used to say the `excess elements in array initializer` warnings
from the generated `esp_video_ipa_config.c` were benign generator noise. **That
was wrong, and believing it cost weeks.** They were the visible symptom of an
ABI mismatch that silently broke autofocus.
The prebuilt `esp_ipa` binary picked for us (`lib/esp32p4/v5.4-`) was compiled
with `ISP_AWB_WINDOW_X_NUM` / `_Y_NUM` = 5. Those macros only exist in ESP-IDF
>= v5.4.4, so on v5.4.0 `esp_ipa_stats_t::awb_subwin` collapses to `[0][0]` —
**400 bytes shorter than the library expects**. Everything after that member
then sits at the wrong offset: measured with a compiled `offsetof` probe,
`af_stats` landed at 196 in a 224-byte struct while the library read it from
596, past the end of the struct entirely. AE and AWB survived because their
members sit *before* `awb_subwin`; AF read unrelated heap memory, so every scan
point returned the same `definition` and the lens always parked at `min_pos`.
The fix is in this example's project `CMakeLists.txt`, set globally before
`project()` so esp_video (writes the struct), the generated config (declares it)
and the library (reads it) all agree:
```cmake
idf_build_set_property(COMPILE_DEFINITIONS "ISP_AWB_WINDOW_X_NUM=5" APPEND)
idf_build_set_property(COMPILE_DEFINITIONS "ISP_AWB_WINDOW_Y_NUM=5" APPEND)
```
The 400 bytes come back as inert padding, and `ESP_VIDEO_ISP_DEVICE_AWB_SUBWIN`
stays off because it keys on the IDF version rather than these macros, so
nothing calls sub-window APIs v5.4.0 lacks. **The warnings disappearing is now
the confirmation that the layouts agree** — if they come back, the struct
layouts have diverged again. Moving to IDF >= v5.4.4 defines the macros itself;
remove the block then, or they are defined twice.
General lesson: treat any struct-shaped warning against a prebuilt library as an
ABI report, not noise.
## Autofocus
The lens is moved by a **DW9807 voice-coil motor at I2C 0x0c** — a separate chip
from the IMX708, on the same bus. Driver:
`components/esp_cam_sensor_imx/motors/dw9807/`.
Focus is closed-loop contrast detection, and it takes four things that each fail
differently if missing:
| Piece | Where | If it's missing |
| --- | --- | --- |
| `CONFIG_CAM_MOTOR_DW9807` | `sdkconfig.defaults` | No driver; nothing can move the lens |
| `cam_motor` entry in `esp_video_init_config_t` | `imx708_snapshot_main.c` | Motor auto-detect array never walked |
| `CONFIG_ESP_IPA_AF_ALGORITHM` | `sdkconfig.defaults` | Nothing decides *where* to focus |
| `CONFIG_ESP_VIDEO_ISP_PIPELINE_CONTROL_CAMERA_MOTOR` | `sdkconfig.defaults` | AF result computed, then discarded |
The ISP scores "definition" (edge energy) inside the three `af.windows`, and the
algorithm hill-climbs the lens position that maximises it — a coarse pass of
`l1_scan_points_num` stops, then a fine pass of `l2_scan_points_num` around the
winner. That takes time, which is why `AIM_SECONDS` is 6.
The log tells you whether it worked:
```
dw9807: detected DW9807 VCM at 0x0c, lens parked at code 450 (range 0..1023)
imx708_snapshot: lens parked at code 450
imx708_snapshot: focus: 450 -> 683, centre sharpness 2841
```
A `focus: 450 -> 450` with a warning means the lens never moved.
### Calibrating the scan band
The DAC is 10-bit, but only the middle of that travel maps to real focus
distances — roughly code 420 at infinity to 920 at closest macro on a Camera
Module 3. Outside that the lens is against a mechanical stop. Those figures come
from libcamera's IMX708 tuning file, and VCM assemblies vary unit to unit, so
verify them once on your own module:
1. Set `CONFIG_ESP_VIDEO_ISP_PIPELINE_CONTROL_CAMERA_MOTOR=n` in `sdkconfig` —
otherwise the AF loop fights the sweep and keeps dragging the lens back. The
build `#error`s if you forget.
2. Set `FOCUS_SWEEP 1` in `main/imx708_snapshot_main.c`.
3. Aim at something with fine detail (text, brickwork) a couple of metres away,
keep the camera still, and flash.
It steps the DAC across the full range, prints a sharpness score per position
and writes `focus_<code>.bmp` for each. Read the **table**, not the pictures:
sharpness rises from the infinity end, peaks where your subject is, and falls
towards macro. The codes where the curve goes flat at each end are the
mechanical stops — those bound the useful range, and belong in `af.min_pos` /
`af.max_pos`. A completely flat column means the lens never moved at all.
The sharpness metric is the mean absolute horizontal gradient of the green
channel over the frame's centre band. It is computed independently of the ISP's
own AF statistic on purpose: if the two disagree about where best focus is, the
problem is the `af.windows` or `edge_thresh`, not the actuator. It cannot tell
defocus from motion blur, so hold the camera still.
### Other things you may want to change
- **`CONFIG_CAM_MOTOR_DW9807_INIT_POS`** (450) — where the lens parks before AF
converges, so it governs the first few frames. 450 sits just inside the
infinity end. Raise it towards 700–900 if the camera mostly looks at things
within arm's reach.
- **`CONFIG_CAM_MOTOR_DW9807_PERIOD_US`** (1000) — settling time allowed per DAC
code. Raise it if autofocus lands somewhere different each run on the same
static scene; that is the signature of definition being sampled while the lens
is still moving.
## Getting frames off the board
Captured frames travel down the same USB cable as the log, so a capture no
longer means powering off and carrying the microSD to a PC. The console runs at
**2 Mbaud** for this (`CONFIG_ESP_CONSOLE_UART_BAUDRATE`); a whole run is about
8 seconds.
Two formats, chosen deliberately:
| | format | size | why |
| --- | --- | --- | --- |
| normal capture | JPEG q90 | ~250-300 KB, ~1.5 s | small, and anything can open it |
| `FOCUS_SWEEP` | raw RGB565 | 4.1 MB, ~21 s | lossless, because the sweep is *measurement* |
**Do not measure sharpness on a JPEG.** Measured here with `JPEG_VALIDATE`: the
same frame scores 1411 as JPEG q90 against 1712 raw — 17.6% low. The focus sweep
peak is only ~1.25x its floor, so that shift is comparable to the entire signal,
and it is content-dependent (a sharper frame has more high-frequency energy, is
quantised harder, and is dragged further down). JPEG is for looking at.
`IMAGE_OUT_SD` (default off) restores the old BMP-to-card path. With it off the
example never touches the SD card, so no card need be inserted.
### Receiving
From this example's directory:
```bash
python ../../../../tools/capture.py --flash
```
or from the repo root, where it defaults to this example:
```bash
python tools/capture.py --flash
```
`tools/` sits at the repo root while `idf.py flash` has to run inside a project,
so the script resolves the two independently: it flashes the current directory
when that is a project, otherwise this example, or whatever `--project` names.
It flashes and then captures in **one process**, which matters: the port has a
single owner. A serial monitor left running in another terminal makes
`idf.py flash` fail with `Access is denied`, and a monitor cannot recover the
image anyway - it mangles binary as it prints it. Close any monitor before
running this. `--flash` is optional; without it the script just listens.
JPEG frames land as `.jpg`. Raw frames are converted to `.bmp` so they open
directly. Use `--seconds` to cover a longer run (a `FOCUS_SWEEP` needs roughly
25 s per position) and `--baud` if the console rate has been changed.
Frames are framed in the stream as:
```
IMGSTART name=<n> fmt=<jpeg|rgb565> w=<w> h=<h> len=<N> crc32=<hex>
<exactly N raw bytes>
IMGEND
```
A length and a CRC, so the receiver knows it got the frame rather than inferring
it from delimiters that could occur inside the data. Verify the CRC — it is what
caught the bug below.
### Two things that will bite you here
**The task watchdog writes into your binary.** A 4 MB frame takes ~21 s, and if
the sending loop never yields, the idle task starves and the TWDT prints a
warning *and a backtrace* straight into the middle of the payload — four
917-byte injections in a 21 s transfer, silently corrupting it. It bypasses
`esp_log_level_set` because it writes directly rather than through the tag
system. The send loop yields every 32 chunks to prevent this.
**The console rewrites `
` as `
`.** Binary payloads must set
`uart_vfs_dev_port_set_tx_line_endings(..., ESP_LINE_ENDINGS_LF)` first, or every
0x0A byte in the image gains a 0x0D.
Also note the baud is only settable under `ESP_CONSOLE_UART_CUSTOM`. Under the
default choice the symbol has no prompt and kconfgen silently keeps 115200 no
matter what `sdkconfig.defaults` says.
## Bring-up switches
`main/imx708_snapshot_main.c` has a block of switches at the top, kept for
diagnosing a new mode or board. Defaults in brackets.
| Switch | Default | What it does |
| --- | --- | --- |
| `IMAGE_OUT_SERIAL` | 1 | Send frames down the console UART |
| `IMAGE_OUT_SD` | 0 | Also write a BMP to the microSD card |
| `AF_DEBUG_LOG` | 1 | Turn on `esp_ipa_af`'s own per-scan-point logging |
| `FOCUS_SWEEP` | 0 | Step the VCM across its range, scoring each position |
| `JPEG_VALIDATE` | 0 | Send the same frame raw *and* as JPEG, to compare |
| `TEST_PATTERN` | 0 | Sensor emits colour bars instead of the scene |
| `POISON_BUFFERS` | 0 | Prefill capture buffers and count survivors |
| `RAW_PATTERN_TEST` | 0 | Send a synthetic counter instead of the image |
The four diagnostics worth understanding before you need them:
- **`TEST_PATTERN`** — the bars are generated after the pixel array, so they
cross the whole MIPI → CSI → ISP path. **Clean bars = the transport works**
and any bad photo is optics/exposure/ISP tuning. **Noisy bars = the fault is
upstream of the ISP** (link rate, lane count, data type, sensor mode). Note
the sensor clips the bars to its own test-pattern window (regs
`0x0620`-`0x0627`, left at power-on defaults), so they come out **cropped at
the left and right edges even when the capture path is perfect** — judge by
the bars that *are* drawn being clean and correctly placed.
- **`POISON_BUFFERS`** — each capture buffer is filled with `0xA5` before
streaming and the log reports how much survived. `poison check: ~100%` means
nothing was DMA'd in and the image is stale PSRAM, not a photo. The fill is
flushed out of L2 first; without that its dirty cache lines overwrite what the
DMA later writes and eat the bottom of the frame in cache-line-sized holes.
- **`RAW_PATTERN_TEST`** — replaces the image with a known counter sequence
before sending. Every byte of image data looks plausible, so an inserted or
dropped byte is invisible in a photo; against a known sequence a
resync-and-diff prints the exact offset, length and contents of each
insertion. This is what identified the watchdog injections described above,
after several rounds of fruitless inference from image structure.
- **`AF_DEBUG_LOG`** — `esp_ipa` ships as a closed `.a` but its `ESP_LOGD`
strings are intact, so `esp_log_level_set("esp_ipa_af", ESP_LOG_DEBUG)` from
the app is enough to make it print one line per scan point. Do **not** raise
`CONFIG_LOG_MAXIMUM_LEVEL` for this — it does nothing for a prebuilt library
and turns `ESP_LOGD` on firmware-wide.
## If the image looks wrong
- **Colours swapped / weird** — the ISP's output byte order may differ from the
assumed RGB565; adjust `rgb565_to_bgr()`.
- **All one colour / black** — exposure or ISP config; check the log for the
captured `bytes=` value and the ISP warning. Turn on `POISON_BUFFERS` to tell
"nothing was captured" apart from "captured, but badly exposed".
- **Measure it, do not eyeball it.** Row-to-row correlation near 0.99 means real
scene content; near 0.0 means there is no image in the buffer at all. A byte
histogram with heavy 0x55/0xAA bias is the signature of never-written DRAM.
Luma *percentiles* beat the mean — a p95 of 61 revealed an underexposure that
a mean of 40 understated.
- **Never compare sharpness across scenes.** The same metric ranged 374-4102
across pre-autofocus captures purely from scene and exposure. Only compare
frames of the same scene, and prefer a within-run sweep to two separate runs.
To create a project from this example, run:
idf.py create-project-from-example "mushbraindave/esp_cam_sensor_imx=0.4.0:imx708_snapshot"