uploaded 3 hours ago
Image process algorithms pipeline system for ISP.
Readme
# Espressif Image Process Algorithm for ISP
Espressif image process algorithm component provides a suit of image process algorithms.[](https://components.espressif.com/components/espressif/esp_ipa)
## 1. Supported Algorithms
| Algorithm | Description |
|:-:|:-|
| Auto Color Correction | Calculate lens shadow correction parameters, color correction matrix, and saturation value |
| Auto Denoising | Calculate Bayer denoising parameters and demosaic parameters |
| Dead Pixel Correction | Calculate ISP dynamic dead-pixel-correction parameters based on sensor gain |
| Auto Enhancement | Calculate GAMMA table, sharpen parameters, and contrast value |
| Auto Gain Control | Calculate exposure and gain |
| Auto White Balance | Calculate red and blue channels' gain |
| Image Analyze | Calculate image color temperature and luma |
| Auto Sensor AE Target Control | This is added for the sensors supported AEG/AGC by themselves |
| IR Control | Switch between day and night from environment luma. Night also requires AWB statistics with non-zero R and G sums |
| White Light Control | Switch the white fill light from environment luma, separating the light contribution from the ambient luma |
## 2. Pipeline Global Variable
This module is to share variables between algorithm modules without local cache in one image process pipeline, for example:
```
algorithm_1.c
int color = 15;
esp_ipa_set_int32(ipa_1, "color", 15);
algorithm_2.c
if (esp_ipa_has_int32_var(ipa_2, "color")) {
int color = esp_ipa_get_int32(ipa_2, "color")
}
algorithm_3.c
if (esp_ipa_has_int32_var(ipa_3, "color")) {
int color = esp_ipa_get_int32(ipa_3, "color")
}
......
```
- Note: "ipa_1", "ipa_2" and "ipa_3" should be in one IPA pipeline
- Prefer typed checkers (`esp_ipa_has_int32_var` / `esp_ipa_has_float_var` / `esp_ipa_has_ptr_var`); `esp_ipa_has_var` still returns true if the name exists in any typed map
## 3. JSON Configuration
Developers can refer to the configuration files in [esp_cam_sensor](https://github.com/espressif/esp-video-components/tree/master/esp_cam_sensor) about the JSON parameters usage:
- [SC2336](https://github.com/espressif/esp-video-components/blob/master/esp_cam_sensor/sensors/sc2336/cfg/sc2336_default.json)
- [OV5647](https://github.com/espressif/esp-video-components/blob/master/esp_cam_sensor/sensors/ov5647/cfg/ov5647_default.json)
- [OV2710](https://github.com/espressif/esp-video-components/blob/master/esp_cam_sensor/sensors/ov2710/cfg/ov2710_default.json)
### 3.1 Global Parameters
---
```json
{
"version": 1,
"writable": false,
"description": "0",
"SC2336": {}
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| version | Integer | >1 | JSON configuration version, this variable adds only when JSON configuration parameters change |
| writable | Boolean | true/false | When `true`, generated IPA config data in this JSON file is non-`const` (writable at runtime); default / omitted is `false` (`const`) |
| description | String | `[A-Za-z0-9_]+` | Per-file configuration identifier used in generated C symbols (`sensor_description_xxxx`) and stored in `esp_ipa_config_t.description`. Default / omitted is `"0"` when that sensor appears in only one JSON file. If the same sensor appears in multiple JSON files, each of those files must set `description` and the values must be unique for that sensor (different sensors may reuse the same `description`). Enumeration order follows the JSON input file list |
| SC2336 | Object | / | Target sensor name, such as "SC2336", "OV5647" and so on |
---
### 3.2 Algorithm Parameters
#### 3.2.1 Auto White Balance
---
```json
"SC2336":
{
"awb":
{
"model": 0,
"min_counted": 2000,
"min_red_gain_step": 0.034,
"min_blue_gain_step": 0.034,
"range":
{
"green":
{
"max": 200,
"min": 180
},
"rg":
{
"max": 1.2,
"min": 0.8
},
"bg":
{
"max": 1.2,
"min": 0.8
}
},
"green_luma_env": "awb_green_luma",
"green_luma_init": 200,
"green_luma_step_ratio": 0.3
}
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| awb | Object | / | Auto white balance configuration parameters |
| model | Integer or String | `0` / `1` / `2` / `3` or aliases | **`0`**: gray world (default). **`1`**: color-temperature index. **`2`**: zone classifier. **`3`**: fixed CT (`fixed_ct.presets[]`). String aliases: `gray_world`, `model_0`, `gw` → 0; `ct_index`, `model_1` → 1; `zone`, `model_2`, `hybrid`, `ct2` → 2; `fixed_ct`, `model_3` → 3 |
| min_counted | Integer | >0 | Minimum white points: Only when the white points number is larger than or equal to this does the auto white balance algorithm run |
| min_red_gain_step | Float | >0 | Minimum red channel gain step: Only when the red channel gain step is larger than or equal to this is the gain set into the ISP |
| min_blue_gain_step | Float | >0 | Minimum blue channel gain step: Only when the blue channel gain step is larger than or equal to this is the gain set into the ISP |
| range | Object | / | AWB statistics parameters |
| green | Object | / | AWB green channel statistics parameters |
| max(green) | Integer | <div style="white-space: nowrap;">(0,255)</div> | AWB green channel maximum lumina |
| min(green) | Integer | <div style="white-space: nowrap;">(0,255)</div> | AWB green channel minimum lumina |
| rg | Object | / | AWB red channel and green channel ratio statistics parameters |
| max(rg) | Integer | <div style="white-space: nowrap;">(0,255)</div> | AWB red channel and green channel maximum ratio |
| min(rg) | Integer | <div style="white-space: nowrap;">(0,255)</div> | AWB red channel and green channel minimum ratio |
| bg | Object | / | AWB blue channel and green channel ratio ratio statistics parameters |
| max(bg) | Integer | <div style="white-space: nowrap;">(0,255)</div> | AWB blue channel and green channel maximum ratio |
| min(bg) | Integer | <div style="white-space: nowrap;">(0,255)</div> | AWB blue channel and green channel minimum ratio |
| green_luma_env | String | / | AWB green channel lumina variable name |
| green_luma_init | Integer | <div style="white-space: nowrap;">(0,255)</div> | AWB green channel initialization lumina |
| green_luma_step_ratio | Float | >0 | AWB green channel minimum lumina step ratio |
**AWB model 2 (zone)** — set `model` to `2` or `"zone"`. Requires **`zones`** (at least one neutral box). **`ref_points`** lists calibrated chroma `(rg, bg)` per color temperature; the algorithm rotates the active ref point, accumulates global AWB statistics, classifies the estimate into a **neutral** zone (`uhct`, `hct`, `mct`, `lct`, `ulct`). **`green`** / **`skin`** boxes are used for classification only and **do not** vote as illuminant. Optional **`export_ct`**: when `true`, writes estimated CT (Kelvin) to the pipeline variable **`ct`** (consumed by ACC). Use **`new_w`** / **`prev_w`** for temporal smoothing of chroma and exported CT.
```json
"awb":
{
"model": "zone",
"min_counted": 1000,
"min_red_gain_step": 0.5,
"min_blue_gain_step": 0.5,
"new_w": 1.0,
"prev_w": 0.0,
"export_ct": false,
"zone_switch_count": 1,
"type_counter_max": 500,
"outlier_rg": 0.0,
"outlier_bg": 0.0,
"zone_hysteresis_ratio": 0.0,
"zones":
[
{
"type": "mct",
"rg": { "min": 0.45, "max": 0.70 },
"bg": { "min": 0.45, "max": 0.70 },
"enabled": true
}
],
"ref_points":
[
{ "ct": 5200, "rg": 0.55, "bg": 0.55, "radius": 0.35 }
]
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| zones | Array | non-empty for model 2 | Each entry: **`type`** (`uhct`, `hct`, `mct`, `lct`, `ulct`, `green`, `skin`), **`rg`**.min/max, **`bg`**.min/max (chromaticity bounds on **R/G** and **B/G**), **`enabled`** (optional, default true). First matching enabled neutral zone wins. |
| ref_points | Array | ≥1 for model 2 | Each entry: **`ct`** (Kelvin), **`rg`**, **`bg`**, **`radius`** (attraction in chroma space; `0` disables snapping for that point). |
| startup_zone | String | `uhct`, `hct`, `mct`, `lct`, `ulct` | Optional model-2 startup zone. Uses the first matching reference point for initial gains and CT, then continues automatic AWB. Omitted: preserve normal white-point scanning without a startup preset. |
| startup_hold_frames | Integer | ≥ 0 | Hold startup WBG for this many calls carrying AWB statistics; requires `startup_zone`. AWB calculation, statistics scanning and CT export continue. Default `0`: no hold. |
| new_w | Float | ≥ 0 | IIR weight for the new frame when smoothing chroma / CT. |
| prev_w | Float | ≥ 0 | IIR weight for the previous frame; with **`new_w`**, both `0` disables smoothing on that path. |
| export_ct | Boolean | true or false | If **true**, publish estimated CT to **`ct`**. |
| zone_switch_count | Integer | ≥ 1 | Frames per ref-point slot before advancing the scan index (use `1` for fastest wrap). |
| type_counter_max | Integer | >0 | Ceiling for per-zone hit counters before decay (generator default 20000 if omitted). |
| outlier_rg | Float | ≥ 0 | Reject sub-window when \|R/G − prev\| exceeds this; `0` disables. |
| outlier_bg | Float | ≥ 0 | Same for B/G. |
| zone_hysteresis_ratio | Float | ≥ 0 | Zone-switch stickiness; `0` disables. |
**AWB model 2 startup white balance** — optionally set `startup_zone` to `uhct`, `hct`, `mct`, `lct`, or `ulct`. Initializes gains and CT from the first valid reference point in the selected enabled zone, then continues automatic AWB. Omitting this parameter preserves normal startup white-point scanning. Returns `ESP_ERR_INVALID_ARG` if no matching reference is available. Startup CT is published to **`ct`**; subsequent CT updates require `export_ct: true`.
**AWB model 2 fixed color temperature** — `ESP_IPA_AWB_S_CT` (`esp_ipa_pipeline_ioctl`, `uint32_t` Kelvin) holds gains from `ref_points[]` and skips statistics. An exact `ct` uses that point. A value between two points interpolates R/G and B/G in mired (`1000000 / Kelvin`). A value outside the table is clamped to the nearest point. The next `esp_ipa_pipeline_process()` publishes WBG on any real gain change (same `1e-4` step as model 3) and keeps **`ct`**. IAN does not overwrite that held CT. `esp_ipa_pipeline_get_state` / `set_state` restore this hold when `awb.fixed.valid` is set. `ESP_IPA_AWB_S_RESET` resumes statistics and the JSON `min_*_gain_step` filter. `ct` 0 returns `ESP_ERR_INVALID_ARG`.
**AWB model 3 (fixed CT)** — set `model` to `3` or `"fixed_ct"`. Does not use AWB statistics. Applies gains from `fixed_ct.presets[]`. A requested Kelvin value that matches a preset uses that preset. A value between two presets interpolates R/G and B/G in mired (`1000000 / Kelvin`). A value outside the table is clamped to the nearest preset. **`fixed_ct.default_ct`** is selected at init the same way. Runtime switch: `ESP_IPA_AWB_S_CT` then `esp_ipa_pipeline_process()`. `ESP_IPA_AWB_S_RESET` restores `default_ct` and stays on the fixed-CT table. Returns `ESP_ERR_INVALID_ARG` if `ct` is 0. KV **`ct`** is the applied Kelvin, after clamping, for ACC CCM/LSC/SAT.
Model 0 and model 1 reject `ESP_IPA_AWB_S_CT` / `ESP_IPA_AWB_S_RESET` with `ESP_ERR_INVALID_ARG`.
```json
"awb":
{
"model": "fixed_ct",
"min_red_gain_step": 0.05,
"min_blue_gain_step": 0.05,
"red_gain_scale": 1.0,
"blue_gain_scale": 1.0,
"fixed_ct":
{
"default_ct": 5000,
"presets":
[
{ "ct": 3000, "rg": 0.52, "bg": 0.58 },
{ "ct": 5000, "rg": 0.5602, "bg": 0.5390 },
{ "ct": 6500, "rg": 0.58, "bg": 0.52 }
]
}
}
```
Legacy single preset (still supported):
```json
"fixed_ct": { "ct": 5000, "rg": 0.5602, "bg": 0.5390 }
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| fixed_ct | Object | required for model 3 | **`esp_ipa_awb_fixed_cfg_t`**: **`default_ct`** (Kelvin; a preset, or a value interpolated between presets), **`presets[]`** each with **`ct`/`rg`/`bg`** (>0). Runtime CT via **`ESP_IPA_AWB_S_CT`**: exact preset, mired interpolation between presets, or clamp to the nearest preset. Legacy `{ ct, rg, bg }` = one preset. Deprecated JSON key **`fixed`** is still accepted by the generator. |
---
#### 3.2.2 Auto Color Correction
---
```json
"SC2336":
{
"acc": {}
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| acc | Object | / | Auto color correction configuration parameters |
---
```json
"acc":
{
"saturation":
[
{
"color_temp": 0,
"value": 128
}
...
],
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| saturation | Array | / | The saturation value and color temperature mapping table adopted the principle of nearest neighbor indexing |
| color_temp | Integer | >0 | Color temperature value |
| value | Integer | <div style="white-space: nowrap;">[0,255]</div> | Saturation value |
Saturation runs after init. `esp_ipa_pipeline_ioctl()` `ESP_IPA_ACC_S_SAT_STATUS` takes an `int`: `0` stops later saturation updates, and any other value runs them again. `ESP_IPA_ACC_G_SAT_STATUS` reads that flag. Stopping does not clear the last saturation value. The next run after resume publishes the table value for the current color temperature.
---
```json
"acc":
{
"ccm": {}
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| ccm | Object | / | Color correction matrix configuration parameters |
---
```json
"ccm":
{
"model": 1,
"min_ct_step": 500
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| model | Integer | {0,1} | CCM data process model |
| min_ct_step | Integer | >0 | Color temperature minimum step value for generating new CCM value |
---
```json
"ccm":
{
"low_luma":
{
"luma_env": "ae.luma.avg",
"threshold": 28,
"matrix":
[
1.00, 0.00, 0.00,
0.00, 1.00, 0.00,
0.00, 0.00, 1.00
]
},
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| low_luma | Object | / | Color correction matrix configuration parameters in low brightness scenes |
| luma_env | String | / | Lumina variable name |
| threshold | Float | >0 | Minimum brightness: if the "luma_env" value is less than this is the low brightness CCM value set into the ISP |
| matrix | Array | <div style="white-space: nowrap;">ESP32-P4: (-4,4)</div> | Low brightness CCM value |
---
```json
"ccm":
{
"table":
[
{
"color_temp": 2320,
"matrix":
[
2.0000, -0.1680, -0.8320,
-0.3716, 2.0000, -0.6284,
-0.7150, -0.2850, 2.0000
]
},
...
]
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| table | Array | / | The CCM and color temperature mapping table adopted the principle of nearest neighbor indexing |
| color_temp | Integer | >0 | Color temperature value |
| matrix | Array | <div style="white-space: nowrap;">ESP32-P4: (-4,4)</div> | CCM value |
---
```json
"ccm":
{
"gain_lut":
{
"enable": true,
"table":
[
{ "gain": 1.0, "strength": 1.0 },
{ "gain": 8.0, "strength": 0.85 },
{ "gain": 16.0, "strength": 0.7 },
{ "gain": 32.0, "strength": 0.5 }
]
}
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| gain_lut | Object | / | Optional gain-based CCM strength LUT |
| enable | Bool | true or false | If true, blend the CT-selected CCM with identity using `strength` from the LUT |
| table | Array | / | Gain and strength mapping table; `strength` is linearly interpolated between entries |
| gain | Float | >0 | Camera sensor gain (same unit as `sensor->cur_gain`) |
| strength | Float | [0,1] | Blend weight `s` in `(1-s)*I + s*ccm`; 0 means identity, 1 means full CCM |
---
```json
"acc":
{
"lsc": {}
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| lsc | Object | / | Lens shadow correction(LSC) configuration parameters |
---
```json
"lsc":
{
"model": 0,
"img_w": 1920,
"img_h": 1080,
"lsc_tbl_size": 558,
"table":
[
{
"gain": 1.0,
"table":
[
{
"ct": 3350,
"calibrations_r_tbl": [ 2.13310074, ... ],
"calibrations_gr_tbl": [ 1.92501747, ... ],
"calibrations_gb_tbl": [ 1.90096950, ... ],
"calibrations_b_tbl": [ 1.67257392, ... ]
},
...
]
},
{
"gain": 8.0,
"scale": 0.8
},
...
]
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| model | Integer | 0 | LSC data process model |
| img_w | Integer | >0 | Image resolution width |
| img_h | Integer | >0 | Image resolution height |
| lsc_tbl_size | Integer | >0 | LSC data array size, different image resolution has different values, such as "lsc_tbl_size" of 1920x1080 resolution is 558 |
| table | Array | / | LSC mapping table. Preferred: two-level nearest-neighbor index (sensor gain → color temperature). Legacy flat CT-only table `[{ct, ...}]` is still accepted and mapped to `lsc_gain_table`. Selected entry is used directly (no runtime LSC generation) |
| gain | Float | >0 | Camera sensor gain of the outer index node (optional in legacy flat CT table) |
| scale | Float | >=0 | Optional. When a gain node has no nested `table`, blend the base LSC arrays toward 1.0 at config generation time: `out = 1 + (base - 1) * scale` (stored as `round(out * 256)`). `scale=1` copies the base table; `scale=0` yields a flat 1.0 table. Optional `base_gain` selects the source node; otherwise the single base table node (or nearest lower-gain base) is used |
| base_gain | Float | >0 | Optional. Source gain node for `scale` when multiple full `table` nodes exist |
| ct | Integer | >0 | Color temperature value of the inner index node (or top-level node in legacy flat table) |
| calibrations_r_tbl | Array[Float] | <div style="white-space: nowrap;">ESP32-P4: (-4,4)</div> | LSC data array for the red channel in RAW field |
| calibrations_gr_tbl | Array[Float] | <div style="white-space: nowrap;">ESP32-P4: (-4,4)</div> | LSC data array for the red-green(RG) channel in RAW field |
| calibrations_gb_tbl | Array[Float] | <div style="white-space: nowrap;">ESP32-P4: (-4,4)</div> | LSC data array for the blue-green(BG) channel in RAW field |
| calibrations_b_tbl | Array[Float] | <div style="white-space: nowrap;">ESP32-P4: (-4,4)</div> | LSC data array for the blue channel in RAW field |
---
```json
"acc":
{
"blc": {}
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| blc | Object | / | Black level correction(BLC) configuration parameters |
---
```json
"lsc":
{
"model": 0,
"stretch": true,
"blc_table":
[
{
"gain": 1,
"blc_param": {
"blc_top_left": 16,
"blc_top_right": 16,
"blc_bottom_left": 16,
"blc_bottom_right": 16
}
},
...
]
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| model | Integer | 0 | BLC data process model |
| stretch | Bool | true or false | Stretch configurations for each channel of the raw Bayer image |
| blc_table | Array | / | The BLC parameters and gain mapping table adopted the principle of nearest neighbor indexing |
| gain | Float | >0 | Sensor gain |
| blc_param | Object | / | The BLC parameters |
| blc_top_left | Integer | <div style="white-space: nowrap;">ESP32-P4: (0,255)</div> | Black level threshold for top left channel of the raw Bayer image |
| blc_top_right | Integer | <div style="white-space: nowrap;">ESP32-P4: (0,255)</div> | Black level threshold for top right channel of the raw Bayer image |
| blc_bottom_left | Integer | <div style="white-space: nowrap;">ESP32-P4: (0,255)</div> | Black level threshold for bottom left channel of the raw Bayer image |
| blc_bottom_right | Integer | <div style="white-space: nowrap;">ESP32-P4: (0,255)</div> | Black level threshold for bottom right channel of the raw Bayer image |
---
#### 3.2.3 Auto Denoising
---
```json
"SC2336":
{
"adn": {}
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| adn | Array | / | Auto denoising configuration parameters |
---
```json
"adn":
{
"bf":
[
{
"gain": 1,
"param":
{
"level": 3,
"matrix":
[
1, 3, 1,
3, 5, 3,
1, 3, 1
],
"sigma": 1
}
},
...
]
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| bf | Array | / | The Bayer filter parameters and sensor gain mapping table adopted the principle of nearest neighbor indexing |
| gain | Float | >0 | Sensor gain |
| param | Object | / | Bayer filter parameters |
| level | Integer | <div style="white-space: nowrap;">ESP32-P4: [2,20]</div> | Bayer filter sigma |
| matrix | Integer | ESP32-P4: [0,15] | Bayer filter matrix |
| sigma | Float | >0 | Sigma parameter to generate Gaussian matrix |
* Note: The Bayer filter is a bilateral filter, level is a parameter of range kernel, and matrix is a parameter of spatial kernel
* Note: If there is a matrix, the JSON tools use it firstly, otherwise, use sigma
---
```json
"adn":
{
"demosaic":
[
{
"gain": 1,
"gradient_ratio": 1.0
},
...
]
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| demosaic | Array | / | The Demosaic parameters and sensor gain mapping table adopted the principle of nearest neighbor indexing |
| gain | Float | >0 | Sensor gain |
| gradient_ratio | Float | <div style="white-space: nowrap;">ESP32-P4: (0,4)</div> | Demosaic grad ratio parameters |
---
#### 3.2.4 Dead Pixel Correction
---
```json
"SC2336":
{
"dpc": {}
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| dpc | Object | / | Dead pixel correction configuration parameters |
---
```json
"dpc":
{
"model": 0,
"debounce_gain": 0.1,
"table":
[
{
"gain": 1.0,
"param":
{
"method": 1,
"high_threshold": 8,
"low_threshold": 8
}
},
{
"gain": 4.0,
"param":
{
"method": 2,
"first_stage_upper_ratio": 1.0,
"first_stage_lower_ratio": 0.5,
"bright_deviation_factor": 0.5,
"dark_deviation_factor": 0.5
}
},
...
]
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| model | Integer | {0,1} | DPC gain mapping model: `0` selects the nearest gain node (with optional debounce); `1` linearly interpolates parameters between neighboring gain nodes |
| debounce_gain | Float | ≥0 | Model 0 switch debounce threshold in sensor-gain units; `0` disables debounce. A candidate node is accepted only when it is closer than the current node by more than this value |
| table | Array | ≥1 entries | DPC dynamic parameters and sensor gain mapping table |
| gain | Float | >0 | Sensor gain |
| param | Object | / | ISP dynamic DPC parameters for this gain node |
| method | Integer | {1,2} | Dynamic correction method selected for this node; method 1 and method 2 can be mixed across nodes |
| high_threshold | Integer | [0,255] | Method 1: a pixel above `max8 + high_threshold` is a bright dead-pixel candidate |
| low_threshold | Integer | [0,255] | Method 1: a pixel below `min8 - low_threshold` is a dark dead-pixel candidate |
| first_stage_upper_ratio | Float | [0,1] | Method 2: upper bound of the first-stage normal-pixel range; must be greater than `first_stage_lower_ratio` |
| first_stage_lower_ratio | Float | [0,1] | Method 2: lower bound of the first-stage normal-pixel range |
| bright_deviation_factor | Float | [0,1] | Method 2: second-stage bright-pixel sensitivity; a smaller value corrects bright pixels more aggressively |
| dark_deviation_factor | Float | [0,1] | Method 2: second-stage dark-pixel sensitivity; a smaller value corrects dark pixels more aggressively |
* Note: Model 0 uses nearest-neighbor indexing like ADN. Model 1 interpolates continuously between two nodes that share the same `method`; if adjacent nodes use different methods, model 1 falls back to the nearer node
* Note: Method 1 / method 2 parameters are stored in `esp_ipa_dpc_t`. Method 2 ratios and deviation factors remain floating-point values in range [0, 1]
* Note: Aliases `"nearest"` / `"linear"` (for `model`) and `"method_1"` / `"method_2"` (for `method`) are also accepted
---
#### 3.2.5 Auto Enhancement
---
```json
"SC2336":
{
"aen": {}
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| aen | Object | / | Auto enhancement configuration parameters |
---
```json
"aen":
{
"gamma":
{
"model": 0,
"use_gamma_param": true,
"luma_env": "ae.luma.avg",
"luma_min_step": 16.0,
"backlight": {}
},
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| model | Integer | {0,1} | GAMMA data process model |
| gamma | Object | / | GAMMA configuration parameters object |
| use_gamma_param | bool | true/false | true: use the given gamma parameter to generate gamma Y table; false: use the given gamma Y table |
| luma_env | String | / | Lumina variable name |
| luma_min_step | Float | >0 | Minimum brightness step: if the "luma_env" step value is larger than this is the new GAMMA table value set into the ISP |
| backlight | Object | / | Optional backlight enhancement configuration; when active, selects a dedicated GAMMA table by backlight degree |
---
```json
"gamma":
{
"table":
[
{
{
"luma": 31.1,
"gamma_param": 0.51,
"y":
[
0, 64, 91, 112, 130, 146, 160, 173, 185, 197, 207, 218, 228, 237, 246, 255
]
},
},
...
]
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| table | Array | / | The GAMMA and luma mapping table adopted the principle of linear interpolation |
| luma | Float | >0 | Luma index value |
| gamma_param | Float | >0 | Parameter to generate GAMMA Y array |
| y | <div style="white-space: nowrap;">Array[Integer]</div> | <div style="white-space: nowrap;">ESP32-P4: [0,255]</div> | GAMMA Y array |
---
```json
"gamma":
{
"backlight":
{
"low_hist_ratio_threshold": 0.2,
"high_hist_ratio_threshold": 0.15,
"env_luma_threshold": 5.0,
"low_index":
{
"start": 0,
"end": 4
},
"high_index":
{
"start": 14,
"end": 15
},
"luma_env": "env.luma.avg",
"detect_count_threshold": 3,
"detect_count_margin": 3,
"hist_ratio_filter": 0.3,
"model": 0,
"luma_min_step": 0.05,
"table":
[
{
"luma": 0.35,
"gamma_param": 0.45
},
{
"luma": 0.55,
"gamma_param": 0.40
}
]
}
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| backlight | Object | / | Backlight enhancement: detect from histogram low/high ratios and env luma, then select a dedicated GAMMA table by degree |
| low_hist_ratio_threshold | Float | [0, 1] | Low-bin histogram ratio threshold; enable when `low > low_hist_ratio_threshold` |
| high_hist_ratio_threshold | Float | [0, 1] | High-bin histogram ratio threshold; enable when `high > high_hist_ratio_threshold` |
| env_luma_threshold | Float | ≥ 0 | Environment luma threshold; enable when env luma > `env_luma_threshold` |
| low_index | Object | / | Histogram index range for the low ratio; defaults to `{start: 0, end: 4}` |
| high_index | Object | / | Histogram index range for the high ratio; defaults to `{start: 14, end: 15}` |
| start / end | Integer | [0, 15] | Inclusive histogram segment indices |
| luma_env | String | / | Environment luma variable name used for the threshold check |
| detect_count_threshold | Integer | ≥ 0 | Anti-jitter: backlight turns on when consecutive detect count > `detect_count_threshold` |
| detect_count_margin | Integer | ≥ 0 | Optional. Extra count above `detect_count_threshold`; detect count saturates at `detect_count_threshold + detect_count_margin`. Omitted: equals `detect_count_threshold` |
| hist_ratio_filter | Float | [0, 1] | IIR filter coefficient for low/high histogram ratios |
| model | Integer | {0,1} | GAMMA process model used in backlight mode; defaults to `0` |
| luma_min_step | Float | >0 | Minimum backlight degree step to update GAMMA; defaults to `0.05` |
| table | Array | / | Backlight GAMMA look-up table keyed by degree (`luma` or alias `degree`) |
* Note: `low = sum(hist[low_index]) / sum(hist)`, `high = sum(hist[high_index]) / sum(hist)`, degree = `low + high`
* Note: when backlight is active, the normal `gamma.table` path is replaced by `backlight.table`
---
```json
"aen"
{
"sharpen":
[
{
"gain": 1,
"param":
{
"h_thresh": 16,
"l_thresh": 3,
"h_coeff": 2.65,
"m_coeff": 2.95,
"matrix":
[
1, 2, 1,
2, 2, 2,
1, 2, 1
]
}
},
...
]
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| sharpen | Array | / | The sharpen parameters and sensor gain mapping table adopted the principle of nearest neighbor indexing |
| gain | Float | >0 | Sensor gain |
| param | Object | / | Sharpen parameters |
| h_thresh | Integer | <div style="white-space: nowrap;">ESP32-P4: [0,255]</div> | Sharpen high threshold, filtered pixel value higher than this threshold multiplies by `h_coeff` |
| l_thresh | Integer | ESP32-P4: [0,255] | Sharpen low threshold, filtered pixel value higher than this threshold but lower than `h_thresh` will be multiplied by `m_coeff` and filtered pixel value lower than this threshold will be set to 0 |
| h_coeff | Float | ESP32-P4: (0,8) | High frequency pixel sharpness coefficient |
| m_coeff | Float | ESP32-P4: (0,8) | Medium frequency pixel sharpness coefficient |
| matrix | <div style="white-space: nowrap;">Array [Integer]</div> | ESP32-P4: [0,31] | 3x3 low pass filter matrix data |
---
```json
"aen":
{
"contrast":
[
{
"gain": 1,
"value": 130
},
...
]
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| contrast | Array | / | The contrast value and sensor gain mapping table adopted the principle of nearest neighbor indexing |
| gain | Float | >0 | Sensor gain |
| value | Integer | <div style="white-space: nowrap;">ESP32-P4: [0,255]</div> | Contrast value |
---
#### 3.2.6 Auto Gain Control
---
```json
"SC2336":
{
"agc": {}
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| agc | Object | / | Auto gain/exposure control configuration parameters |
---
```json
"agc":
{
"exposure":
{
"frame_delay": 2,
"adjust_delay": 0
}
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| exposure | Object | / | Exposure control configuration parameters object |
| frame_delay | Integer | >0 | The delayed frames number of setting exposure parameter taking effect, this value is from sensor hardware information |
| adjust_delay | Integer | >0 | The delayed frames number of calculating and setting new exposure parameter, higher value makes brightness change slower |
| fixed_exposure_time | Integer | >0 | Fixed exposure time in microseconds. Required when `gain_only` is `true`; ignored otherwise |
---
```json
"agc":
{
"gain_only": true,
"exposure":
{
"frame_delay": 2,
"adjust_delay": 0,
"fixed_exposure_time": 20000
}
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| gain_only | Bool | true or false | Optional. `true`: AGC only adjusts gain after init and keeps exposure at `exposure.fixed_exposure_time`; `false` or omitted: adjust both exposure and gain as usual |
* Note: when `gain_only` is `true`, `exposure.fixed_exposure_time` is mandatory and must be within the sensor exposure range
* Note: init may still set exposure once to `exposure.fixed_exposure_time` if the current sensor exposure differs
---
```json
"agc":
{
"gain":
{
"min_step": 0.0001,
"frame_delay": 2,
"max": 8.0
}
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| gain | Object | / | Gain control configuration parameters object |
| frame_delay | Integer | >0 | The delayed frames number of setting gain parameter taking effect, this value is from sensor hardware information |
| min_step | Float | >0 | Minimum gain step: Only when the gain step is larger than or equal to this value is the gain set into the sensor |
| max | Float | ≥ 0 | Optional. When **> 0**, caps the analogue gain requested by AGC at this value (applied after the sensor’s own `min_gain` / `max_gain`). **0** or omitted: no extra limit beyond the sensor capability |
---
```json
"agc":
{
"anti_flicker":
{
"mode": "full",
"ac_freq": 50
},
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| anti_flicker | Object | / | Anti-flicker control configuration parameters object |
| mode | String enumeration | "full"<hr>"part"<hr>"none" | `"full"`: force to use anti-flicker exposure configuration <hr>`"part"`: if the gain can adjust the image brightness, force to use anti-flicker exposure configuration, otherwise change the exposure ignore anti-flicker <hr> `"none"`: adjust exposure and gain fully ignore anti-flicker |
| ac_freq | Integer | >0 | Alternating current frequency for lighting, in general, the value is 50Hz or 60Hz |
---
```json
"agc":
{
"f_n0": 0.32,
"f_m0": 0.32
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| f_n0 | Float | (0,1] | The parameter to control gain or exposure increasing speed, with a higher value, makes the increasing speed faster |
| f_m0 | Float | (0,1] | The parameter to control gain or exposure decreasing speed, with a higher value, makes the decreasing speed faster |
---
```json
"agc":
{
"luma_adjust":
{
"target_low": 105,
"target_high": 121,
"target": 113,
"low_threshold": 10,
"low_regions": 5,
"high_threshold": 240,
"high_regions": 3,
"weight":
[
1, 1, 2, 1, 1,
1, 2, 3, 2, 1,
1, 3, 5, 3, 1,
1, 2, 3, 2, 1,
1, 1, 2, 1, 1
]
}
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| luma_adjust | Object | / | Image brightness control configuration parameter |
| target_low | Integer | (0,target) | Hold band low: if brightness is in [target_low, target_high] (shifted with PWL/meter target), AGC does not change exposure or gain |
| target_high | Integer | (target,255) | Hold band high: same as target_low; brightness outside the band is driven toward target |
| target | Integer | [3,252] | Image brightness target used when luma is outside the hold band |
| low_threshold | Integer | [3,252] | Image brightness sampled data low threshold: if the image brightness sampled data is less than this value is the low threshold counter increased by 1 |
| high_threshold | Integer | [3,252] | Image brightness sampled data high threshold: if the image brightness sampled data is higher than this value is the high threshold counter increased by 1 |
| low_regions | Integer | <div style="white-space: nowrap;">ESP32-P4:(0,25)</div> | Image low threshold counter low regions number: if low threshold counter is less than this value will all sampled data which is less than "low_threshold" be dropped |
| high_regions | Integer | ESP32-P4:(0,25) | Image high threshold counter high regions number: if high threshold counter is less than this value will all sampled data which is higher than "high_threshold" be dropped |
| weight | <div style="white-space: nowrap;">Array[Integer]</div> | [0,255] | Image brightness sampling weight |
**Runtime AE weight** — `ESP_IPA_AGC_S_METER_LUMA_WEIGHT` replaces the 5×5 AE weight table. Index is `x * ISP_AE_BLOCK_Y_NUM + y`. An all-zero table is rejected. Blocks whose weight differs from the JSON table stay in the average when the low-threshold or high-threshold region drop would otherwise set their weight to 0. `ESP_IPA_AGC_S_RESET_METER_WEIGHT` restores the JSON table. The application maps points or rectangles onto AE blocks.
**Luma target offset** — `ESP_IPA_AGC_S_METER_TARGET_OFFSET` and `ESP_IPA_AGC_G_METER_TARGET_OFFSET` add the same integer shift to JSON `luma_target`, `luma_low`, and `luma_high`. NaN and infinity are rejected. The stored shift is limited to [`ESP_IPA_AGC_METER_TARGET_OFFSET_MIN`, `ESP_IPA_AGC_METER_TARGET_OFFSET_MAX`] so an 8-bit JSON target can reach either end of 1..255. The resulting target, low, and high are then clamped to 1..255. AGC continues to split exposure and gain. Resetting the weight table does not change the offset.
---
```json
"agc":
{
"mode": "high_light_priority",
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| mode | String enumeration |"high_light_priority"<hr>"low_light_priority"<hr>"light_threshold_priority" | Measurement weight calculation mode |
---
```json
"agc":
{
"high_light_priority":
{
"use_env_luma": true,
"env_luma_low_threshold": 150.5,
"env_luma_high_threshold": 250.5,
"low_threshold": 141,
"high_threshold": 204,
"weight_offset": 5,
"luma_offset": -3
}
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| high_light_priority | Object | / | High light brightness priority configuration |
| use_env_luma | Bool | true or false | If true, uses `env_luma_low_threshold` and `env_luma_high_threshold`; if false, uses `low_threshold` and `high_threshold` |
| env_luma_high_threshold | Float | > 0 | Upper threshold for environmental brightness: if an environmental brightness value is less or equal to this threshold, its data weight and the total weight are increased by weight_offset |
| env_luma_low_threshold | Float | > 0 | Lower threshold for environmental brightness: if an environmental brightness value is larger or equal to this threshold, its data weight and the total weight are increased by weight_offset |
| low_threshold | Integer | <div style="white-space: nowrap;">[3,252]</div> | Image brightness sampled data low threshold: if one image brightness sampled data is higher than this value will the data's weight and total weight be increased by weight_offset |
| high_threshold | Integer | [3,252] | Image brightness sampled data high threshold: if one image brightness sampled data is less than this value will the data's weight and total weight be increased by weight_offset |
| weight_offset | Integer | [0,255] | Image brightness sampled data weight increasing value |
| luma_offset | Integer | [-127,127] | Image brightness target increasing value if one sampled data value is between "low_threshold" and "high_threshold" |
* Note: if "mode" is "high_light_priority" this configuration applies
* Note: if "use_env_luma" is true, the environmental luminance is used instead of statistics data
---
```json
"agc":
{
"low_light_priority":
{
"use_env_luma": true,
"env_luma_high_threshold": 250.5,
"env_luma_low_threshold": 150.5,
"low_threshold": 65,
"high_threshold": 104,
"weight_offset": 5,
"luma_offset": 1
}
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| low_light_priority | Object | / | Low light brightness priority configuration |
| use_env_luma | Bool | true or false | If true, uses `env_luma_low_threshold` and `env_luma_high_threshold`; if false, uses `low_threshold` and `high_threshold` |
| env_luma_high_threshold | Float | > 0 | Upper threshold for environmental brightness: if an environmental brightness value is less or equal to this threshold, its data weight and the total weight are increased by weight_offset |
| env_luma_low_threshold | Float | > 0 | Lower threshold for environmental brightness: if an environmental brightness value is larger or equal to this threshold, its data weight and the total weight are increased by weight_offset |
| low_threshold | Integer | <div style="white-space: nowrap;">[3,252]</div> | Image brightness sampled data low threshold: if one image brightness sampled data is higher than this value will the data's weight and total weight be increased by weight_offset |
| high_threshold | Integer | [3,252] | Image brightness sampled data high threshold: if one image brightness sampled data is less than this value will the data's weight and total weight be increased by weight_offset |
| weight_offset | Integer | [0,255] | Image brightness sampled data weight increasing value |
| luma_offset | Integer | [-127,127] | Image brightness target increasing value if one sampled data value is between "low_threshold" and "high_threshold" |
* Note: if "mode" is "low_light_priority" this configuration applies
* Note: if "use_env_luma" is true, using the environmental luminance instead of statistics data
---
```json
"agc":
{
"light_threshold_priority_use_env_luma": true,
"light_threshold_priority":
[
{
"env_luma_threshold": 30.5,
"luma_threshold": 20,
"weight_offset": 1
},
...
]
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| light_threshold_priority_use_env_luma | Bool | true or false | If true, uses the `env_luma_threshold`; if false: uses the `luma_threshold` |
| light_threshold_priority | Object | / | Light brightness threshold priority configuration |
| env_luma_threshold | Float | > 0 | Environmental brightness threshold. If an environmental brightness value is less than this threshold, the data's weight and the total weight will be increased by weight_offset |
| luma_threshold | Integer | <div style="white-space: nowrap;">[0,255]</div> | Light brightness threshold, if one sampled data is larger than this value and less than the next the data's weight and the total weight will be increased by weight_offset |
| weight_offset | Integer | [0,255] | Image brightness sampled data weight increasing value |
* Note: if "mode" is "light_threshold_priority" this configuration applies
* Note: if "use_env_luma" is true, using the environmental luminance instead of statistics data
---
```json
"agc":
{
"luma_pwl":
{
"enable": true,
"table":
[
{ "env_luma": 500, "luma_shift": -15 },
{ "env_luma": 2000, "luma_shift": -5 },
{ "env_luma": 5000, "luma_shift": 0 },
{ "env_luma": 12000, "luma_shift": -10 }
]
}
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| luma_pwl | Object | / | Environment luma → target luma shift PWL (piecewise linear) configuration |
| enable | Bool | true or false | `true`: read `env.luma.avg` each frame and interpolate `luma_shift` from the table; `false`: feature disabled, `luma_target` is unchanged |
| table | Array | ≥2 entries | Sorted breakpoint list mapping `env_luma` to `luma_shift`; intermediate values are linearly interpolated, out-of-range values are clamped to the first or last entry |
| env_luma | Float | >0 | Environment luma breakpoint, corresponding to pipeline variable `env.luma.avg` |
| luma_shift | Integer | [-128, 127] | Target luma offset applied when the environment luma matches this breakpoint; positive values brighten the target, negative values darken it |
* Note: `luma_pwl` must be sorted in ascending order of `env_luma`
* Note: the effective `luma_target` = configured `target` + interpolated `luma_shift`
* Note: the `target_low`/`target_high` hold band is shifted by the same amount, so AGC stays still when luma is already near the PWL target
---
#### 3.2.7 Auto Sensor AE Target Control
---
```json
"OV2710":
{
"atc": {}
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| atc | Object | / | Auto sensor AE target control configuration parameters |
---
```json
"atc":
{
"model": 0,
"init_value": 48,
"delay_frames": 5,
"luma_env": "env.luma.avg",
"min_ae_value_step": 5,
"luma_lut":
[
{
"luma": 100,
"ae_value": 75
},
{
"luma": 200,
"ae_value": 100
},
{
"luma": 300,
"ae_value": 150
}
]
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| model | Integer | {0} | ATC data process model |
| init_value | Integer | >0 | Auto sensor AE target control initialization value |
| delay_frames | Integer | >0 | Frames number btween AE target level changing |
| luma_env | String | / | Lumina variable name |
| min_step | Integer | >0 | Minimum AE target level step: Only when the AE target level step is larger than or equal to this value is the AE target level set into the sensor |
| luma_lut | Array | / | The luma value and AE target level mapping table adopted the principle of linear interpolation |
| luma | Integer | >0 | Luma value threshold |
| ae_value | Integer | >0 | AE target level value |
---
#### 3.2.8 Auto Focus Control
---
```json
"OV5647":
{
"af": {}
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| af | Object | / | Auto focus control configuration parameters |
---
```json
"af":
{
"model": 0,
"windows":
[
{
"left": 680,
"top": 300,
"width": 390,
"height": 410
}
],
"edge_thresh": 32,
"definition_high_threshold_ratio": 1.3,
"definition_low_threshold_ratio": 0.7,
"luminance_high_threshold_ratio": 1.1,
"luminance_low_threshold_ratio": 0.9,
"max_change_time": 2000000,
"l1_scan_points_num": 10,
"l2_scan_points_num": 10,
"max_pos": 500
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| model | Integer | 0 | AF data process model |
| windows | Array | / | AF statistics window region, ESP32-P4 supports 3 windows |
| left | Integer | / | Window Left coordinate |
| top | Integer | / | Window Top coordinate |
| width | Integer | / | Window width |
| height | Integer | / | Window height |
| edge_thresh | Integer | / | AF statistics threshold, and value less will be dropped |
| definition_high_threshold_ratio | Float | >1.0 | When the (current definition statistics)/(previous focus definition) ratio is larger than this value, a new AF process will trigger |
| definition_low_threshold_ratio | Float | <1.0 | When the (current definition statistics)/(previous focus definition) ratio is less than this value, a new AF process will trigger |
| luminance_high_threshold_ratio | Float | >1.0 | When the (current luminance statistics)/(previous focus luminance) ratio is larger than this value, a new AF process will trigger |
| luminance_low_threshold_ratio | Float | <1.0 | When the (current luminance statistics)/(previous focus luminance) ratio is less than this value, a new AF process will trigger |
| max_change_time | Integer | >0 | The time how long the AF statistics definition or luminance keeps out of the threshold, the unit is microsecond |
| l1_scan_points_num | Integer | >0 | Level 1 scanning point numbers |
| l2_scan_points_num | Integer | >0 | Level 2 scanning point numbers |
| max_pos | Integer | >0 | Maximum focus position, less value can reduce scanning time cost, but also shorten the focus distance |
**Runtime AF windows** — `ESP_IPA_AF_S_WINDOWS` replaces the three runtime windows and restarts the focus scan. `ESP_IPA_AF_S_WINDOW_WEIGHT` replaces window weights (software only; no ISP settle). `ESP_IPA_AF_S_RESET` restores the JSON windows and `weight_table`, then restarts the scan. After IPA publishes new windows to the ISP, the next AF stats frame is ignored so the scan does not use stale geometry. `ESP_IPA_AF_G_WINDOWS` and `ESP_IPA_AF_G_WINDOW_WEIGHT` read the runtime windows and weights. The application maps points or rectangles onto AF windows.
---
#### 3.2.9 IR Control
---
```json
"SC2336":
{
"ir":
{
"model": 0,
"model0_param":
{
...
}
}
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| ir | Object | / | IR day/night switching configuration. Omitted `luma_env` uses `env.luma.avg`. JSON `null` or `"NULL"` stays C `NULL` and that frame has no luma |
---
```json
"ir":
{
"model": 0,
"model0_param":
{
"luma_env": "env.luma.avg",
"day_luma_threshold": 0.5,
"day_switch_night_time_us": 500000,
"day_keep_time_us": 2000000,
"night_ir_factor": 10.0,
"night_luma_threshold": 0.8,
"night_switch_day_time_us": 500000,
"night_keep_time_us": 3000000,
"night_awb":
{
"green": { "max": 255, "min": 0 },
"rg": { "max": 1.5, "min": 0.5 },
"bg": { "max": 1.5, "min": 0.5 }
},
"night_ccm":
{
"matrix": [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]
},
"night_red_gain": 1.0,
"night_blue_gain": 1.0
}
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| model | Integer | {0} | IR data process model |
| model0_param | Object | / | Model 0 parameters |
| luma_env | String | / | Pipeline variable for environment luma. Omit the key to use `env.luma.avg`. JSON `null` or the string `NULL` is C `NULL`, and that frame has no luma |
| day_luma_threshold | Float | / | Day environment luma threshold. Bright when luma is greater |
| day_switch_night_time_us | Integer | >=0 | Transition time from day to night, microseconds. Omitted key is `0` |
| day_keep_time_us | Integer | >=0 | Dark must hold this long before leaving day, microseconds |
| night_ir_factor | Float | / | Multiply environment luma at night before the threshold compare. Omitted key is `1.0` |
| night_luma_threshold | Float | / | Night environment luma threshold. Bright when `luma * night_ir_factor` is greater |
| night_switch_day_time_us | Integer | >=0 | Transition time from night to day, microseconds. Omitted key is `0` |
| night_keep_time_us | Integer | >=0 | Bright luma and non-zero AWB R and G must hold this long before leaving night, microseconds |
| night_awb | Object | / | AWB statistics window written when the switch to night starts. `green`/`rg`/`bg` each have `max` and `min` |
| night_ccm | Object | / | CCM written when the switch to night starts. `matrix` has 9 elements, row-major |
| night_red_gain | Float | / | Red gain written when the switch to night starts |
| night_blue_gain | Float | / | Blue gain written when the switch to night starts |
When the `luma_env` key is omitted, it becomes `env.luma.avg`. JSON `null` and the string `NULL` stay C `NULL`. IR does not compute luma from AE statistics.
Day compares luma with `day_luma_threshold`. Night multiplies luma by `night_ir_factor` before comparing with `night_luma_threshold`. A bright night frame starts leaving only when AWB statistics are present and both R and G are non-zero. No luma, missing AWB, R=0, or G=0 keeps night and does not advance the timer. Day turns IR on when it stays dark for `day_keep_time_us`. Night turns IR off when bright luma and AWB R and G stay non-zero for `night_keep_time_us`. `day_switch_night_time_us` is the day-to-night transition; night judging starts after it. `night_switch_day_time_us` is the night-to-day transition; day judging starts after it. Switching publishes `metadata.ir_enable` (`IPA_METADATA_FLAGS_IR`).
The switch to night stops AWB (`ESP_IPA_AWB_S_STATUS`), IAN color-temperature estimation (`ESP_IPA_IAN_S_CT_STATUS`), CCM (`ESP_IPA_ACC_S_CCM_STATUS`) and saturation, then writes `night_awb`, `night_ccm`, `night_red_gain`, `night_blue_gain` and saturation `0`. Entering day starts those modules again. Each module clears its applied index (`-1`, or the AWB statistics window) and republishes on its next process. IR does not store or restore the previous saturation.
**Note**: The IR module should be put the last node of the Json configuration file.
**Note**: `esp_ipa_pipeline_process()` requires the lower layer to keep uploading real-time ISP statistics every cycle. IPA may update statistics parameters from configuration or metadata (for example region or range), but the statistics stream must not stop. With the default `luma_env` (`env.luma.avg`), IAN only refreshes that variable when the current frame carries AE; IR still reads the last published value for `day_keep_time_us` and `night_keep_time_us`. Stopping AE upload or calling process without AE can therefore advance IR hold timing on stale luma. That cached-luma path is not a skipped decision. JSON `null` / `"NULL"` (`luma_env == NULL`) means this frame has no luma, so IR does not advance the day/night decision. `esp_ipa_pipeline_process()` still returns `ESP_OK`. The application does not drop the frame.
---
#### 3.2.10 White Light Control
---
```json
"SC2337P":
{
"ian": { "luma": { "env": { ... } } },
"wl":
{
"enter_luma": 4.0,
"exit_luma": 16.0,
"light": { "ct": 5000 }
},
"awb": { ... },
"agc": { ... },
"acc": { ... }
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| wl | Object | / | White fill light control configuration parameters |
---
```json
"wl":
{
"enable": true,
"enter_luma": 4.0,
"exit_luma": 16.0,
"verify_ratio": 1.5,
"k_tolerance": 0.1,
"enter_frames": 30,
"exit_frames": 30,
"settle_frames": 5,
"settle_max_frames": 60,
"settle_tolerance": 0.02,
"probe_period_s": 600,
"light":
{
"ct": 5000
},
"gate":
{
"enable": false,
"min_gain_of_max": 0.8,
"min_exposure_of_max": 0.9
},
"enable_log": false
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| enable | Boolean | true or false | Enable the algorithm, only takes effect at init. `false`: the module does nothing and the fill light stays off. Default `true` |
| enter_luma | Float | >0 | Required. Enter night when environment luma `E` is below this value for `enter_frames` |
| exit_luma | Float | >enter_luma | Required. Start a probe when the ambient luma `A = E - K` is above this value for `exit_frames` |
| verify_ratio | Float | ≥1 | Leave night when the luma with the light off is above `enter_luma * verify_ratio`. Default `1.5` |
| k_tolerance | Float | [0, 1) | Allowed measurement error of the light contribution `K` as a fraction of `K`, the mismatch threshold is `max(enter_luma, k_tolerance * K)`. Default `0.1` |
| enter_frames | Integer | >0 | Consecutive frames to enter night. Default `30` |
| exit_frames | Integer | >0 | Consecutive frames to start a probe. Default `30` |
| settle_frames | Integer | ≥0 | Frames with unchanged gain and exposure regarded as AE stable. Default `5` |
| settle_max_frames | Integer | >0 | Maximum frames to wait for AE stable, and then for `E` stable. Default `60` |
| settle_tolerance | Float | [0, 1) | `E` is stable when it stays within `max(enter_luma, settle_tolerance * E)` for `settle_frames`. Default `0.02` |
| probe_period_s | Integer | ≥0 | Probe in night when `K` was measured this long ago, recovers from an overestimated `K`; `0` disables. Default `600` |
| light | Object | / | Fill light parameters |
| ct | Integer | ≥0 | Fill light color temperature, Kelvin; `0`: unknown. Out of day, AWB is locked to it by `ESP_IPA_AWB_S_CT`, see AWB model 2 fixed color temperature. Default `0` |
| gate | Object | / | Enter night gate parameters |
| enable(gate) | Boolean | true or false | Also require gain and exposure near the sensor maximum to enter night. Default `false` |
| min_gain_of_max | Float | [0, 1] | Minimum current gain as a fraction of the sensor maximum gain. Default `0.8` |
| min_exposure_of_max | Float | [0, 1] | Minimum current exposure as a fraction of the sensor maximum exposure. Default `0.9` |
| enable_log | Boolean | true or false | Enable the algorithm debug log |
The environment luma is `E = A + K`, the ambient luma plus the light contribution. Entering night turns the light on in two steps, off → 50% → 100%, and measures `K` from the luma of each step after AE settles. In night, `A = E - K` decides when to probe: the light is dimmed to 50% to measure the ambient luma again, and the night is left only when the luma with the light off confirms it. The brightness is published in percent by `metadata.wl.brightness` (`IPA_METADATA_FLAGS_WL`); the camera driver converts it to the LED output, which must be linear to it. The state is published to pipeline variable **`wl.state`** (`esp_ipa_wl_state_t`). When WL leaves day, it locks AWB to `light.ct` by `ESP_IPA_AWB_S_CT`, and when it returns to day, it releases AWB by `ESP_IPA_AWB_S_RESET`; nothing is sent if `light.ct` is `0`. AWB model 0 and model 1 reject the command, and WL goes on without the lock. The application should not send `ESP_IPA_AWB_S_CT` / `ESP_IPA_AWB_S_RESET` while WL is enabled, because WL releases AWB when it returns to day.
**Note**: WL requires IAN `luma.env`, which publishes `env.luma.avg`, so it must be put after IAN in the Json configuration file. Putting it before AWB lets AWB follow the lock in the same frame, otherwise one frame later. WL and IR are exclusive.
---
#### 3.2.11 Extended Control
---
```json
"OV5647":
{
"ext": {}
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| ext | Object | / | Extended control configuration parameters |
---
```json
"ext":
{
"hue": 0,
"brightness": 0,
"stats_region":
{
"left": 100,
"top": 100,
"width": 400,
"height": 400
}
}
```
| Parameter | Type | Range | Description |
|:-:|:-:|:-:|:-|
| hue | Integer | [0,360] | ISP color HUE |
| brightness | Integer | [-128,127] | ISP color brightness |
| stats_region | Object | / | ISP statistics region |
| left | Integer | / | Window Left coordinate |
| top | Integer | / | Window Top coordinate |
| width | Integer | / | Window width |
| height | Integer | / | Window height |
Links
- Homepage
-
Repository
f9414be3aad71589cdbe4fb963e4a6ffeb080f75
- Issues
- License
- Changelog
Target
To add this component to your project, run:
download archive
idf.py add-dependency "espressif/esp_ipa^2.5.0"
Stats
-
Archive size ~ 1016.69 KB
-
Downloaded in total 233.8k times
-
Weekly Downloads (All Versions)
-
This version: 0 times
Badge
