espressif/esp-serial-flasher

readme

# esp-serial-flasher

`esp-serial-flasher` is a portable C library for flashing or loading apps to RAM of Espressif SoCs from other host microcontrollers.

## Using the library
`esp-serial-flasher` supports a variety of host/target/interface combinations:

Supported **host** microcontrollers:

- STM32
- Raspberry Pi SBC
- ESP32
- Any MCU running Zephyr OS

Supported **target** microcontrollers:

- ESP32
- ESP8266
- ESP32-S2
- ESP32-S3
- ESP32-C3
- ESP32-C2
- ESP32-H2
- ESP32-C6

Supported hardware interfaces:
- UART
- SPI (only for RAM download)
- USB CDC ACM (experimental)

For example usage check the `examples` directory.

## Configuration

These are the configuration toggles available to the user:

* `SERIAL_FLASHER_INTERFACE_UART`/`SERIAL_FLASHER_INTERFACE_SPI`/`SERIAL_FLASHER_INTERFACE_USB`

This defines the hardware interface to use.

Default: SERIAL_FLASHER_INTERFACE_UART

* `MD5_ENABLED`

If enabled, `esp-serial-flasher` is capable of verifying flash integrity after writing to flash.

Default: Enabled
> Warning: As ROM bootloader of the ESP8266 does not support MD5_CHECK, this option has to be disabled!

* `SERIAL_FLASHER_WRITE_BLOCK_RETRIES`

This configures the amount of retries for writing blocks either to target flash or RAM.

Default: 3

* `SERIAL_FLASHER_RESET_HOLD_TIME_MS`

This is the time for which the reset pin is asserted when doing a hard reset in milliseconds.

Default: 100

* `SERIAL_FLASHER_BOOT_HOLD_TIME_MS`

This is the time for which the boot pin is asserted when doing a hard reset in milliseconds.

Default: 50

Configuration can be passed to `cmake` via command line:

```
cmake -DMD5_ENABLED=1 .. && cmake --build .
```

### STM32 support

The STM32 port makes use of STM32 HAL libraries, and these do not come with CMake support. In order to compile the project, `stm32-cmake` (a `CMake` support package) has to be pulled as submodule.

```
git clone --recursive https://github.com/espressif/esp-serial-flasher.git
```

If you have cloned this repository without the `--recursive` flag, you can initialize the submodule using the following command:

```
git submodule update --init
```

In addition to configuration parameters mentioned above, following definitions has to be set:

- STM32_TOOLCHAIN_PATH: path to arm toolchain (i.e /home/user/gcc-arm-none-eabi-9-2019-q4-major)
- STM32_CUBE_<CHIP_FAMILY>_PATH: path to STM32 Cube libraries (i.e /home/user/STM32Cube/Repository/STM32Cube_FW_F4_V1.25.0)
- STM32_CHIP: name of STM32 for which project should be compiled (i.e STM32F407VG)
- CORE_USED: core used on multicore devices (i.e. M7 or M4 on some STM32H7 chips)
- PORT: STM32

This can be achieved by passing definitions to the command line, such as:

```
cmake -DSTM32_TOOLCHAIN_PATH="path_to_toolchain" -DSTM32_CUBE_<CHIP_FAMILY>_PATH="path_to_cube_libraries" -DSTM32_CHIP="STM32F407VG" -DPORT="STM32" .. && cmake --build .
```

Alternatively, those variables can be set in the top level `cmake` directory:

```
set(STM32_TOOLCHAIN_PATH path_to_toolchain)
set(STM32_CUBE_H7_PATH path_to_cube_libraries)
set(STM32_CHIP STM32H743VI)
set(CORE_USED M7)
set(PORT STM32)
```

### Zephyr support

The Zephyr port is ready to be integrated into Zephyr apps as a Zephyr module. In the manifest file (west.yml), add:

```
    - name: esp-flasher
      url: https://github.com/espressif/esp-serial-flasher
      revision: master
      path: modules/lib/esp_flasher
```

And add

```
CONFIG_ESP_SERIAL_FLASHER=y
CONFIG_CONSOLE_GETCHAR=y
CONFIG_SERIAL_FLASHER_MD5_ENABLED=y
```

to the project configuration `prj.conf`.

For the C/C++ source code, the example code provided in `examples/zephyr_example` can be used as a starting point.

## Supporting a new host target

The port layer for the given host microcontroller can be implemented if not available, in order to support a new target, following functions have to be implemented by user:

- `loader_port_read()`
- `loader_port_write()`
- `loader_port_enter_bootloader()`
- `loader_port_delay_ms()`
- `loader_port_start_timer()`
- `loader_port_remaining_time()`

For the SPI interface ports
- `loader_port_spi_set_cs()`
needs to be implemented as well.

The following functions are part of the [io.h](include/io.h) header for convenience, however, the user does not have to strictly follow function signatures, as there are not called directly from library.

- `loader_port_change_transmission_rate()`
- `loader_port_reset_target()`
- `loader_port_debug_print()`

Prototypes of all functions mentioned above can be found in [io.h](include/io.h).

After that, the target implementing these functions should be linked with the `flasher` target and the `PORT` CMake variable should be set to `USER_DEFINED`.

## Contributing

We welcome contributions to this project in the form of bug reports, feature requests and pull requests.

Issue reports and feature requests can be submitted using [Github Issues](https://github.com/espressif/esp-serial-flasher/issues). Please check if the issue has already been reported before opening a new one.

Contributions in the form of pull requests should follow ESP-IDF project's [contribution guidelines](https://docs.espressif.com/projects/esp-idf/en/latest/esp32/contribute/style-guide.html) and use the [conventional commit message style](https://www.conventionalcommits.org/en/v1.0.0/).

To automatically enforce these rules, use [pre-commit](https://pre-commit.com/) and install hooks with the following commands:
```
pre-commit install
pre-commit install -t commit-msg
```

## Licence

Code is distributed under Apache 2.0 license.

## Known limitations

Size of new binary image has to be known before flashing.

changelog

## v1.3.0 (2024-03-22)

### New Features

- Add a convenient public API way to read the WIFI MAC

### Bug Fixes

- Correctly compare image size with memory size including offset

## v1.2.0 (2024-02-28)

### New Features

- Move flash size detection functionality to the public API

### Bug Fixes

- Fix inferring flash size from the flash ID
- **docs**: Fix table in SPI load to RAM example

## v1.1.0 (2024-02-13)

### New Features

- USB CDC ACM interface support
- Add the ability for ESP ports to not initialize peripherals

### Bug Fixes

- **docs**: Remove notes about SPI interface being experimental
- Document size_id and improve comments in places

## v1.0.2 (2023-12-20)

### Bug Fixes

- Fix flash size ID sanity checks

## v1.0.1 (2023-12-19)

### Bug Fixes

- Fix md5 timeout values
- **ci**: Add more compiler warnings for the flasher in the examples

## v1.0.0 (2023-12-10)

- Initial release

                                        
                                        # Flash multiple partitions example

## Overview

Example demonstrates how to flash ESP32/ESP32-S2/ESP8266 from another (host) MCU using esp_serial_flash component API. In this case, ESP32 is also used as host MCU. Binaries to be flashed from host MCU to another Espressif SoC can be found in `binaries` folder and are converted into C-array during build process.

Following steps are performed in order to re-program target's memory:

1. Filesystem is initialized and mounted.
2. UART1 through which new binary will be transfered is initialized.
3. Host puts slave device into boot mode tries to connect by calling `esp_loader_connect()`.
4. Binary file is opened and its size is acquired, as it has to be known before flashing.
5. Then `esp_loader_flash_start()` is called to enter flashing mode and erase amount of memory to be flashed.
6. `esp_loader_flash_write()` function is called repeatedly until the whole binary image is transfered.

Note: In addition, to steps mentioned above, `esp_loader_change_transmission_rate`  is called after connection is established in order to increase flashing speed. This does not apply for ESP8266, as its bootloader does not support this command. However, ESP8266 is capable of detecting baud rate during connection phase, and can be changed before calling `esp_loader_connect`, if necessary.

## SPI pin initialization

In majority of cases `ESP_LOADER_CONNECT_DEFAULT` helper macro is used in order to initialize `loader_connect_args_t` data structure passed to `esp_loader_connect`. Helper macro sets `spi_pin_config` field of the data structure to zero, thus, default SPI pins are used to connect to FLASH memory. In special cases, such as custom design in which FLASH is connected to different pins, `spi_pin_config` field has to be set accordingly. For more detailed information refer to [serial protocol](https://docs.espressif.com/projects/esptool/en/latest/esp32s3/advanced-topics/serial-protocol.html).

## Hardware Required

* Two development boards with ESP32 SoC (e.g., ESP32-DevKitC, ESP-WROVER-KIT, etc.).
* One or two USB cables for power supply and programming.

## Hardware connection

Table below shows connection between two ESP32 devices.

| ESP32 (host) | ESP32 (slave) |
|:------------:|:-------------:|
|    IO26      |      IO0      |
|    IO25      |     RESET     |
|    IO4       |      RX0      |
|    IO5       |      TX0      |

Note: interconnection is the same for all three targets (slaves). 

## Build and flash

To run the example, type the following command:

```CMake
idf.py -p PORT flash monitor
```

(To exit the serial monitor, type ``Ctrl-]``.)

See the Getting Started Guide for full steps to configure and use ESP-IDF to build projects.

## Configuration

For details about available configuration option, please refer to top level [README.md](../../README.md). 
Compile definitions can be specified on command line when running `idf.py`, for example:

```
idf.py build -DMD5_ENABLED=1
```
Binaries to be flashed are placed in separate folder (binaries.c) for each possible target and converted to C-array. Without explicitly enabling MD5 check, flash integrity verification is disabled by default.

## Example output

Here is the example's console output:

```
...
I (342) example: Initializing SPIFFS
I (482) example: Image size: 144672
I (902) example: Connected to target
I (1732) example: Start programming
I (1832) example: packet: 0  written: 1024 B
I (1932) example: packet: 1  written: 1024 B
...
I (16052) example: packet: 140  written: 1024 B
I (16152) example: packet: 141  written: 288 B
I (16152) example: Finished programming
```

                                    

                                        
                                        # ESP32 Get target info example

## Overview

This example demonstrates how to get the target info including the flash chip size and the WIFI MAC address of an Espressif SoC from another (host) MCU using `esp-serial-flasher`. In this case, an ESP32 is used as the host MCU.

Following steps are performed:

1. UART1 through which new binary will be transfered is initialized.
2. Host puts target device into boot mode and tries to connect by calling `esp_loader_connect()`.
3. Host attempts to read the target flash size and WIFI MAC and prints them out

Note: In addition, to steps mentioned above, `esp_loader_change_transmission_rate()`  is called after connection is established in order to increase communication speed. This does not apply for ESP8266, as its bootloader does not support this command. However, ESP8266 is capable of detecting baud rate during connection phase, and can be changed before calling `esp_loader_connect()`, if necessary.

## Hardware Required

* Two development boards with ESP32 SoC (e.g., ESP32-DevKitC, ESP-WROVER-KIT, etc.).
* One or two USB cables for power supply and programming.

## Hardware connection

Table below shows connection between two ESP32 devices.

| ESP32 (host) | ESP32 (target) |
|:------------:|:--------------:|
|    IO26      |      IO0       |
|    IO25      |     RESET      |
|    IO4       |      RX0       |
|    IO5       |      TX0       |

## Build and flash

To run the example, type the following command:

```CMake
idf.py -p PORT flash monitor
```

(To exit the serial monitor, type ``Ctrl-]``.)

See the Getting Started Guide for full steps to configure and use ESP-IDF to build projects.

## Example output

Here is the example's console output:

```
...
Connected to target
Transmission rate changed changed
I (744) serial_flasher: Target flash size [B]: 8388608
I (744) serial_flasher: Target WIFI MAC:
I (744) serial_flasher: 84 f7 03 80 01 c4 
I (744) main_task: Returned from app_main()
```

                                    

                                        
                                        # Example of loading the program into RAM through SPI

## Overview

This example demonstrates how to upload an app to RAM of an Espressif MCU with SPI download support from another (host) MCU using the `esp_serial_flash` component API. In this case, another Espressif MCU is used as the host. Binaries to be uploaded to RAM from host MCU to the target MCU can be found in `binaries/RAM_APP` folder and are converted into C-array during build process.

Following steps are performed in order to re-program the target's memory:

1. SPI2 through which the binary will be transfered is initialized.
2. Host puts slave device into SPI download mode tries to connect by calling `esp_loader_connect()`.
3. Then `esp_loader_mem_start()` is called for each segment in RAM.
4. `esp_loader_flash_write()` function is called repeatedly for every segment until the whole binary image is transfered.
5. `esp_loader_mem_finish()` is called with the binary entrypoint, telling the chip to start the uploaded program.
6. UART2 is initialized for the connection to the target
7. Target output is continually read and printed

## Hardware Required

* Two development boards, one with any Espressif MCU (e.g., ESP32-DevKitC, ESP-WROVER-KIT, etc.) and one with an Espressif MCU with SPI download support. Here is a short list of supported MCUs:
1. ESP32-C3
2. ESP32-C2
3. ESP32-S3
4. ESP32-S2
5. ESP32-H2
* One or two USB cables for power supply and programming.

## Hardware connection

Table below shows connection between two Espressif MCUs.

| Host         | Slave         |
|:------------:|:-------------:|
|    IO_5      |    RESET      |
|    IO_12     |    CLK        |
|    IO_10     |    CS         |
|    IO_13     |    MISO       |
|    IO_11     |    MOSI       |
|    IO_14     |    QUADWP     |
|    IO_9      |    QUADHD     |
|    IO_13     |    STRAP_B0   |
|    IO_2      |    STRAP_B1   |
|    IO_3      |    STRAP_B2   |
|    IO_4      |    STRAP_B3   |
|    IO_6      |    UART0_RX   |
|    IO_7      |    UART0_TX   |

> Note 1: Strapping bit pins are documented in the TRM for each respective chip

> Note 2: For achieving highest speeds, check which pins go through the IO MUX bypassing the GPIO matrix and use those. Pins chosen here are IO MUX pins for ESP32S3 and ESP32S2 chips.

## Build and flash

To run the example, type the following command:

```CMake
idf.py -p PORT flash monitor
```

(To exit the serial monitor, type ``Ctrl-]``.)

See the Getting Started Guide for full steps to configure and use ESP-IDF to build projects.

## Example output

Here is the example's console output:

```
Connected to target
I (682) spi_ram_loader: Loading app to RAM ...
Start loading
Downloading 7840 bytes at 0x3fc96e00...
Downloading 312 bytes at 0x3fca0020...
Downloading 93164 bytes at 0x40380000...
Finished loading
I (802) spi_ram_loader: ********************************************
I (802) spi_ram_loader: *** Logs below are print from slave .... ***
I (812) spi_ram_loader: ********************************************
Hello world!
Hello world!
...
```
                                    

                                        
                                        # Flashing multiple partitions over USB CDC ACM interface

## Overview

This example demonstrates how to flash an ESP32-S3 from another ESP32-S3 or an ESP32-S2 MCU using the `esp_serial_flash` component API. Binaries to be flashed from host MCU to another Espressif SoC can be found in `binaries` folder and are converted into C-array during build process.

> **Note:** The `esp32_usb_cdc_acm` port requires ESP-IDF v4.4 or newer to build

Following steps are performed in order to re-program target's memory:

1. The system is started
2. The USB CDC ACM driver is initialized and a task to handle USB events is created
3. As soon as an ESP32-S3 is connected to the bus, a connection is opened with it using `loader_port_esp32_usb_cdc_acm_init()`. If the target USB Serial/JTAG peripheral is not active (e.g the device firmware is using the USB OTG peripheral), it is necessary to manually put it in download mode.
4. The flasher connection is started with the connected ESP32-S3 by calling `esp_loader_connect()`.
5. Binary file is opened and its size is acquired, as it has to be known before flashing.
6. Then `esp_loader_flash_start()` is called to enter flashing mode and erase amount of memory to be flashed.
7. `esp_loader_flash_write()` function is called repeatedly until the whole binary image is transferred.
8. After completion, the device can be manually reset and another ESP32-S3 can be connected to perform the flashing on.

> **Note:** The USB CDC ACM device of the ESP32-S3 does not support changing the baudrate, so the argument to `esp_loader_connect()` is irrelevant

## USB host driver usage

This example makes use of the Espressif [USB Host Driver](https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/api-reference/peripherals/usb_host.html).
In addition to initializing the host driver, a FreeRTOS task needs to be created to handle host usb events.

A binary semaphore is used as a lock for the connected device, and a callback is registered with the loader port which releases the lock so that a new device can be connected only after disconnection.

## Hardware Required

* One ESP32-S3 target board and one ESP32-S3 or ESP32-S2 board, with each board having USB connections
* An USB OTG adapter for the host board
* One or two USB cables for power supply and programming.

> **Note:** The USB connector on most ESP32-S3 and ESP32-S2 boards cannot supply power to the target, so a separate power connection is required

## Building and flashing

To run the example, type the following command:

```CMake
idf.py -p PORT flash monitor
```

(To exit the serial monitor, type ``Ctrl-]``.)

See the Getting Started Guide for full steps to configure and use ESP-IDF to build projects.

## Configuration

For details about available configuration option, please refer to top level [README.md](../../README.md). 
Compile definitions can be specified on command line when running `idf.py`, for example:

```
idf.py build -DMD5_ENABLED=1
```
Binaries to be flashed are placed in separate folder (binaries.c) for each possible target and converted to C-array. Without explicitly enabling MD5 check, flash integrity verification is disabled by default.

## Example output

Here is the example's console output:

```
...
I (541) main_task: Calling app_main()
I (541) usb_flasher: Installing USB Host
I (571) usb_flasher: Installing the USB CDC-ACM driver
I (571) usb_flasher: Opening CDC ACM device 0x303A:0x1001...
Connected to target
I (1121) usb_flasher: Loading bootloader...
Erasing flash (this may take a while)...
Start programming
Progress: 100 %
Finished programming
I (1681) usb_flasher: Loading partition table...
Erasing flash (this may take a while)...
Start programming
Progress: 100 %
Finished programming
I (1771) usb_flasher: Loading app...
Erasing flash (this may take a while)...
Start programming
Progress: 100 %
Finished programming
I (5011) usb_flasher: Done!
```