shootao/esp-vfs-mem

readme

# ESP Memory VFS Component

A lightweight, memory-based virtual file system component for ESP-IDF. This component implements a RAM-based filesystem that allows applications to create, read, write, and manipulate files entirely in memory.

## Overview

The Memory VFS component provides a virtual file system that operates entirely in RAM, making it useful for:

- Temporary file storage
- Fast data exchange between components
- Testing and development
- Scenarios where flash storage is limited or unavailable
- Applications that need high-speed file operations

## Features

- **Memory-based storage**: All files are stored in RAM
- **Standard file operations**: Full support for POSIX-style operations (open, read, write, seek, close)
- **File management**: Create, delete, and rename files
- **Dynamic memory allocation**: Automatic buffer resizing for growing files
- **Configurable capacity**: Set the maximum number of files at initialization
- **ESP-IDF integration**: Seamlessly works with ESP-IDF's VFS layer
- **Thread safety**: Mutex protection for concurrent access to files

## Directory Structure

```
esp-vfs-mem/
├── include/
│   └── esp_vfs_mem.h       # Public API header
├── src/
│   ├── vfs_mem.c           # Main implementation
│   └── vfs_mem_internal.h  # Internal definitions
├── test_apps/              # Example applications
│   └── stdio_example.c     # Example using standard C library API
├── CMakeLists.txt          # Build configuration
└── README.md               # This file
```

## Installation

### As a Component in Your Project

1. Copy this repository into your project's `components` directory:

   ```bash
   mkdir -p components
   cp -r esp-vfs-mem components/
   ```

2. Or add it as a git submodule:

   ```bash
   git submodule add https://github.com/yourusername/esp-vfs-mem.git components/esp-vfs-mem
   ```

### Using Component Manager

Add the following to your project's `idf_component.yml`:

```yaml
dependencies:
  esp-vfs-mem:
    version: "^1.0.0"
    # Replace with actual repository URL
    git: https://github.com/yourusername/esp-vfs-mem.git
```

## API Reference

### Registering the Filesystem

```c
esp_err_t esp_vfs_mem_register(const char* base_path, size_t max_files);
```

- **base_path**: Mount point for the filesystem (e.g., "/mem")
- **max_files**: Maximum number of files the filesystem can handle (default: 5)
- **Returns**: ESP_OK on success, or an error code

### Unregistering the Filesystem

```c
esp_err_t esp_vfs_mem_unregister(const char* base_path);
```

- **base_path**: Mount point of the filesystem to unregister
- **Returns**: ESP_OK on success, or an error code

## Usage Examples

### Basic Usage

```c
#include "esp_vfs_mem.h"
#include "esp_log.h"

static const char *TAG = "mem_vfs_example";

void app_main(void)
{
    // Register memory filesystem at /mem path with support for up to 10 files
    ESP_ERROR_CHECK(esp_vfs_mem_register("/mem", 10));
    ESP_LOGI(TAG, "Memory filesystem registered at /mem");
    
    // Create and write to a file
    FILE *f = fopen("/mem/hello.txt", "w");
    if (f == NULL) {
        ESP_LOGE(TAG, "Failed to open file for writing");
        return;
    }
    
    fprintf(f, "Hello, ESP Memory VFS!");
    fclose(f);
    
    // Read from the file
    f = fopen("/mem/hello.txt", "r");
    if (f == NULL) {
        ESP_LOGE(TAG, "Failed to open file for reading");
        return;
    }
    
    char buf[64];
    size_t bytes_read = fread(buf, 1, sizeof(buf) - 1, f);
    buf[bytes_read] = '\0';
    ESP_LOGI(TAG, "Read from file: %s", buf);
    fclose(f);
    
    // Unregister when done
    ESP_ERROR_CHECK(esp_vfs_mem_unregister("/mem"));
}
```

### POSIX-style File Operations

```c
// Open a file for writing
int fd = open("/mem/data.txt", O_WRONLY | O_CREAT | O_TRUNC, 0666);
if (fd < 0) {
    ESP_LOGE(TAG, "Failed to open file: %s", strerror(errno));
    return;
}

// Write data
const char *data = "POSIX API example";
write(fd, data, strlen(data));
close(fd);

// Open for reading
fd = open("/mem/data.txt", O_RDONLY);
if (fd < 0) {
    ESP_LOGE(TAG, "Failed to open file for reading");
    return;
}

// Read data
char buf[64];
ssize_t bytes_read = read(fd, buf, sizeof(buf) - 1);
if (bytes_read > 0) {
    buf[bytes_read] = '\0';
    ESP_LOGI(TAG, "Read content: %s", buf);
}
close(fd);

// File management
rename("/mem/data.txt", "/mem/renamed.txt");
unlink("/mem/renamed.txt");  // Delete file
```

### Advanced Usage

```c
// Create a binary file
FILE *f = fopen("/mem/binary.dat", "wb");
if (f == NULL) {
    ESP_LOGE(TAG, "Failed to create binary file");
    return;
}

// Write structured data
typedef struct {
    uint32_t id;
    float value;
    char name[16];
} record_t;

record_t records[] = {
    {1, 3.14, "Record 1"},
    {2, 2.71, "Record 2"},
    {3, 1.41, "Record 3"}
};

fwrite(records, sizeof(record_t), 3, f);
fclose(f);

// Random access
f = fopen("/mem/binary.dat", "rb");
if (f == NULL) {
    ESP_LOGE(TAG, "Failed to open binary file");
    return;
}

// Seek to second record
fseek(f, sizeof(record_t), SEEK_SET);

// Read just one record
record_t record;
if (fread(&record, sizeof(record_t), 1, f) == 1) {
    ESP_LOGI(TAG, "Record ID: %lu, Value: %.2f, Name: %s", 
             record.id, record.value, record.name);
}
fclose(f);
```

### Thread Safety

The memory filesystem is thread-safe, allowing multiple tasks to safely access files concurrently:

```c
// Task 1
void writer_task(void *pvParameters)
{
    while (1) {
        FILE *f = fopen("/mem/shared.txt", "w");
        if (f) {
            fprintf(f, "Data from writer task");
            fclose(f);
        }
        vTaskDelay(pdMS_TO_TICKS(100));
    }
}

// Task 2
void reader_task(void *pvParameters)
{
    while (1) {
        FILE *f = fopen("/mem/shared.txt", "r");
        if (f) {
            char buf[64];
            size_t bytes_read = fread(buf, 1, sizeof(buf) - 1, f);
            buf[bytes_read] = '\0';
            ESP_LOGI(TAG, "Read: %s", buf);
            fclose(f);
        }
        vTaskDelay(pdMS_TO_TICKS(150));
    }
}
```

## Performance Considerations

- **Memory Usage**: Each file consumes memory for both its content and metadata
- **Automatic Resizing**: Files grow dynamically as needed, doubling in size when more space is required
- **Speed**: Operations are very fast since they occur entirely in RAM
- **Volatility**: All data is lost on power cycle or when the filesystem is unregistered
- **Mutex Protection**: Thread safety adds a small overhead to file operations

## Limitations

- All data is stored in RAM, so it's volatile and limited by available memory
- No directory structure support (flat file organization)
- No file permissions or access control
- Mutex timeout is set to 1000ms by default

## Contributing

Contributions to improve the Memory VFS component are welcome! Please feel free to submit issues or pull requests.