mittelab/kaproto

readme

# KAProto (C++)

[![pipeline status](https://git.mittelab.org/proj/kaproto-cpp/badges/master/pipeline.svg)](https://git.mittelab.org/proj/kaproto-cpp/-/commits/master)
[![Component Registry](https://components.espressif.com/components/mittelab/kaproto/badge.svg)](https://components.espressif.com/components/mittelab/kaproto)

KAProto is a [JSON-RPC](https://www.jsonrpc.org/specification) library based on
Niels Lohmann's [JSON library](https://github.com/nlohmann/json) and
[Libsodium](https://libsodium.gitbook.io/), targeting C++20.

**Original repo:** https://git.mittelab.org/proj/kaproto-cpp  
**ESP-IDF component release:** https://components.espressif.com/components/mittelab/kaproto  
**Python port:** https://git.mittelab.org/proj/kaproto-py  
**C++/Python interoperability demo:** https://git.mittelab.org/proj/kaproto-cross

We use this in our access control system (Keycard Access, hence KA-Proto), and
although it is intended for ESP-IDF, it works *by design* on desktop as well, with minimal
dependencies. It also has a [python port](https://git.mittelab.org/proj/kaproto-py) which we use
in our server-side software, and a small [demo project](https://git.mittelab.org/proj/kaproto-cross)
which checks python-c++ interoperability.

**Disclaimer:** this library is being developed as part of the access control system and is
released as a separate component for ease of testing, so for the time being documentation is
virtually non-existent beyond this file and unit/integration tests. But nobody will stop you from trying
it out 😉!

## Features
It a nutshell, it's [JSON-RPC](https://www.jsonrpc.org/specification) as protocol,
using [CBOR](https://www.rfc-editor.org/rfc/rfc8949.html) as encoding, wrapped in
Libsodium's [secret stream](https://libsodium.gitbook.io/doc/secret-key_cryptography/secretstream),
over TCP. It provides:
* **Bidirectional communication.**  
  One peer must act as a server and one as a client to establish the communication order, but
  other than that, either peer can perform request or respond to requests, or both.
* **Secure.**  
  As much as Libsodium's [secret streams](https://libsodium.gitbook.io/doc/secret-key_cryptography/secretstream), using
  an ephemeral key for each connection. However, for any meaningful exchange you probably want mutual authentication.
* **Mutual authentication.**  
  Each peer can be identified by its Ed25519 public key (which is just 32 bytes). A simple two-way
  mutual authentication challenge proves that the peer is in control of the corresponding private key.
* **Versionable.**  
  To support future development we include a simple version byte. At the moment this identifies client/server roles, and
  whether mutual authentication is enabled.
* **Relatively low overhead.**  
  CBOR is a pretty compact encoding already. We use UUID4 to identify a request-response pair, and
  encode it compactly using CBOR's own support for UUID. Beyond the redundant `"jsonrpc": "2.0"`
  entry which is requested by spec, the remaining overhead (packet length and crypto headers) is in the order of 40
  bytes.
* **Keepalive.**  
  The connection can be kept open if necessary, both peers ping each other regularly, so it can detect if the other end
  disconnected.
* **Exception-free.**   
  By default, ESP does not enable C++ exceptions, so we do not either. Every error state bubbles up in the returned
  value.
* **Multiplatform.**  
  ESP-IDF and desktop development, as well as a python port. We use all three, so we plan to keep supporting it.
* **Minimal dependencies.**  
  Only Libsodium, Nlohmann JSON, and a small library of ours which we use for string manipulation and work around the
  temporary lack of `std::expected` in ESP-IDF. We look forward to be able to drop that.

### Why

We needed to communicate between ESP and our server backend in a secure way (including mutual authentication).
We liked the idea of web APIs because of the flexibility (especially thinking of having the server written in python).
However, using SSL certificates and mutual TLS (especially on a local network with no Internet access and on ESP)
is quite cumbersome (and burdens the maintainer with the whole PKI infrastructure, certificate renewal, and transfer
onto the ESP), especially when considering that our system already had an Ed25519 keypair for each device.
That should be enough to authenticate and securely communicate between two devices...

Also, we do this for fun so why not.

### Should I use it?
Try taking a look at already established solutions, e.g. [Protobuf](https://protobuf.dev/),
[HTTP signatures](https://datatracker.ietf.org/doc/html/rfc9421),
[Mutual TLS](https://datatracker.ietf.org/doc/html/rfc8705); no one has audited this library for security after all.
Beyond that, why not! Reach out if you encounter issues (e.g. a different compiler setup needs more `typename`s 🙄).

## Usage

A short introduction on how to get started.

### Including the library on ESP
```shell
idf.py add-dependency "mittelab/kaproto^1.0.1"
```
That's it.

### Including the library on desktop
We did not commit to any specific C++ dependency manager, however,
since the project is CMake-based (because of ESP-IDF), here is how we recommend to do it:
1. Add `kaproto-cpp` as a submodule into your repo, e.g.
   ```shell
   git submodule add https://git.mittelab.org/proj/kaproto-cpp.git
   git submodule update --init kaproto-cpp
   cd kaproto-cpp
   git checkout 1.0.1  # Or any other version you need
   ```
2. In your root `CMakeLists.txt`, include all necessary directories:
   ```cmake
   cmake_minimum_required(VERSION 3.16)
   project(YourApp)
   
   # ...
   add_subdirectory(kaproto-cpp/kaproto)
   add_subdirectory(kaproto-cpp/managed/mittelib)
   add_subdirectory(kaproto-cpp/managed/libsodium)
   add_subdirectory(kaproto-cpp/managed/nlohmann-json/nlohmann-json)
   
   # ...
   target_link_libraries(YourAppTarget kaproto)
   set_property(TARGET YourAppTarget PROPERTY CXX_STANDARD 20)
   ```

See https://git.mittelab.org/proj/kaproto-cross for an example.

#### How are those dependencies actually provided?
Refer to https://git.mittelab.org/proj/kaproto-cpp/-/tree/master/managed; it's three files only (and a submodule).

* Libsodium is fetched from source using FetchContent in CMake, and we provide a super simple `version.h` file.  
* Mittelib is fetched from source using FetchContent in CMake, nothing else needed.
* Nlohmann's JSON is a git submodule (not of the original repo, but of the ESP-IDF component that downloads the
  header-only version of the library, so it is very lightweight). This cannot be downloaded via FetchContent because at
  the moment the source `json.hpp` need to sit next to the donwloaded header-only library (FetchContent downloads in the
  build directory). Probably it can be worked around but at the moment that is the state.

### Connection setup (client)
```c++
#include <cassert>
#include <ka/net/tcp.hpp>
#include <ka/proto/context.hpp>

void client() {

  // 1. ESTABLISH A TCP CONNECTION
  
  const auto addr = ka::net::address::from_string("127.0.0.1:8080");
  assert(addr);  // else: invalid address.
  
  auto socket = ka::net::tcp_socket::connect(*addr);
  assert(socket);  // else: could not connect.
  
  // 2. CREATE A KAPROTO CONTEXT
  
  ka::proto::context ctx{ka::proto::context_role::client};
  
  // NEXT:
  establish_comms(std::move(socket), ctx);
}
```

### Connection setup (server)
```c++
#include <cassert>
#include <ka/net/tcp.hpp>
#include <ka/proto/context.hpp>

void server() {

  // 1. LISTEN FOR A TCP CONNECTION
  
  const auto addr = ka::net::address::from_string("127.0.0.1:8080");
  assert(addr);  // else: invalid address.
  
  auto socket = ka::net::tcp_socket::listen(*addr, 1);
  assert(socket);  // else: could not bind.
  
  // 2. ACCEPT A TCP CONNECTION
  
  if (auto accepted = socket.accept(ka::net::no_timeout); accepted) {
      // Accept result contains a new socket, connected to the peer. We only serve one
      // connection so we can forget about the listening socket and continue.
      socket = std::move(accepted->first);
  } else {
    // Connection failed.
    return;
  }
  
  // 3. CREATE A KAPROTO CONTEXT
  
  ka::proto::context ctx{ka::proto::context_role::server};
  
  // NEXT:
  establish_comms(std::move(socket), ctx);
}
```

### Establishing communication (server and client)

Follows from the examples above.

**Note:** The connection is already secure here. There is no "plaintext" mode for KAProto; however you have
no information about the peer's identity (see Mutual authentication further down).
```c++
#include <cstdio>

void establish_comms(ka::net::tcp_socket socket, ka::proto::context &ctx) {
    ctx.connect(std::move(socket));

    if (not ctx.await_stable()) {
        std::printf("Handshake failed: %s\n", to_string(ctx.exit_status().error()));
        return;
    }
}
```

### Performing requests (server and client)

Follows from the examples above (in particular, requires a successful handshake).

```c++
#include <chrono>
#include <cstdio>
#include <ka/proto/context.hpp>
#include <string>

void demo_request(ka::proto::context &ctx) {
    // 1. SEND A REQUEST: ctx.request("request_name", <timeout>, <args...>).

    using namespace std::chrono_literals;

    if (auto request = ctx.request("method", 10s, "arg1", 2, 3.0); request) {
        // Request was successful, await response:

        // 2. AWAIT RESPONSE: ctx.response<ReturnType>(<request object>, <timeout>)

        if (auto response = ctx.response<std::string>(*request, 10s)) {

            // Response obtained:
            std::printf("Response: %s\n", response->c_str());
        } else {
            std::printf("Response failed: %s\n", to_string(response.error()));
            return;
        }
    } else {
        std::printf("Request failed: %s\n", to_string(request.error()));
        return;
    }

    // 3A. EXIT
    // ctx.close(); Optional: can just destroy the object

    // 3B. WAIT FOR PEER TO DISCONNECT
    ctx.await_disconnect(ka::net::no_timeout);
}
```

### Handling requests (server and client)

Handling requests must be done by registering handlers onto a `responder` object which is passed to the
context. Each request has a string name and is handled by a single "request handler" (which is a wrapper
for callables).
```c++
#include <chrono>
#include <cstdio>
#include <ka/proto/context.hpp>
#include <string>

// 0. OTHER OBJECTS THAT YOU MIGHT WANT TO SERVE THROUGH JSONRPC

void my_function(int arg) {
    // Do nothing
}

struct godzilla {
    void rawr() {
        std::printf("Rawr!\n");
    }
};

[[nodiscard]] godzilla &the_one_and_only() {
    static godzilla _instance{};
    return _instance;
}

// ------

void setup_handler() {
    // 1. CREATE A HANDLER OBJECT FOR EACH OF YOUR APIs
    auto handler_lambda = ka::proto::make_request_handler(
            [](std::string arg1, int arg2, float arg3) -> std::string {
                return "Hello!";
            });

    auto handler_function = ka::proto::make_request_handler(my_function);

    auto handler_method = ka::proto::make_request_handler(the_one_and_only(), godzilla::rawr);

    // 2. CREATE A RESPONDER OBJECT
    auto responder = std::make_shared<ka::proto::responder>();

    // 3. REGISTER ALL YOUR APIs:
    responder->register_handler("method", handler_lambda);
    responder->register_handler("my_function", handler_function);
    responder->register_handler("rawr", handler_method);

    ka::proto::context{ka::proto::context_role::client /* or server */,
                       std::move(responder) /* provide the responder */};

    // Continue with establishing connections and performing requests if needed.
    // At the end you probably want to serve all these calls until the peer is done
    // via ctx.await_disconnect().
}
```

### Notifications (server and client)

Notifications are just requests that do not need any response.

```c++
#include <chrono>
#include <cstdio>
#include <ka/proto/context.hpp>
#include <string>

void setup_handler_notifications() {
    // 1. CREATE A HANDLER OBJECT FOR EACH OF YOUR NOTIFICATIONS

    auto notification_handler = ka::proto::make_notification_handler([](std::string arg) {
        std::printf("%s\n", arg.c_str());
    });

    // 2. CREATE A RESPONDER OBJECT
    auto responder = std::make_shared<ka::proto::responder>();

    // 3. REGISTER ALL YOUR NOTIFICATIONS
    responder->register_handler("greet", notification_handler);

    // 4. PASS THE RESPONDER
    ka::proto::context{ka::proto::context_role::client /* or server */,
                       std::move(responder) /* provide the responder */};

    // Continue with establishing connections and performing requests if needed.
}

void notify_example(ka::proto::context &ctx) {
    using namespace std::chrono_literals;

    if (auto notification_result = ctx.notify("greet", 10s, "my name"); not notification_result) {
        std::printf("Failed to notify: %s\n", to_string(notification_result.error()));
    }
}
```

### Mutual authentication (server and client)

```c++
#include <cstdio>
#include <ka/proto/context.hpp>
#include <mlab/strutils.hpp>
#include <sodium/randombytes.h>
#include <string>

void setup_with_mauth() {
    // 0. HAVE A SECRET KEY THAT IDENTIFIES EACH PEER UNIQUELY!

    ka::crypto::raw_sec_key secret_key = {/* this should be initialized nontrivially! */};

    // For testing purposes, you can generate it with:
    randombytes_buf(secret_key.data(), secret_key.size());

    {
        // You can always obtain the public key:
        ka::crypto::raw_pub_key public_key = *ka::crypto::public_from_secret(secret_key);
        const std::string public_key_hex = mlab::data_to_hex_string(public_key);
        std::printf("Public key: %s\n", public_key_hex.c_str());
    }

    // 1. PASS IT INTO THE CONTEXT OBJECT

    ka::proto::context ctx{ka::proto::context_role::client /* or server */,
                           nullptr, /* responder */
                           secret_key /* that's it! */};

    // 2. OPTIONAL: SET A LOGIN VALIDATOR

    // A login validator returns true or false depending on whether the peer public key is acceptable or not
    ctx.set_login_validator([](ka::crypto::raw_pub_key own_pk, ka::crypto::raw_pub_key peer_pk) -> bool {
        return true;// Accept all logins, default
    });

    // 2. CONNECT AND AWAIT STABLE

    // ...Omitted, see examples above

    // 3. OPTIONAL: CHECK THE PEER IDENTITY (NEEDED IF YOU DO NOT USE A LOGIN VALIDATOR)
    {
        const std::string public_key_hex = mlab::data_to_hex_string(*ctx.peer_public_key());
        std::printf("Peer public key: %s\n", public_key_hex.c_str());
    }

    // 4. PERFORM YOUR EXCHANGES

    // ...Omitted, see examples above
}
```

## More help and examples

* This is a demo app from which the examples above are taken, we use it in our testing CI:  
  https://git.mittelab.org/proj/kaproto-cross/-/blob/master/main.cpp
* Unit test included in kaproto:  
  https://git.mittelab.org/proj/kaproto-cpp/-/blob/master/tests/main.cpp