Introduction

Copapy is a Python framework for deterministic, low-latency real-time computation with automatic differentiation support, targeting hardware applications - for example in the fields of robotics, aerospace, SDR, embedded systems and control systems in general.

GPU frameworks like PyTorch, JAX and TensorFlow jump-started the development in the field of AI. With the right balance of flexibility and performance, they allow for fast iteration of new ideas while still being performant enough to test or even use them in production.

This is exactly what Copapy aims for - but in the field of embedded realtime computation. While making use of the ergonomics of Python, the tooling, and the general Python ecosystem, Copapy runs seamlessly optimized machine code. Despite being highly portable, the copy-and-patch compiler allows for effortless and fast deployment without any dependencies beyond Python. It’s designed to feel like writing Python scripts with a shallow learning curve, but under the hood it produces high-performance, statically typed and memory-safe code with a minimized set of possible runtime errors[1]. To maximize productivity, the framework provides detailed type hints to catch most errors even before compilation.

Embedded systems come with a variety of CPU architectures. The copy-and-patch compiler already supports the most common ones[3], and porting it to new architectures is straightforward if a C compiler for the target architecture is available[2]. The generated code depends only on the CPU architecture. The generated binaries neither perform system calls nor rely on external libraries like libc. This makes Copapy both highly deterministic and easy to deploy on different realtime operating systems (RTOS) or bare metal.

The main features can be summarized as:

• Fast to write & easy to read

• Memory and type safety with a minimal set of runtime errors

• Deterministic execution

• Automatic differentiation for efficient realtime optimization (reverse-mode)

• Optimized machine code for x86_64, ARM32 (Cortex-A and Cortex-M), ARM64, RISC-V (32 and 64 Bit) and TriCore (AURIX)

• Highly portable to new architectures

• Small Python package with minimal dependencies and no cross-compile toolchain required

Execution of the compiled code is managed by a runner application. The runner is implemented in C and handles I/O and communication with the Copapy framework. The overall design emphasizes minimal complexity of the runner to simplify portability, since this part must be adapted for the individual hardware/application. Because patching of memory addresses is done by the runner, the different architecture-specific relocation types are unified to an architecture-independent format by Copapy before sending the patch instructions to the runner. This keeps the runner implementation as minimal as possible.

The design targets either an architecture with a realtime-patched Linux kernel - where the runner uses the same CPU and memory as Linux but executes in a realtime thread - or a setup where even higher determinism is required. In such cases, the runner can be executed on a separate crossover MCU running on bare metal or a RTOS.

The Copapy framework also includes a runner as a Python module built from the same C code. This allows frictionless testing of code and might be valuable for using Copapy in conventional application development.

Current state

The computation engine is fully functional - for all above mentioned target architectures - and available for testing and experimentation simply by installing the package.

Hardware I/O and real-time execution are available with the runner for the Zephyr RTOS. It executes periodic tasks with deterministic timing (logical execution time): a task reads its inputs at its release and publishes its outputs at the end of its period. The host is connected over a serial port (e.g. USB) by cp.SerialTarget to deploy the tasks, start and stop them, and to read and write values while they are running. Device I/O channels - GPIO, PWM, quadrature encoder and ADC - are bound to the values of the tasks. If the tasks need more CPU time than available, the tasks with the lowest priority are degraded to keep the connection to the host responsive. Supported boards are currently the Teensy 4.1 (Cortex-M7).

Values of periodic tasks are signals in concept: one value per run of the task. cp.delay, cp.integrate, cp.rate and cp.feedback operate on these signals without side effects. Their state is kept in the memory of the task, cp.dt is the period of the task. Orientations as quaternions are integrated from angular velocities by cp.integrate_rotation(gyro), cp.angular_rate(q) is the angular velocity of an orientation:

setpoint = cp.value(1.0)
position = cp.value(0.0)  # e.g. bound to an encoder with tg.bind
error = setpoint - position
control = 2.0 * error + 5.0 * cp.integrate(error) + 0.1 * cp.rate(error)
smoothed = cp.feedback(lambda prev: prev + 0.2 * (control - prev), init=0.0)

tg = cp.LocalTarget(tick=1e-3)
tg.deploy(cp.Task(smoothed, period=1e-3))
tg.step(0.1)  # 100 runs in simulated time

Array stencils for handling large arrays - e.g., for machine vision and neural network applications - are available and use auto-vectorizing of the C compiler. Automatic differentiation covers element-wise array operations, reductions and matrix products, but not yet convolutions, pooling, sorting, transposing and slicing of arrays.

Furthermore in development is currently constant regrouping for further symbolic optimization of the computation graph.

Benchmark performance shows promising numbers. The following chart compares three ways of running the same computation with Copapy and NumPy 2.0.1:

For the benchmark (tests/benchmark.py) the median time per iteration of calculating the term sum((v1 + i) @ v2 for i in range(10)) was measured on a Ryzen 5 3400G (15 runs of 30,000 iterations each). The vectors v1 and v2 both have lengths of v_size, which was varied from 10 to 500 according to the chart. For the NumPy case the i in range(10) loop was vectorized like this: np.sum((v1 + i) @ v2) with i being an NDArray of shape [10, 1]. The number of calculated scalar operations is the same for all implementations. The Copapy variants are:

• Copapy scalar: cp.vector(..., packed=False), one scalar stencil per element. The code grows with v_size but they profit from sparcity (no benefit in this benchmark).

• Copapy packed: cp.vector(..., packed=True), array stencils that are auto-vectorized by the C compiler. They compile much faster for large arrays. Without the packed argument, vectors with more than 64 elements are packed automatically.

• Copapy jit: the term as a function decorated with @cp.jit, called with the two NumPy arrays. Unlike the other variants, this includes copying the input arrays and reads back the result.

Furthermore, for many applications Copapy performance will benefit by reducing the actual number of operations significantly compared to a NumPy implementation, by precomputing constant values known at compile time and benefiting from sparsity. Multiplying by zero (e.g., in a diagonal matrix) eliminates a whole branch in the computation graph. Operations with no effect, like multiplications by 1 or additions with zero, get eliminated at compile time.

For testing and using Copapy to speed up computations in conventional Python programs there is also the @cp.jit decorator available, to compile functions on first use and cache the compiled version for later calls:

import numpy as np
import copapy as cp

@cp.jit
def rms_error(measured, reference):
    error = measured - reference
    return cp.sqrt((error * error).sum() / error.size)

@cp.jit
def normalize(v):
    return cp.map_to_numpy(v / cp.sqrt((v * v).sum()))

measured = np.array([1.0, 2.1, 2.9, 4.2], dtype=np.float32)
reference = np.array([1.0, 2.0, 3.0, 4.0], dtype=np.float32)

# Compile and run:
rms = rms_error(measured, reference)  # 0.1225

# Run cached compiled version:
rms = rms_error(measured * 2, reference)

# Array result as NumPy array:
unit = normalize(reference)  # array([0.183, 0.365, 0.548, 0.730], dtype=float32)

Arguments can be numbers, sequences of numbers, or arrays with the buffer protocol, such as NumPy arrays. A function is compiled again for each new combination of argument types and array shapes. NumPy arrays of byte, 32 bit int or 32 bit float are copied to the compiled code as they are, other numeric types are converted to 32 bit. Array results are returned as nested lists, or as NumPy arrays if marked with cp.map_to_numpy. Such a NumPy array is a view of the memory of the compiled code and changes with the next call, unless copy=True is passed.

Install

To install Copapy, you can use pip. Precompiled wheels are available for Linux (x86_64, AArch64, ARMv7, i686), Windows (x86_64) and macOS (x86_64, AArch64):

pip install copapy

License

This project is licensed under the MIT license - see the LICENSE file for details.

---

[1]

Errors like divide-by-zero are currently still possible. The feasibility of tracking value ranges in the type system is under investigation to enable compile-time checks.

[2]

The compiler must support tail-call optimization (TCO). Currently, GCC is supported. Porting to a new architecture requires implementing a subset of relocation types used by that architecture.

[3]

Supported architectures: x86_64, x86 (32 Bit), AArch64, ARMv6/7 (non-Thumb), ARMv7 Thumb for Cortex-A and Cortex-M (M4 and M7), ARMv8.1-M Thumb with Helium (Cortex-M55 and M85), RISC-V (RV32/RV64 with hardware floating point) and TriCore 1.6.1 (AURIX TC2xx).