Introduction

Copapy is a Python framework for deterministic, low-latency real-time computation with automatic differentiation support, targeting hardware applications - for example in the fields of robotics, aerospace, SDR, embedded systems and control systems in general.

GPU frameworks like PyTorch, JAX and TensorFlow jump-started the development in the field of AI. With the right balance of flexibility and performance, they allow for fast iteration of new ideas while still being performant enough to test or even use them in production.

This is exactly what Copapy aims for - but in the field of embedded realtime computation. While making use of the ergonomics of Python, the tooling, and the general Python ecosystem, Copapy runs seamlessly optimized machine code. Despite being highly portable, the copy-and-patch compiler allows for effortless and fast deployment without any dependencies beyond Python. It’s designed to feel like writing Python scripts with a shallow learning curve, but under the hood it produces high-performance, statically typed and memory-safe code with a minimized set of possible runtime errors[1]. To maximize productivity, the framework provides detailed type hints to catch most errors even before compilation.

Embedded systems come with a variety of CPU architectures. The copy-and-patch compiler already supports the most common ones[3], and porting it to new architectures is straightforward if a C compiler for the target architecture is available[2]. The generated code depends only on the CPU architecture. The generated binaries neither perform system calls nor rely on external libraries like libc. This makes Copapy both highly deterministic and easy to deploy on different realtime operating systems (RTOS) or bare metal.

The main features can be summarized as:

• Fast to write & easy to read

• Memory and type safety with a minimal set of runtime errors

• Deterministic execution

• Automatic differentiation for efficient realtime optimization (reverse-mode)

• Optimized machine code for x86_64, 32 Bit ARM (Cortex-A and Cortex-M) and AArch64

• Highly portable to new architectures

• Small Python package with minimal dependencies and no cross-compile toolchain required

Execution of the compiled code is managed by a runner application. The runner is implemented in C and handles I/O and communication with the Copapy framework. The overall design emphasizes minimal complexity of the runner to simplify portability, since this part must be adapted for the individual hardware/application. Because patching of memory addresses is done by the runner, the different architecture-specific relocation types are unified to an architecture-independent format by Copapy before sending the patch instructions to the runner. This keeps the runner implementation as minimal as possible.

The design targets either an architecture with a realtime-patched Linux kernel - where the runner uses the same CPU and memory as Linux but executes in a realtime thread - or a setup where even higher determinism is required. In such cases, the runner can be executed on a separate crossover MCU running on bare metal or a RTOS.

The Copapy framework also includes a runner as a Python module built from the same C code. This allows frictionless testing of code and might be valuable for using Copapy in conventional application development.

Current state

While hardware I/O is obviously a core aspect of the project it is not yet available. However the computation engine is already fully functional - for all above mentioned target architectures - and available for testing and experimentation simply by installing the package. The project focuses now on integration into the first demonstration hardware platform.

Furthermore in development are currently:

• Array stencils for handling large arrays and generating SIMD-optimized code - e.g., for machine vision and neural network applications

• Constant regrouping for further symbolic optimization of the computation graph

Despite missing SIMD-optimization, benchmark performance shows promising numbers. The following chart plots the results in comparison to NumPy 2.3.5:

For the benchmark (tests/benchmark.py) timings for 30,000 iterations of calculating the term sum((v1 + i) @ v2 for i in range(10)) were measured on a Ryzen 5 3400G. The vectors v1 and v2 both have lengths of v_size, which was varied from 10 to 500 according to the chart. For the NumPy case the i in range(10) loop was vectorized like this: np.sum((v1 + i) @ v2) with i being an NDArray of shape [10, 1]. The number of calculated scalar operations is the same for both implementations. Copapy benefits from lower overhead by calling a single function from Python per iteration, whereas the NumPy variant requires three. Interestingly, the chart shows no indication that for increasing v_size the calling overhead for NumPy will be compensated by faster SIMD instructions. Note that in this benchmark the Copapy case does not move any data between Python and the compiled code.

Furthermore, for many applications Copapy performance will benefit by reducing the actual number of operations significantly compared to a NumPy implementation, by precomputing constant values known at compile time and benefiting from sparsity. Multiplying by zero (e.g., in a diagonal matrix) eliminates a whole branch in the computation graph. Operations with no effect, like multiplications by 1 or additions with zero, get eliminated at compile time.

For testing and using Copapy to speed up computations in conventional Python programs there is also the @cp.jit decorator available, to compile functions on first use and cache the compiled version for later calls:

import copapy as cp

@cp.jit
def calculation(x: float, y: float) -> float:
    return sum(x ** 2 + y ** 2 + i for i in range(10))

# Compile and run:
result1 = calculation(2.5, 1.2)

# Run cached compiled version:
result2 = calculation(3.1, 4.7)

Note that cp.jit is not currently highly optimized for data transfer between Python and the compiled code.

Install

To install Copapy, you can use pip. Precompiled wheels are available for Linux (x86_64, AArch64, ARMv7), Windows (x86_64) and macOS (x86_64, AArch64):

pip install copapy

License

This project is licensed under the MIT license - see the LICENSE file for details.

---

[1]

Errors like divide-by-zero are currently still possible. The feasibility of tracking value ranges in the type system is under investigation to enable compile-time checks.

[2]

The compiler must support tail-call optimization (TCO). Currently, GCC is supported. Porting to a new architecture requires implementing a subset of relocation types used by that architecture.

[3]

Supported architectures: x86_64, AArch64, ARMv6/7 (non-Thumb) and ARMv7 Thumb for Cortex-A and Cortex-M. Code for x86 32-bit exists but has unresolved issues and a low priority.