Antigoals

The motivation behind this REP does not include:

• Dictating policy implementation specifics on maintainers to achieve hardware acceleration and/or create acceleration kernels (e.g use of CUDA, HLS, OpenCL, or similar)
  • Policy requirements about which framework to use (e.g. ROCm, JetPack, etc.) or which specific C++ preprocessor instructions are intentionally generic and to remain technology-agnostic.
  • Maintainers can come up with their own policies depending on the technology targeted (FPGAs, GPUs, etc.) and framework used (HLS, ROCm, CUDA, etc).
• Enforcing specific hardware solutions on maintainers
  • No maintainer is required to target any specific hardware solution by any of the guidelines in this REP.
  • Instead, maintainers are free to choose from either using the proposed generic CMake macros meant to support all accelerators (e.g. acceleration_kernel ), or instead use vendor-specific CMake extensions for particular supported technologies and hardware solutions (e.g. vitis_acceleration_kernel ).
• Excluding any supported ROS 2 OS.

Kernel levels in ROS 2

To favour modularity, organize acceleration kernels and allow robotics architects to select only those accelerators needed to meet the requirements of their application, acceleration kernels in ROS 2 shall be classified in 3 levels according to the ROS layer/underlayer they impact:

• Level 1 - ROS 2 applications and libraries: This group corresponds with acceleration kernels that speed-up OSI L7 applications or libraries on top of ROS 2. Any computation on top of ROS 2 is a good a candidate for this category. Examples include selected components in the navigation, manipulation, perception or control stacks.

• Level 2 - ROS 2 core packages: This includes kernels that accelerate or offload OSI L7 ROS 2 core components and tools to a dedicated acceleration solution (e.g. an FPGA). Namely, we consider rclcpp, rcl, rmw, and the corresponding rmw_adapters for each supported communication middleware. Examples includes ROS 2 executors for more deterministic behaviours [4], or complete hardware offloaded ROS 2 Nodes [5].

• Level 3 - ROS 2 underlayers: Groups together all accelerators below the ROS 2 core layers belonging to OSI L2-L7, including the communication middleware (e.g. DDS). Examples of such accelerators include a complete or partial DDS implementation, an offloaded networking stack or a data link layer for real-time deterministic, low latency and high throughput interactions.

Hardware acceleration solutions complying with this REP should aspire to support multiple kernel levels in ROS 2 to maximize consumer experience.

Pillar I - Extensions to ament

The first pillar represents extensions of the ament ROS 2 build system. These CMake extensions help achieve the objective of simplifying the creation of acceleration kernels. By providing an experience and a syntax similar to other ROS 2 libraries targeting CPUs, maintainers will be able to integrate acceleration kernels into their packages easily. The ament_hardware_acceleration ROS 2 package abstracts the build system extensions from technology-specific frameworks and software platforms. This allows to easily support hardware acceleration across FPGAs and GPUs while using the same syntax, simplifying the work of maintainers. The code listing below provides an example that instructs the CMakeLists.txt file of a ROS 2 package to build a vadd acceleration kernel indicating the corresponding sources without the need to define a target accelerator:

acceleration_kernel(
  NAME vadd
  FILE src/vadd.cpp
  INCLUDE
    include
)

Under the hood, each specialization of ament_hardware_acceleration should rely on the corresponding frameworks, tools and libraries to enable building the acceleration kernel. For example, ament_vitis relies on Vitis Unified Software Platform (Vitis for short) to generate the appropriate acceleration kernels. The developer of such kernel would need to choose and implement how the CPU ROS abstractions (e.g. Nodes) would communicate with the acceleration kernel, if either through OpenCL or through the Xilinx Runtime (XRT) library, but that's abstracted away. In other words, the definition of the communication between the application code (ROS Node) and the acceleration kernels is decided by the developer and reflected in the source code of the Node. When using ament_hardware_acceleration macros such as acceleration_kernel , Vitis, OpenCL and XRT are completely hidden from the robotics engineer, simplifying the creation of kernels through simple CMake macros in the CMakeLists.txt file of the ROS package. If desired, the developer can express the same kernel using specializations ament_hardware_acceleration. In the case of ament_vitis , the developer can use the vitis_acceleration_kernel macro to express the same kernel as above, but with finer-grained details:

vitis_acceleration_kernel(
  NAME vadd
  FILE src/vadd.cpp
  CONFIG src/kv260.cfg
  INCLUDE
    include
  TYPE
    sw_emu
    # hw_emu
    # hw
  PACKAGE
)

While ament_hardware_acceleration CMake macros are preferred and encouraged, maintainers are free to choose among all the CMake macros available within each of the specializalizations. After all, it'll be hard to define a generic set of macros that fits all use cases across technologies and silicon vendors. Maintainers are, however, encouraged to produce ROS packages that consider various accelerators. To do so, each extension of the ament ROS 2 build system for hardware acceleration purposes shall define CMake hardware acceleration variables. These variables are meant to be used by the maintainers to conditionally compile their ROS 2 packages for specific accelerators. The following table lists the variables defined by ament_hardware_acceleration and its specializalization ament_vitis for hardware acceleration purposes. Other specializalizations should follow along the same lines:

Through ament_hardware_acceleration and technology-specific specializations (like ament_vitis), the ROS 2 build system is automatically enhanced to support producing acceleration kernel and related artifacts as part of the build process when invoking colcon build. To facilitate the work of maintainers, this additional functionality is configurable through mixins that can be added to the build step of a ROS 2 workspace, triggering all the hardware acceleration logic only when appropriate (e.g. when --mixin kv260 is appended to the build effort, it'll trigger the build of kernels targeting the KV260 hardware solution). For a reference implementation of these enhacements, refer to ament_vitis.

In turn, additional extensions to the existing CMake macros might be proposed which would allow to support more technologies and hardware solutions. For example, ament_vitis provides a vitis_acceleration_kernel macro that can be used to generate kernels for the Xilinx Vitis platform. Similarly, ament_rocm could provide a rocm_acceleration_kernel macro that can be used to generate kernels for the AMD ROCm platform. This way, maintainers can choose to use the generic acceleration_kernel macro, or instead use the technology-specific macros to target specific hardware solutions. Also, future extensions to the ament_hardware_acceleration package could be proposed to support the use of accelerators in binary format instead of source builds. This would allow to support accelerators which are are not fully integrated into ROS 2 through their corresponding technology libraries (e.g. FPGAs that are not supported by Vitis).

Pillar II - firmware

The second pillar is firmware, it is meant to provide firmware artifacts for each supported technology so that the kernels can be compiled against them, simplifying the process for consumers and maintainers, and further aligning with the ROS typical development flow.

Each ROS 2 workspace can have one or multiple firmware packages deployed. The selection of the active firmware in the workspace is performed by simlinking the preferred one or by using auxiliary tools that extend the colcon ROS 2 meta-build system (see the colcon-hardware-acceleration ROS package and with it, the colcon acceleration select subverb). To get a technology solution aligned with this REP's architecture, each vendor should provide and maintain an acceleration_firmware_<solution> package specialization that delivers the corresponding artifacts. Firmware artifacts should be deployed at <ros2_workspace_path>/acceleration/firmware/<solution> and be ready to be used by the ROS 2 build system extensions at (pillarI) . For a reference implementation of specialized vendor firmware package, see acceleration_firmware_kv260.

By splitting vendors across packages, consumers and maintainers can easily switch between hardware acceleration solutions within the same workspace.

Methodology for developing ROS 2 packages that integrate hardware acceleration

                                                  rebuild

                                             +---------------+
                                             |               |
                                             |               |
                                             |4. benchmark   +--+
                                             |   acceleration|  |
                                          +-->               |  |
                                          |  +---------------+  |
                                          |                     | acceleration
                                          |                     | tracing
              trace dataflow              |                     |
             +--------------+             |   +---------------+ |
             |              |             +---+               +<+
+------------v---+ +--------v-------+         |               |
|                | |                |         |               |
| 3.2 accelerate | | 3.1 accelerate <---------> 3. hardware   |
|     graph      | |     nodes      |  trace  |  acceleration |
|                | |                |  nodes  |               <-+
+----------------+ +----------------+         |               | |
                                              +---------------+ |
                                                                |
                                                                | CPU
                                                                | tracing
                                              +--------------+  |
                  +----------------+  rebuild |              |  |
                  |                +---------->              |  |
start  +----------> 1. trace graph |          | 2. benchmark +--+
                  |                |          |    CPU       |
                  +----+------^--^-+          |              |
                       |      |  |            +-------+------+
                       |      |  |                    |
                       +------+  |                    |
                        Tracer   +--------------------+
                                     re-instrument

This can be summarized as follows:

• 1. instrument the core components of a ROS 2 application using a tracing framework and trace its execution to obtain information about its time bottlenecks. Refer to REP-2014 for guidelines on tracing and benchmarking performance in ROS 2, including examples of tracing frameworks and how to use them.
• 2. trace, benchmark and analyze the ROS 2 application on the CPU to establish a compute baseline; Determine which functions are subject to be hardware accelerated.
• 3. develop a hardware acceleration kernels for those functions identified before, and optimize the dataflow across Nodes

  • 3.1 accelerate computations at the Node or Component level for each one of those functions identified in 2. as good candidates.
  • 3.2 accelerate inter-Node exchanges and reduce the overhead of the ROS 2 message-passing system across all its abstraction layers. To do so, consider leveraging REP-2007 and REP-2009.

• 4. trace, benchmark and analyze the ROS 2 application including the acceleration kernels and compare against the CPU baseline obtained in 2. to determine the performance improvement. Re-iterate 3. and 4. until the desired performance is achieved.

Additional about tracing and benchmarking are beyond the scope of this REP. For more details about tracing and benchmarking, see REP-2014.

Acceleration examples

For the sake of illustrating maintainers and consumers how to build their own acceleration kernels and guarantee interoperability across technologies, a ROS 2 meta-package named acceleration_examples will be maintained and made available. This meta-package will contain various packages with simple common acceleration examples. Each one of these examples should aim to support all hardware acceleration solutions complying with this REP.

Template for Vendors

Silicon vendors and solution manufacturers can help set the expectations of the level of support their hardware acceleration technology provides in alignment with this REP by providing a template in the README.md files of their firmware and/or ament extensions. Doing so will facilitate the process for consumers and maintainers.

For a Markdown syntax example of such template, refer to acceleration_firmware_kr260.