The OpenVX Working Group releases new extensions plus roadmap guidance to help developers ensure compliance with safety standards, process synchronized metadata more efficiently, and provide insights into future OpenVX enhancements that will make it more powerful, flexible, and easier to implement.

In today’s rapidly evolving landscape of automotive and embedded systems, sensor data processing is a cornerstone for achieving advanced functionalities like autonomous driving, parking assistance, and driver monitoring. These applications rely on the seamless orchestration of diverse sensors such as radar, video, lidar, and ultrasonic systems, which produce varied data streams optimized for distinct purposes like computer vision, human vision, or infrared imaging. OpenVX™, an open royalty-free standard from the Khronos® Group, has emerged as a vital framework for developers, enabling them to create high-performance, cross-platform applications without rewriting code for each processor.

With OpenVX, developers can optimize performance across a variety of sensor arrays while benefiting from a low memory footprint. OpenVX enables portability across ISPs, DSPs, GPUs, and multi-core CPUs. It also enables efficiency across low-power domains, making it a critical tool for building real-time applications in embedded and mobile platforms. OpenVX’s unique graph-based abstraction and reliable hardware acceleration ensure that developers can achieve performance comparable to hand-optimized, non-portable code.

Five New OpenVX Extensions for Enhanced Security and Performance

To make it easier to efficiently handle metadata, optimize memory usage, and manage complex data flows, OpenVX has introduced new extensions that are setting the stage for enhanced safety, usability, performance, and advanced data handling:

• Safe Casts 1.3.1: OpenVX's C-based API employs an inheritance-like model, allowing objects to be cast to references and vice versa. However, C lacks true inheritance, and pointer casting between different types can pose safety risks, particularly when adhering to coding standards like MISRA. The X_KHR_SAFE_CASTS extension introduces mechanisms to enforce safe type checking during these operations. By querying runtime type information before casting, developers can ensure compliance with safety standards and reduce errors, bolstering confidence in mission-critical applications like automotive systems.
• Supplementary Data: This extension allows developers to associate supplementary metadata with standard OpenVX data types. Instead of passing additional parameters to kernels, developers can embed synchronized metadata—such as exposure times, gain values, or error conditions—directly with output images or other objects. This approach ensures that critical information is propagated throughout the graph without disrupting execution. For example, supplementary data can help track processing conditions across graph nodes or retain key metrics for debugging and analysis.
• Bidirectional Parameters Extension 1.3.1: This extension reintroduces bidirectional parameters to OpenVX, allowing for efficient in-place data modification. By eliminating unnecessary data duplication and transfer operations, it significantly reduces memory usage and computational overhead. The extension provides clear rules and graph formalism to address the difficulties that led to their removal in previous versions. For tasks like modifying specific channels of multi-channel images, bidirectional parameters can reduce the node count and eliminate redundant read/write operations, resulting in substantial performance improvements for image processing pipelines.
• Raw Image Extension 1.0: This extension defines a standardized framework for handling raw camera sensor data prior to ISP processing. It accommodates various sensor readout types with minimal required attributes, supporting multi-exposure wide dynamic range sensors through both separate CSI Virtual Channels and single-channel interleaved formats. The extension provides flexible data access APIs for working with pixel data across different exposure configurations and includes support for vendor-specific metadata that may contain critical information for auto-exposure and white balance algorithms. This foundation enables consistent handling of raw sensor data across the diverse range of image sensors used in the industry.
• Swap And Move Kernel Extension 1.3.1: This extension introduces two specialized kernels that leverage bidirectional parameters to enable efficient data management within OpenVX graphs. The SWAP kernel exchanges data contents between two references of compatible types, while the MOVE kernel transfers data from a bidirectional parameter to an output parameter, leaving the source undefined. These operations facilitate sophisticated graph structures, such as pipelined processing, inter-graph data exchange, and interaction with delay objects or arrays. By exploiting bidirectional parameter properties, these kernels provide fundamental building blocks for constructing complex, high-performance vision processing pipelines with minimal computational overhead.

OpenVX Roadmap: Future Direction

As the OpenVX Working Group looks toward the future, OpenVX 2.0 looks to extend this innovation further, addressing evolving industry needs and unlocking new possibilities in processing not only vision data but also radar, ultrasonic, and other sensor data that require hardware acceleration. 

The OpenVX Working Group is actively developing the roadmap for the next generation of the framework. Upcoming developments are expected to address several key areas:

• Expanded Hardware Support: Plans include enhancing compatibility with diverse heterogeneous processors and emerging sensor technologies.
• Framework Optimization: Proposed improvements to the core framework aim to increase efficiency and streamline execution flow.
• Enhanced Kernel Integration: The roadmap includes simplifying the process for implementing both user-defined and target-specific kernels.
• Data Handling Enhancements: Building on existing capabilities, future updates are expected to offer improved methods for embedding and synchronizing supplementary data.
• Ease of Implementation: Efforts are underway to modularize the API and streamline the conformance process to lower the barriers to entry for developers and vendors implementing and adopting OpenVX.
• Broader Workload Support: Explore expanding support for custom workloads on heterogeneous SoCs to offer greater flexibility in application design.

The OpenVX Working Group continues to focus on creating a framework that is powerful, flexible, and applicable beyond traditional image processing to address the evolving needs of vision processing across multiple industries.

Learn More

Members of the OpenVX Working Group will share updates, use cases, and progress over the coming months at several industry conferences, including:  

• Embedded World - March 12 - Nuremberg, Germany.
  The talk “OpenVX for Automotive: Accelerating Vision Processing Across Diverse Sensors”, will be presented by Raphael Cano, senior engineer at Bosch. In this talk Raphael will present OpenVX’s key features for handling automotive sensor data, including a real-world use case that demonstrates how OpenVX enhances performance and portability in automotive applications while reducing memory footprint. Attendees will gain insights into how OpenVX simplifies the integration of multiple sensor types and improves overall system efficiency in automotive domains. He will present this topic again at Embedded World China, taking place June 12-14.
• OpenVX Town Hall Meeting - March 26 - Online.
  Hosted by Khronos, this event brings together hardware vendors and application developers for an in-depth discussion on the use of OpenVX to enhance performance and portability across a wide range of vision processing applications.  Members of the OpenVX working group will present the latest updates, and everyone from the OpenVX community is welcome to provide feedback and contribute to the discussions. Learn More and Register
• Embedded Vision Summit - May 20-22 - Santa Clara, USA
  Kiriti Gowda, AMD, and chair of the OpenVX Working Group will highlight key use cases for OpenVX, demonstrating how it addresses real-world vision processing challenges. He will also provide attendees with insights into the future direction of OpenVX.
• Embedded World China - June 12-14 - Shanghai, China.
  The talk “OpenVX for Automotive: Accelerating Vision Processing Across Diverse Sensors” will be presented by Raphael Cano, senior engineer at Bosch. This will be an update to the presentation given at Embedded World Germany.

About OpenVX

OpenVX™ is an open, royalty-free standard for cross platform acceleration of computer vision applications. OpenVX enables performance and power-optimized computer vision processing, especially important in embedded and real-time use cases such as face, body and gesture tracking, smart video surveillance, advanced driver assistance systems (ADAS), object and scene reconstruction, augmented reality, visual inspection, robotics and more.

Khronos welcomes feedback from the development community at the OpenVX GitHub.

Learn more at: https://www.khronos.org/openvx.