The Vulkan® Working Group at The Khronos® Group has delivered a series of video decode and encode extensions since 2022 collectively referred to as "Vulkan Video." These extensions integrate hardware-accelerated stream compression and decompression using widely adopted codecs with the full power of Vulkan, enabling developers to seamlessly combine GPU-powered rendering and compute acceleration with video processing in a single highly efficient runtime. 

Today, with the release of Vulkan 1.3.302, Khronos is proud to announce two new encode extensions. First, the highly anticipated Encode AV1 extension enhances Vulkan Video by adding AV1 encode functionality to complement its existing AV1 decode support. This milestone means that Vulkan Video now provides full decode AND encode acceleration for the H.264, H.265 and AV1 codec standards. Additionally, the new Encode Quantization Map extension introduces advanced encoding features for all supported codecs to Vulkan Video developers for the first time. We are confident these extensions provide the necessary building blocks for your advanced Vulkan Video applications!

Figure 1. Released Vulkan Video extensions and some of the extensions under development

Encode AV1 extension

The new VK_KHR_video_encode_av1 extension naturally builds upon the existing core Vulkan Video extensions as did the previous H.264 and H.265 encode extensions. This new extension introduces APIs needed to query implementation AV1 encode capabilities, specify AV1 sequence header parameters, specify AV1 per-picture parameters, provide AV1 reference management information, and configure rate control for AV1. In addition, a new video std header is introduced specifically for AV1 encoding: vk_video/vulkan_video_codec_av1std_encode.h. Together with the existing vk_video/vulkan_video_codec_av1std.h header released with the Decode AV1 extension, developers have access to the AV1 codec parameters needed for configuring the AV1 encode session and operations.

Details can be found in the proposal document and specification for this extension.

Encode Quantization Map extension

The new VK_KHR_video_encode_quantization_map extension enables applications to provide a “map” with each input picture to influence per-coding-block quantization during encoding. This feature is invaluable for applications that perform external encoder input image analysis or are aware of critical regions of the input image that would benefit from “spending more bits” during encoding to result in overall better encoding quality. Two flavors of maps are supported: delta quantization maps for explicit codec-specific control of the final quantization for each block, and emphasis maps for a codec-independent hint on the relative importance of different image blocks.

Details can be found in the proposal document and specification for this extension.

Figure 2 The Vulkan Video encoding process at a high-level, showing the addition of quantization maps that may be provided during H.264, H.265 or AV1 encoding operations.

Call for Action, Feedback & Support!

The Vulkan Working Group encourages developers to utilize the Vulkan Video extensions to bring new levels of performance and functionality to their video applications. We welcome all developer feedback on the Vulkan issue trackers on GitHub for Encode AV1 and Encode Quantization Map. These issues will be updated to provide links to Vulkan Video-related resources as they become available. We also look forward to hearing about any additional features important for your use cases that should be added to Vulkan Video!

An upcoming release of Vulkan SDK will include updated Vulkan headers and Validation Layer support for the newly released video extensions. In the meantime, you can find the Vulkan headers here.

In addition, all are invited to attend the various presentations about Vulkan Video and other topics at the Vulkanised 2025 event in Cambridge, UK, on February 11-13, 2025.

Vulkan drivers supporting both Encode AV1 and Encode Quantization Map extensions in addition to all released Vulkan Video extensions are already available, including:

• NVIDIA: Windows and Linux beta drivers
• AMD: Windows beta driver
• Intel: Windows driver coming soon

There is strong open-source community adoption of Vulkan Video, including decode and encode support in the GStreamer and FFmpeg multimedia frameworks, as well as the RADV/ANV open-source Vulkan drivers for AMD/Intel GPUs.

NVIDIA’s open-source Vulkan Video sample is also being updated to illustrate the use of the Encode AV1 and the Encode Quantization map extensions.

Victor’s Vulkan Video status page provides an excellent compilation of Vulkan Video-related ecosystem resources.

We also encourage your participation in extending Vulkan Video to support more codecs and features. See khronos.org/members for information about how to join Khronos and participate in the definition of any of our standards.

Thank you for your interest and support of Vulkan Video. We hope you find it effective for your use cases and applications, and we look forward to supporting your needs with more codecs and features!

Industry Support for Vulkan Video 

“Collaborating with industry leaders and end-users to bring new technologies to the masses is at the heart of AMD’s culture. We are proud to continue leading the Vulkan Video group at Khronos to extend the video codecs and related features to developers,” said Andrej Zdravkovic, senior vice president and chief software officer at AMD. “The release of Encode AV1 enables using one of the most popular codecs on AMD RDNA™ architecture-based products and solutions. We are looking forward to supporting end-users with an upcoming release of an AMD Software: Adrenalin Edition™ driver optimized for these features.”

“The AV1 encode extension brings Vulkan support to a state-of-the-art codec, known for its royalty-free licensing model, backed by a industry-wide consortium," said Daniel Almeida, software engineer at Collabora. "That licensing format has driven wide adoption of AV1 in Open Source software, particularly on the web, and the addition of Vulkan Video gives the Open Source community yet another venue to access the underlying hardware accelerators present in all classes of devices.”

“With this AV1 encode extension, Vulkan Video is positioned to become a successful and complete offering as a multi-platform video processing API for a large variety of use cases, both in industrial and consumer products and applications. Igalia is committed to, and involved in, the definition of the Vulkan Video specification as well as the implementation of its extensions and related tools. The addition of Vulkan Video support to GStreamer is a very good example of our contributions to the Vulkan Video ecosystem,” said Samuel Iglesias, coordinator of the GPU driver development team at Igalia.

“The ratification of the Vulkan Video extension for AV1 encode will further improve access to state-of-the-art codec technology, a great news for developers and end-users,” said Vivian Lien, vice president of Client Graphics at Intel. “By building dedicated AV1 hardware capabilities in our products and working with the Vulkan Video Group, we aim to empower developers to build immersive applications across platforms. All Intel® Arc™ Graphics products will fully support the new Vulkan Video extension for AV1 encode with a future software update.”

“As someone involved with AV1 all the way since its very start, I am glad to have seen this extension through to its publishing. Royalty free standards fostered a spirit of innovation and cooperation across multimedia, the same way that Vulkan did for graphics and compute. Furthermore, the quantization map extension provides important feedback to the encoder that will help boost its quality and expand its flexibility in a cross-vendor way,” said Lynne Iribarren, Khronos member and FFmpeg developer.

“The addition of AV1 encoding completes the initial set of codecs I wanted to see supported. This makes Vulkan Video a competitive baseline to challenge the non-standard industry APIs and hopefully provides a great base to implement cross-vendor features moving forward. The addition of one such feature, Encode Quantization mappings is a great example of where the standard will develop into the future,” said Dave Airlie, Mesa developer and Linux Kernel maintainer.

“With the inclusion of AV1 encoding and Encoder Quantization Map support, developers can now further harness the power of NVIDIA GPUs to accelerate their Video workflow using Vulkan Video. NVIDIA strives to work with developers and partners to enhance its video capabilities and further advance the adoption of Vulkan Video. We are proud to continue to contribute and drive innovation with the Vulkan Working Group,” said Bob Pette, vice president of enterprise platforms, NVIDIA. 

"Today we celebrate another major milestone in the evolution of the Vulkan Video APIs. The long-awaited AV1 encode extension is out and the quantization map extension is the first extension to enable advanced video encoding use cases across all supported video codecs," said Daniel Rakos, CEO, RasterGrid. "We are proud that the Vulkan Working Group continues to trust us with delivering the specifications and Validation Layer support for all video coding extensions and looking forward to further expanding our involvement in building the Vulkan Video ecosystem."

“This is a great step forward towards being able to have a single video encode path across a variety of platforms and GPUs,” said Sam Lantinga, Valve Corporation.