Software Tips & Tutorials

What Are 3 Levels of Hardware Acceleration?

An overview of 3 different hardware acceleration processes

Updated by on

Questions
• What are the 3 levels of hardware acceleration?
• Which level makes best use of hardware acceleration?

Answers
Level-1: Hardware encoder
Level-2: Hardware decoder + hardware encoder
Level-3: Hardware decoder + hardware process + hardware encoder

Based on the complete loop of video transcoding, hardware acceleration can be divided into three levels as mentioned above, each of which delivers different transcoding speed. Theoretically, only the Level-3 can go all out to accelerate video encoding decoding by leaps and bounds, for all of the three main steps [1] can make use of hardware. It used to be an-hour trip to decode and encode videos but can be reached by an 8-minute hop now after turning on Level-3 hardware acceleration.

WinX DVD Ripper Platinum v8.8.0

Much faster than software that only supports hardware encoder or decoder, WinX DVD Ripper Platinum, the only Level-3 hardware accelerator, makes best use of Intel QSV and Nvidia NVENC/NVDEC/CUDA to encode, decode and process videos with best balance in high quality and fast ripping speed!

Level 1 - Hardware Encoder

X264 encoder was the most preferred to encode videos when 1080p/720p HD reigns, which however can't deal with 4K UHD videos easily, despite exhausting an 8-core processor. Nowadays 4K pervades our lives, the encoding of which is much more resource-intensive and time-consuming than ever before, making hardware encoder, a dedicated graphics unit good at parallel computing, a necessity. Nearly 99% of DVD/video converter software claiming to be hardware accelerated in the market is just only able to pass raw data (or processed data) to hardware encoder for compressing, known as the Level-1 hardware encoding. Surely, this is one of the three key steps that can affect overall video transcoding speed, and can be speeded up by utilizing Intel QSV, Nvidia CUDA/NVENC, etc.

1. Software Encoder vs. Hardware Encoder
According to our test on converting a 4K (3840x2160) 30 fps video from H.264 to HEVC without resizing image, X265 encoder transcodes videos at 2.9-5.5, a snail's pace compared to NVENC's speed of 25-33. Clearly, hardware encoder wins.

  Decoder Encoder Speed on i7-4770K 3.5GHz Speed on i8-8700K 3.7GHz
No hardware acceleration Software X265 (libx265) 2.9 fps 5.5 fps
Level-1 Software Nvidia NVENC 25 fps 33 fps

2. What are the file formats supported by hardware encoder?
Intel QSV: MPEG-2, H.264, VP8, HEVC (partly), JPEG (partly)
Nvidia NVENC: H.264, H.265, Lossless

3. How does hardware encoder work?
-> Firstly initialize and configure hardware (set the video format, profile or level you want)
-> Software will pass the video stream to Graphics Driver via API/SDK.
-> Then the QSV or NVENC driver may work alone or together with other drivers if necessary to get ready.
-> Next, firmware and hardware are booted to start video encoding.
-> At last, the encoded stream of bits will be passed back to software for packing in a container.
Does hardware acceleration lower video quality? >>

Hardware encoder workflow

Level 2 - Hardware Decoder + Hardware Encoder

Generally speaking, all videos that you can download online or extract from DVDs are encoded files (probably encrypted). Take MP4 H.264 as an example. H.264 is the codec that your video stream is encoded in while MP4 refers to a media container to integrate videos, audios, subtitles, etc.[2] To playback, edit or re-encode a video file, you have to decode it into raw data firstly, which can be accelerated by hardware as well. Due to the outstanding computing power of hardware, it is easy to come to a conclusion that hardware decoder + hardware encoder is faster than software decoder + hardware encoder.

1. Hardware Acceleration: Level-1 vs. Level-2
This is the same test of converting a 4K (3840x2160) 30 fps video from H.264 to HEVC without resizing image. This time we use the combination of hardware decoder + hardware encoder. While the Level-1 runs at maximally 25 fps or 33 fps, the Level-2 hardware acceleration reachs up to 43 fps.

  Decoder Encoder Speed on i7-4770K 3.5GHz Speed on i8-8700K 3.7GHz
Level-2 Intel Quick Sync Nvidia NVENC 37 fps 43 fps
CUVID Nvidia NVENC 37 fps 43 fps
Level-1 Software Nvidia NVENC 25 fps 33 fps
No hardware acceleration Software X265 (libx265) 2.9 fps 5.5 fps

Hardware encoder workflow

2. What are the file formats supported by hardware decoder?
Intel QSV: MPEG-2, H.264, VP8, HEVC, JPEG
Nvidia NVDEC: MPEG-2, VC1, VP8, VP9, H.264, H.265, Lossless

3. Do we have to decode video into raw data for processing?
Yes. There are so many types of video encoding methods, each of which adopts different data compression algorithm and formats. Besides, there is strong correlation in the data itself. So if we don't decode video to raw data, there will be a lot of algorithm inventing work to be done.

4. How does hardware decoder work?
-> Firstly, software decouple video data streams from audios and other meta-data through a DEMux.
-> If you are dealing with an encrypted DVD/video file, there will be a CSS removing process. Note that decoding is not the same thing as decrypting.
-> The decrypted video stream will then be passed on to Graphics Driver via NVDEC or QSV API/SDK.
-> Next, firmware and hardware are booted to decode video back into raw data.

Hardware decoder workflow

Level 3 - Hardware Decoder + Hardware Process + Hardware Encoder

Level-3 hardware acceleration actually means that all of the three main parts of video transcoding can be assisted by hardware, not just decoding or encoding, but decoding + processing + encoding. Hardware processing happens after video decoding but before encoding. It plays a big role especially when you need to change image size, change aspect ratio, rotate videos, add subtitles, adjust bit rate, etc. If you have ever used video editors like Adobe Premier Pro, you would know how much fretful it is to suffer through processing 4K videos without turning on hardware acceleration. CPU usage increases steeply to 100%, software crashes and even your computer freezes.

Hardware Acceleration: Level-1 vs. Level-2 vs. Level-3
Here is the final test of speed comparison using 3 levels of hardware acceleration. We try to convert a 4K H.264 (3840x2160) 30 fps video to HD H.264 (1920x1080) on i7-4770K 3.5GHz. The Level-3 hardware acceleration amazes us with up to 133 fps.

  Decoder Encoder Hardware Processing Speed
Level-3 Intel Quick Sync Intel Quick Sync Yes 133 fps
Level-2 Intel Quick Sync Intel Quick Sync No 89 fps
Level-1 Software Nvidia NVENC No 69 fps
No hardware acceleration Software X264 (libx264) No 23 fps

Hardware encoder workflow

More Information about Hardware Acceleration

1. Hardware acceleration technology developed by different companies can be used together to maximize video transcoding speed, which however is only possible with a Level-3 hardware accelerated software. For instance, you can choose to
• Decode video with Intel QSV + do the processing via Nvidia GPU memory + encode through Nvidia NVENC
• Or choose Nvidia NVDEC decoder + Intel QSV encoder. Note that it is not permitted to use two different encoders or decoders at the same time.
• Nvidia NVDEC decoder + QSV decoder (X) / Nvidia NVENC encoder + QSV encoder (X)

External Resources

[Wiki] QSV Encoding Decoding: Support for Quick Sync hardware accelerated decoding of H.264, MPEG-2, and VC-1 video is widely available. One common way to gain access to the technology on Microsoft Windows is by use of the free ffdshow filter. Some other free software like VLC media player support Quick Sync as well.
NVIDIA VIDEO CODEC SDK: NVIDIA GPUs contain one or more hardware-based decoder and encoder(s). The SDK consists of two hardware acceleration interfaces: NVENCODE API for video encode acceleration and NVDECODE API for video decode acceleration.

[1] Three main steps to convert DVD/video: Source media file -> Decode file -> Process -> Re-encode file -> Target video
[2] The relationship between video container and video codec:

Decode MP4 H.264 to raw data

Please feel free to share this tutorial if you think it is helpful!

Search more video audio solutions and guides:

Note: The names of other companies, products and services are the property of their respective owners. Any logo, trademark and image relating to other companies that may be contained wherein are applied in the internet and are not used for commercial use. Digiarty's Software is not developed by or affiliated with such companies.