Hardware Acceleration is Real

Hardware acceleration for video codecs is almost mandatory - vp9 codec is getting a performance boost.

There are three things that keep VP8 in the game when compared to H.264:

1. It was the only video codec in Chrome for WebRTC in the last 5 years, giving it a headstart in deployments
2. H.264 while available in mobile chipsets isn't always accessible for the developer (or works as it should when it is accessible)
3. VP8 and H.264 are rather old now, so software implementations of them are quite decent

With VP9 codec, the main worry was that it will be left behind and not get the love and attention from chipset vendors - leading it to the same fate as VP8 - abysmal, if any, hardware acceleration support. It is probably why Google went to great lengths to make it running on YouTube so soon and is publicizing its stats all the time.

👉 Check out if VP9 codec is suitable for your WebRTC application

This worry is now rather behind us. Recent signs show some serious adoption from the companies that we should really care about:

#1 - ARM and VP9 acceleration

Mobile=ARM

Without checking stats, I'd say that 99% or more of all smartphones sold in the past 5 years are based on ARM.

If and when ARM decides to support a feature directly, that brings said feature very close towards world domination in future smartpones.

Which is somewhat what happened last week - ARM announced its Mali Egil Video Processor with VP9 acceleration.

Here's a deck they shared:

http://www.slideshare.net/PhilHughes14/arm-mali-egil-technical-preview

Being farther away from chipsets than I were 5 years ago, it is hard for me to say if this is an integral part of an ARM processor, but I believe that it isn't. It is an add-on component that takes care of video processing that chipset vendors add next to their ARM core. They can source the design from ARM or other suppliers - or they can develop their own.

Not sure how popular the ARM alternative is for video processing, but they have the advantage of being the first alternative for any chipset vendor (hell - they already source the ARM core itself, so why not bundle?). Which also means every other vendor needs to match up to their feature set - and improve on it.

Now that VP9 encode/decode capabilities are front and center in the ARM Mali Egil, it has become a mandatory checkmark for everyone else as well.

#2 - Intel on the video codec

If ARM is the king of mobile, then Intel rules the desktop.

As with ARM, I haven't been following up on Intel CPU acceleration lately. And as with ARM, it was Fippo who got my attention with this link here: the new Intel Media SDK.

For those who don't know, Intel is providing several interesting software packages that make direct use of its chipset capabilities. Especially when it comes to optimizing different types of workloads. The Intel IPP and Media SDKs handle media related processing, and are quite popular by low level developers who need access to such facilities.

From the release page itself:

With this release we are happy to announce new full hardware accelerated support for HEVC and VP9.

• So... HEVC (=H.265) has encode and decode while VP9 only has decode support.
• Probably because HEVC has been in the works for a lot longer than VP9, but there's hope still.

#3 - Alliance of Open Media

The Alliance of Open Media. I've published a recent update on the alliance.

Intel was there from the start. The recent additions include ARM, AMD and NVIDIA. Sadly, Qualcomm isn't part of the alliance.

I am sure additional chipset vendors will be joining in the coming months - there seems to be a ramp up in memberships there.

The alliance is working on AV1, a video codec that is planned to inherit VP9 and have wide industry acceptance and adoption. We are already seeing discussions around HEVC vs AV1 in WebRTC.

While the alliance is about what comes after VP9, it is easy to see how these vendors may sway to using VP9 in the interim.

The Future - VP9 codec and beyond

The future is most definitely one of royalty free video codecs. We've got there with voice, now that we have OPUS (though Speex and SILK were there before to pave the way). We will get there with video as well.

Coding technologies need to be accessible and available to everyone - freely - if we are to achieve Benedict Evans' latest claims: Video is the new HTML. But for that, I'll need another post.

FAQ on video codec acceleration

More about VP9 codec and WebRTC.