Hardware acceleration for video codecs is almost mandatory.
- It was the only video codec in Chrome for WebRTC in the last 5 years, giving it a headstart in deployments
- H.264 while available in mobile chipsets isn’t always accessible for the developer (or works as it should when it is accessible)
- VP8 and H.264 are rather old now, so software implementations of them are quite decent
With VP9, the main worry was that it will be left behind and not get the love and attention from chipset vendors – leading it to the same fate as VP8 – abysmal, if any, hardware acceleration support. It is probably why Google went to great lengths to make it running on YouTube so soon and is publicizing its stats all the time.
This worry is now rather behind us. Recent signs show some serious adoption from the companies that we should really care about:
#1 – ARM
Without checking stats, I’d say that 99% or more of all smartphones sold in the past 5 years are based on ARM.
If and when ARM decides to support a feature directly, that brings said feature very close towards world domination in future smartpones.
Which is somewhat what happened last week – ARM announced its Mali Egil Video Processor with VP9 acceleration.
Here’s a deck they shared:
Being farther away from chipsets than I were 5 years ago, it is hard for me to say if this is an integral part of an ARM processor, but I believe that it isn’t. It is an add-on component that takes care of video processing that chipset vendors add next to their ARM core. They can source the design from ARM or other suppliers – or they can develop their own.
Not sure how popular the ARM alternative is for video processing, but they have the advantage of being the first alternative for any chipset vendor (hell – they already source the ARM core itself, so why not bundle?). Which also means every other vendor needs to match up to their feature set – and improve on it.
Now that VP9 encode/decode capabilities are front and center in the ARM Mali Egil, it has become a mandatory checkmark for everyone else as well.
#2 – Intel
If ARM is the king of mobile, then Intel rules the desktop.
As with ARM, I haven’t been following up on Intel CPU acceleration lately. And as with ARM, it was Fippo who got my attention with this link here: the new Intel Media SDK.
For those who don’t know, Intel is providing several interesting software packages that make direct use of its chipset capabilities. Especially when it comes to optimizing different types of workloads. The Intel IPP and Media SDKs handle media related processing, and are quite popular by low level developers who need access to such facilities.
From the release page itself:
With this release we are happy to announce new full hardware accelerated support for HEVC and VP9.
- HEVC Main 10 (10-bit) encoder and decoder support
- VP9 8-bit and 10-bit decoder support
So… HEVC (=H.265) has encode and decode while VP9 only has decode support.
Probably because HEVC has been in the works for a lot longer than VP9, but there’s hope still.
#3 – Alliance of Open Media
The Alliance of Open Media. I’ve published a recent update on the alliance.
Intel was there from the start. The recent additions include ARM, AMD and NVIDIA.
I am sure additional chipset vendors will be joining in the coming months – there seems to be a ramp up in memerships there, with Ateme and Adobe added to their logos just last week.
While the alliance is about what comes after VP9, it is easy to see how these vendors may sway to using VP9 in the interim.
The future is most definitely one of royalty free video codecs. We’ve got there with voice, now that we have OPUS (though Speex and SILK were there before to pave the way). We will get there with video as well.
Coding technologies need to be accessible and available to everyone – freely – if we are to achieve Benedict Evans’ latest claims: Video is the new HTML. But for that, I’ll need another post.
So… which of these video codecs should you use in your application? Here’s a free mini video course to help you decide.