WebRTC iOS 11 Support. Are We There Yet?

12/06/2017

Last week WWDC was a happy occasion for those who deal with WebRTC. For the first time, we got the official word – and code – of WebRTC on Apple devices – WebRTC iOS and WebRTC Safari support is finally here.

I spent the time since then talking to a couple of developers and managers who either tinkered with Safari 11 already or have to make plans in their products for this development.

I came out a bit undecided about this whole thing. Here are where things stand.

Need to know where WebRTC is available? Download this free WebRTC Device Cheat Sheet.

Apple’s Coverage

The WebKit site has its own announcement – as close as we’ll ever get to an official announcement from Apple with any detail it seems.

What I find interesting with this one (go read it – I’ll wait here):

  • Google isn’t mentioned (god forbid). But their “LibWebRTC open source framework” is. Laughed my pants off of this one. The lengths companies would go to not mention one another is ridiculous
  • WebRTC in WebKit isn’t mentioned. Two years ago I opined it wouldn’t move the needle. And it seems like it didn’t – or at least not enough for Apple to mention it in this WebKit announcement
  • TokBox and BlueJeans are mentioned as early beta. TokBox I understand. Can’t miss one of the dominant players in this space. But BlueJeans? That was a surprise to me

Previous Coverage

First things first, here are some posts that were published already about Apple’s release of WebRTC Safari (in alphabetical order):

I’ve ignored a few generic posts that just indicated WebRTC is out there.

Most relevant Twitter thread on this?

Who’s Missing?

Well… the general media.

I haven’t really seen the keynote. I find the Apple keynotes irrelevant to my needs, so prefer not to waste my time on them. My understanding is that WebRTC took place there as a word in a slide. And that’s HUGE! Especially considering the fact that none of the general technology media outlets cared to mention it in any way.

Not even the general developers media.

What was mentioned instead were things like the iOS file manager. That seems a lot more transformative.

This isn’t a rant about general media and how they miss WebRTC. It is a rant about how WebRTC as a technology has not caught the attention at all, outside a little circle of people.

Is it because communications is no longer interesting?

At a large developers event here in Israel on that same week, there were 6 different tracks: Architecture, Full stack, Philosophy, Big Data, IOT, Security. No comms.

Not sure what the answer…

On one hand, WebRTC is transformative and important. On the other hand, it is mostly ignored.

WebRTC Safari Support

So what did get included in the WebRTC Safari implementation by Apple?

  • The basics. GetUserMedia and PeerConnection
  • One-on-one voice and video calls seem to work well across browsers AND devices (including things like Safari to Edge)
  • Opus is there for audio
  • H.264 only for video. There is no VP8 or VP9. More on that later
  • The code supports Plan B, if you are into multiparty media routing
  • Data Channel is supported, but too buggy to be used apparently
  • No screen sharing. Yet
  • This is both for the desktop and mobile:
    • WebRTC Safari support is there for the macOS in the new version of safari
    • WebRTC iOS support is there for iOS 11 – via the Safari web browser, and maybe(?) inside WebViews

This is mostly expected. See below why.

WebKit, Blink and WebRTC

A long time ago, in a galaxy far far away. There was a browser rendering engine called WebKit. Everyone used WebKit. Especially Google and Apple. Google used WebKit for Chrome. And Apple used WebKit for Safari.

And then one day, in the middle of the development of WebRTC, Google decided enough is enough. They forked WebKit, renamed it to Blink, removed all excess code baggage and never looked back.

Apple never did care about WebRTC. At least not enough to make this thing happen until last week. I hope this is a move forward and a change of pace in Apple’s adoption of WebRTC.

Here’s what I think Apple did:

  • Seems like they just took the Google code for WebRTC and hammered at it until it fit nicely back into WebKit (ignoring WebRTC in WebKit in the process)
  • How did they modify it? Remove VP8. Add H.264 by hooking it up with the hardware codec in iOS and on the Mac
  • And did the rest of the porting work – connecting devices, etc.
  • Plan B is there because. Well. Google uses Plan B at the moment. And it just stands to reason that the code Apple had was Plan B

WebRTC Safari and Interoperability

When it comes to WebRTC, the question is one of browser interoperability. There aren’t many browser vendors (I am counting four major ones).

The basics seem to work fine. If you run a simple peer-to-peer call between any of the 4 browsers, you’ll get a call going. Voice and video. The lowest common denominator for that seems to be Opus+H.264 due to Safari. Otherwise, Opus+VP8 would also be a possibility.

The challenge starts when what you’re trying to do is multiparty. While H.264 is supported by all browsers, the ability to use simulcast with H.264 isn’t. At the moment, Chrome doesn’t support simulcast with H.264. The current market perception is that multiparty video without simulcast is meh.

Doing group calling?

  • Go for audio only
  • Force everyone to use H.264 if you need Safari (either as a general rule or when the specific session has someone using Safari) – and understand that simulcast won’t be available to you

Now it is going to become a matter of who blinks first: Google by adding H.264 simulcasting to Chrome; or Apple by adding VP8 to Safari.

The Next Video Codec War

Which leads us to the next frontier in the video codec wars.

If you came in late to the game, then know that we had over 3 years of continuous bickering regarding the mandatory video codec in WebRTC. Here’s the last I wrote about the codec wars when the Alliance of Open Media formed some two years ago.

At the time, both VP8 and H.264 were defined as mandatory video codecs in WebRTC. The trajectory was also quite obvious:

After  H.264 and VP8, there was going to be a shift towards VP9 and then a leap towards AV1 – the new video codec currently being defined by the Alliance of Open Media.

Who isn’t in the alliance? Apple.

And it seems that Apple decided to bank on HEVC (H.265) – the successor of H.264. This is true for both iOS and macOS. This is done through hardware acceleration for both the encoder and the decoder, with the purpose of reducing storage requirements for video files and reducing bandwidth requirements for streaming.

But it goes to show where Apple will be going with WebRTC next. They will be adding HEVC support to it, ignoring VP9 altogether.

There’s a hefty cost in taking that route:

  • H.264 is simple yet expensive – when you use it, you need to pay up royalties to a single patent pool – MPEG-LA
  • HEVC is complex AND expensive – when you use it, you need to pay up royalties for MULTIPLE patent pools – MPEG-LA, HEVC Advance, Velos Media. Wondering which one you’ll need to pay for and how much? Me too

Which is why I think Apple is taking this route in the first place.

Apple has its own patents in HEVC, and is part of the MPEG-LA patent pool.

And it knows royalties is going to be complex and expensive. Which makes this a barrier for other vendors. Especially those who aren’t as vertically integrated – who needs to pay royalties here? The chipset vendor? The OS vendor? The handset manufacturer?

By embedding HEVC in iOS 11 and macOS High Sierra, Apple is doing what it does best – differentiates itself from anyone else based on quality:

  • It has hardware acceleration for HEVC. Both encoding and decoding
  • It starts using it today on its devices, and “magically” media quality improves and/or storage/network requirements decrease

Google, and Android by extension, won’t be adding HEVC. Google has taken the VP9 route. But in VP9 most solutions are software based – especially for the encoder. Which means that using VP9 eats up CPU. Results look just as good as HEVC, but the cost is higher on CPU and memory needs. Which means an “inferior” solution.

Which is exactly what Apple wants and needs. It just doesn’t care if interoperability with other devices requires lowering quality as the perception of who’s at fault will fall flat on Android and Google and not on Apple.

So…

Don’t expect to see VP9 or AV1 anytime soon in Apple’s roadmap. Not for WebRTC and not for anything else.

Dan Rayburn covers the streaming side (non-WebRTC) of this HEVC decision quite nicely on StreamingMedia.

Oh, and while at it, Jeremy Noring wrote a well thought rant about this lack of support for VP8 and VP9. His suggestion? Go vote for bug #173141 on WebKit. It probably won’t help, but it will make you feel a bit better 🙂

The only way I see this being resolved? If Google retracts their support for H.264 and just blatantly removes it from Chrome until Apple adds VP8 to Safari. I’ll be happy to see this happening.

FaceTime , Multiparty and WebRTC

Apple has FaceTime.

And FaceTime probably doesn’t use WebRTC. I am not sure if it ever will.

When Google came out with WebRTC, they had the Hangouts (now Meet) team about a year behind in its adoption of WebRTC as their underlying technology – but the intent and later execution was there.

When Microsoft came out with WebRTC, Skype didn’t support WebRTC. But they did launch Skype for Linux which is built somewhat on top of WebRTC, and Skype for Web which is taking the same route. Call it ORTC instead of WebRTC – they are one and the same as they are set to merge anyway.

Apple? Will they place FaceTime on top of WebRTC? I see no incentive there whatsoever.

Can Cisco change this? Rowan Trollope broke the news titled “Cisco and Apple Announce New Features” that WebEx and Cisco Spark now work on Safari – no download needed. I’ll translate it for you by adding the missing keyword here – WebRTC. Cisco is using WebRTC to do that. And since their stack is built atop H.264, they got that working on Safari.

Cisco and Apple here is interesting. While Cisco mentions this in the headline as if these new features were done jointly, Apple isn’t really acknowledging it. There’s no quote from an Apple representative, and at the same time, Cisco isn’t mentioned in the WebKit announcement – TokBox and BlueJeans are.

One wonders.

Back to FaceTime. Which is a 1:1 video chat service.

And the fact that many look into group video chat and other multiparty video configurations.

Will Apple care enough to support it well in WebRTC? Will it move from Plan B to Unified Plan? Will it care about simulcast? Will it invest in SVC? Will it listen and work with Cisco on its multiparty needs for the benefit of us all?

Older Devices

Apple will not be upgrading iPhone 5 devices to iOS 11. That’s a 5 years old device.

In many requirement documents I see a request to support iPhone 4.

Will this bump the general audience out there to focus on iPhone 6 and upwards, seeing what Apple is doing as well? Will this mean vendors will need to port WebRTC on their own to support older devices?

Time will tell, but I think that switching to iPhone 6 and above and focusing there makes sense.

Chrome/Firefox support on iOS

Here’s a question for you.

If you use Chrome or Firefox on iOS. And open a URL to a site using WebRTC. Will that work?

Here’s the catch.

The reason there was no real browser for iOS that supported iOS until today? Apple mandates WebKit as the rendering engine on any browser app that comes to its AppStore.

Now that WebKit is getting official WebRTC support – will Chrome and Firefox add WebRTC support to their iOS browsers?

And if they do – you’ll be getting the Apple restrictions. I can just see the WebRTC developer teams at Google and Mozilla cringing at this thought.

This can get really interesting if and when Apple decides to add HEVC support to WebRTC in its WebKit implementation of iOS. You’ll get Chrome on iOS with H.264 and HEVC and Chrome everywhere else with H.264, VP8 and VP9.

Fun times.

What Should Developers Do?

Here’s what you’ve been waiting for. The question I’ve been asked multiple times already:

Do I need to build an app? Should I wait?

The suggest at the moment is wait. Question is for what and until when exactly.

If you are looking for a closed app and planning on developing native, then go with whatever worked for you until today. This news item just isn’t for you.

If you are looking for browser support on iOS, then go with Safari and plan on enabling H.264 video codec in your service. Don’t wait up for VP8.

If you want something that will be cross platform app development using HTML5, then wait. Webview WebRTC support in iOS is still an unknown. If it gets there in the coming months then waiting a few more minutes probably won’t make a real difference for you anyway.

My Updated Cheat Sheet

As it is, this change with Safari, iOS and macOS required some necessary updates in my resources.

First to update is the WebRTC Device Cheat sheet. You can find the updated one in the same download page.

One last thing –

Join my and Philipp Hancke for Virtual Coffee

I planned for a different Virtual Coffee session this month. One about developer tools. It got bumped into July.

The one in June will cover the iOS announcement and its ramifications. My guest this time will be Philipp Hancke.

Need to know where WebRTC is available? Download this free WebRTC Device Cheat Sheet.

Responses

Lorenzo Miniero says:
June 12, 2017

A really interesting read, thanks for sharing these considerations!

While I’m really happy that Safari eventually decided to leave the shadows and embrace the WebRTC world, I have to admit I find the (apparently purposely, by reading Jeremy Noring’s blog post) lack of VP8 disconcerting, if not disappointing. I’ve been a very vocal supporter of VP8 (and openness) during the infamous codec wars at the IETF, and was against the Cisco’s openh264 hack as well for a number of reasons. When that managed to get H.264 on par with VP8, and both MTI for WebRTC, I was pretty sure this was going to be exploited at the expense of VP8, sooner or later, with big companies only implementing H.264 (having interests in that) to basically force you to either do that or be with no video at all. This move by Apple seems to be a step in that direction, and I don’t like it one bit.

That said, it may indeed be too early to tell. I thought the same when WebRTC support in Edge saw the light, with only H.264 in it (and a custom, non-interoperable, flavour of H.264, by the way). To my surprise, though, Microsoft did eventually make the effort to integrate VP8 in Edge. Hopefully the same will happen with Safari as well…

Reply
    Tsahi Levent-Levi says:
    June 12, 2017

    Lorenzo,

    I think Apple and Microsoft here are different. Edge didn’t catch significant market share to make a difference. I’d even say that you have more developers who already tried out WebRTC on Safari that there are developers who tried WebRTC on Edge (and Safari has been with us less than a week).

    That in itself means that there’s less incentive for Apple to add VP8 to Safari, while there was huge incentive for Microsoft to add VP8 to Edge.

    Add to that the fact that Microsoft has joined AOMedia on day one and Apple still isn’t there but they do have HEVC on their devices, and you get a rather grim picture.

    Reply
      Lorenzo Miniero says:
      June 12, 2017

      I hadn’t thought about that… grim picture indeed!

      Reply
      Philipp Hancke says:
      June 13, 2017

      Edge with interoperable video codecs is generally available since the creators update in April and unsurprisingly the numbers I see for minutes-spent greatly outnumber the ones for Safari Tech Preview.

      This will be an interesting game in autumn but so far Edge is way ahead.

      Reply
        Tsahi Levent-Levi says:
        June 13, 2017

        The use I am referring to is use by developers and not end users.
        My guess is that more developers played with WebRTC on Safari than WebRTC on Edge.

        Reply
          Philipp Hancke says:
          June 13, 2017

          I have opinions about the native WebRTC in Edge 🙂

          Reply
Igor Khomenko says:
June 12, 2017

Tsahi, I really like this article

so you mean for Apple there is no sense to support something else but H264 because they have own % from royalties?

The following chain came to my mind:
1) Apple support H264 only
2) because of this fact, other vendors have to support H264 if we are talking about interoperability
3) because of this fact, a lot of people use H264 -> Apple receive its %

so money first? then bright and comfortable technology future?

Reply
Erik Lagerway says:
June 12, 2017

Tsahi, great read, as always, thanks for the mention, , yes, it’s been a long winter 😉

Reply
Art Matsak says:
June 13, 2017

Unfortunately, even the peer-to-peer interop case isn’t covered. H.264 is far from guaranteed on Android where Google’s WebRTC implementation lacks a software encoder. So even a simple Android to Safari call could fail.

Reply
Philippe N says:
June 13, 2017

About the WebRTC in WebKit initiative, it is true that it isn’t mentioned in the announcement but Apple did leverage the huge amount of the code that was upstreamed back in 2016 thanks to this initiative. Then Apple engineers imported the libwebrtc code into WebKit and built a custom PeerConnection backend for it.

Reply
RealTimeWeekly | RealTimeWeekly #184 says:
June 19, 2017

[…] WebRTC iOS 11 Support. Are We There Yet? […]

Reply
Stephen Dame says:
June 28, 2017

Do you have a link to the crowdcast recording on IOS 11 Safari. I attended last week lots of good details, wanted to share with my team.

thanks for your hard work in helping us understand this topic.

Reply
Chuck and Tim | TalkingPointz says:
July 6, 2017

[…] the importance of easily starting meetings — perhaps that’s why Apple is finally embracing WebRTC in Safari. He implied that Apple and Cisco together can improve security. That was big of him considering […]

Reply
Laduram says:
July 27, 2017

Yes its working, We tested in ios 11 safari and MacOS 10.13 safari. Also its working with all cross browser.

Reply
Danny says:
September 8, 2017

Hi, i have a major question. Will iOS 11 actually support getusermedia on the safari browser. The reason I’m asking is because I have iOS 11 beta 10 and the getusermedia demo still doesn’t work. I. Getting a just black screen. Any knowledge as to what may Happen would be a great help to me. I have been waiting for getusermedia to come to iPhone for 6 years

Reply
dan says:
September 13, 2017

Hi, i have a major question. Will iOS 11 actually support getusermedia on the safari browser. The reason I’m asking is because I have iOS 11 now supposedly the final release and the getusermedia demo still doesn’t work. I. Getting a just black screen. Any knowledge as to what may Happen would be a great help to me. I have been waiting for getusermedia to come to iPhone for 6 years

Reply
    Tsahi Levent-Levi says:
    September 13, 2017

    Dan,

    I don’t have an iOS device myself, so can’t really say. From what I hear (and if you look at Laduram’s comment above), it should work rather well.

    Reply
      Danny Black says:
      September 19, 2017

      The getusermedia does not work on the iPhone 7 Plus. All you get is a black screen. There is a bug. Apple needs to fix this!!

      Reply
        Andrew says:
        July 21, 2018

        You likely need to add the playsinline attribute on the tag. That’s needed for iOS.

        Reply
What is AstroPrint doing, exactly? | Fabricating Dreams Messily says:
January 6, 2018

[…] Daniel Arroyo, 3DaGoGo’s co-founder, CTO, and tireless community rep, has been quick to honestly answer my questions about incomplete or faulty features. He informed me that access to Thingiverse collections was limited by the Thingiverse API, but will become available in future releases. Cool. He told other forum members that the iOS video issue was “a limitation of the iOS system.” He’s right, and the problem may soon be solved. […]

Reply
Temasys.io - Embedded Real-time Communication says:
January 16, 2018

[…] else has to say about this milestone in the journey of WebRTC. Here are some posts that you might like, though I’m sure there are others worth reading, that I’ve […]

Reply
Devin says:
April 11, 2018

Thanks for writing this up. Looks like they’re still working out which codec to use for standard. At the meantime, does this means it will be necessary to convert codec to make it work on iOS – either in Safari or native app? Or is there a codec that we can use that would work everywhere including iOS without any need to convert? Thanks.

Reply
Victor says:
March 25, 2019

What on this post changes from the announcement of VP8 https://webkit.org/blog/8672/on-the-road-to-webrtc-1-0-including-vp8/ ?

Reply

Comment