AWS is NOT your WebRTC Hosting of Choice

23/01/2014

Amazon Web Services? Keep it off my WebRTC service.

WebRTC and AWS

Here’s the thing:

  1. This is going to be a controversial post
  2. I haven’t got a solid opinion, but I will take a side here, for the fun of argumentation

I have been talking to WebRTC vendors for a long time now. It is a lot of fun and very educating for me. One of the questions where I get the most interesting answers is about the deployment model of these services – they range from fully virtualized-redundant-cloudified-dynamic solution to dedicated servers hosted in select data centers. The whole rainbow of choices is out there.

But here’s the thing – WebRTC is still video communications. And it is still done in real time. As such, it isn’t that fond of latencies and surprises. It has mechanisms to deal with these, but at the end of the day, give it a healthy dose of stability and predictability (with a touch of low latency) and it will give you back in media quality.

Virtualization… how would I say it… isn’t that fond of predictability. It likes keeping its options open. It tries to squeeze whatever fits on the CPU to make sure it increases utilization as a whole.

See the contention here? The misalignment in purpose?

WebRTC has a hard time today with smartphones (and desktop browsers) because they run on operating systems that aren’t real time in nature. It is hard to commit to processing frames at 20 milliseconds intervals and calculating the echo cancellation required if the operating system oscillates between 15 to 30 millisecond intervals (or more if it decided to garbage collect memory somewhere on another application). These same headaches? They exist at a whole different level on the server side – especially if you intend on doing some video processing work there (complex or otherwise).

Thinking of deploying your server?

Striving for the best media quality possible under the circumstances? Steer away from AWS and cloud/virtualize data centers. Go for dedicated ones.

Think otherwise? Let’s hear your voice in the comments below – as I said, I am not sure I am in agreement with myself on this one.

Responses

Anton Rodriguez says:
January 23, 2014

In this point, I am not fully agree with you. AWS is an amazing and useful platform, as other virtualization platforms, for instance, OpenStack. They have a great potential and we are seeing a lot of activity regarding to Software Defining-Networking (SDN) and Network Function Virtualization (NFV) so, this market cannot be separated of WebRTC. Of course, there are a lot of different architectures and requirements, I am not saying virtualization is the silver bullet but it’s a option to be checked. For example, TeleStax is doing great things in AWS with excellent results: http://www.telestax.com/load-testing-smsc-on-ec2/ and we’ve deployed OCWSC in our own environment with great results. As I’ve said before, IMHO virtualization can be a good idea in some scenarios.

Reply
    Tsahi Levent-Levi says:
    January 23, 2014

    SMS is a simple case – it doesn’t require predictability at the millisecond level as voice and video does, so I don’t see this as a case for virtualization in WebRTC media. That said, virtualization definitely have its place for most of the scenarios today (and growing in that area by the day).

    Reply
Tim Panton says:
January 23, 2014

Latency and jitter matter most at the endpoints. If your webRTC media is peer to peer between 2 browsers, you don’t much care if the signalling is delayed slightly in an amazon VM.

Even if the media is going through a gateway on a VM (say being decrypted or something) – you probably don’t mind much as the network latency and jitter on your wifi or 3g last mile will probably exceed anything EC2 is likely to add.

If however you want to mix webRTC media with PSTN originated media, then you may care much more. The PSTN has historically had super accurate clocks (sync’d by GPS these days) and gets all upset by delayed media.

So – It depends on your use case. Middle boxes are probably fine in the cloud, endpoints are risky but possible if you examine the detail correctly.

Reply
Chad Hart says:
January 23, 2014

From a media server perspective, we have been able to make AWS work but its very unreliable if you are using a shared instance – especially the cheapest micro instances. We are finding many optimizations you can do to make it work better.

Some of our customers have had better results with other hosters like Rackspace. The use of shared instances is definitely something that needs to be considered if you care about quality.

Twilio uses AWS and seems to survive fine, although I do not think they do not do any server-side video processing.

Reply
    Tsahi Levent-Levi says:
    January 23, 2014

    It would be interesting to see Chad how Dialogic will work this out.
    One of the best things going for you is the fact that you are pure software. Next step is probably easy cloud deployment for customers and from there, full virtualization.

    Reply
Shubh Agarwal says:
January 23, 2014

Big difference between shared/virtualized instances running AWS own OS.. vs. dedicated server instances (on-demand), also from AWS. We have our own selection of OS and optimized libraries for real time video services. We had to do the hard work to “eliminate” the unpredictability and maximize performance to provide the benefits of cloud and the reliability of telco.

Reply
galeal says:
January 23, 2014

The abstracted point that you make is critical: use case specific deployments and no one size fits all. Glad you wrote it up in nice manner as we gloss over it too often.

My guess is the minority of overall WebRTC deployments will need dedicated compute:

Yes, if you are doing video transcoding and need to keep audio and video streams in sync with minimal frame delays and jitter buffers, especially if you are dealing with lossy networks.

Likely no, if you aren’t doing video, are doing P2P media, or even if you are *just* switching media streams in the cloud…would really need to do some benchmarking and use case analysis to determine if the dedicated compute is necessary.

Reply
    Tsahi Levent-Levi says:
    January 23, 2014

    Well… almost any use case will need TURN. TURN relays media. The less it affects the stream (=jitter) the better it will be to handle. Routing and switching video is also critical because it will usually imply multipoint scenarios where again, there will be a need to sync not only a single session but a couple of them.

    Reply
      Philipp Hancke says:
      January 26, 2014

      I suspect that TURN (shuffling around network packets) has somewhat different requirements here than anything that does mixing.

      I don’t have data to prove it though.

      Reply
silvia pfeiffer says:
January 24, 2014

That’s a pretty broad recommendation. I wouldn’t agree with it. There are different server services involved in setting up a webrtc call. You should be able to safely host the webpages, the signalling server, and a stun server in the clound. Turn server, MCU and any kind of transcoding/real-time manipulation service is likely better on a dedicated machine. Remember that the low latency is required for transport of Audio and video, not so much for anything else. Most calls should be peer-to-peer so the speed of the server should eventually not matter.

Reply
Jean-Philippe Papillon says:
January 25, 2014

I was inclined to be in your camp but it seems that Zoom.us is using AWS for video routing.

Reply
Bernard Aboba says:
January 27, 2014

To test your thesis, I put up a JitMeet site in Azure, with the jitsi-videobridge audio mixer and video conferencing server running in a small VM. I haven’t done extensive load tests, but it seems to work ok for a small number of users. Check it out:
http://meet.azurewebsites.net/

Note that the jitsi-videobridge is not a conventional MCU, but rather a Forwarding Unit (e.g. RTP translator). Since it just forwards packets, it doesn’t add much latency.

Reply
Yash Sharma says:
November 26, 2014

We have also tried to use AWS and a Dedicated server. Dedicated server is better anytime.

Reply

Comment