How your WebRTC optimizations are costing you money while killing your business

Optimize your WebRTC applications for better performance. Discover effective tips for superior connectivity and user experience.

WebRTC is a balancing act of resources and requirements. This makes optimizing WebRTC applications a tricky business. One where being over enthusiastic with your optimization can lead to the opposite results.

I wanted to share a few of the examples I bump into over and over again – so you don’t end up making similar mistakes.

Deploying TURN servers in all possible regions

When you deploy TURN servers everywhere it is going to cost you money. Having servers stand there in the sun and rot just because someone at some point might be using them is a waste of money. I’d also venture and dare say that if they aren’t used – then they might not be really working properly and you wouldn’t have any real way of knowing that (besides placing monitoring, which in itself adds more to your cost and complexity).

This isn’t to say don’t put TURN servers in a lot of regions. It is just to say that you should decide in which regions to deploy your TURN servers.

Where would I put them? Not sure, but here’s a suggestion to start from:

• Collect the regions your users come from
• Place that on a histogram graph, showing how many users/connections come from which region
• Make sure that for 90% of your users, you have a TURN server in their region

Not happy with this one? Here’s another suggestion, quite different in its direction:

• Measure the RTT for your users
• Filter out and only look at the users connecting via your TURN servers (the rest don’t matter for this optimization)
• Sort the users based on their RTT – the higher the RTT – the more interested you should be about optimizing their value
• Do you notice any specific user regions that show off higher RTT values? That’s where you need to add TURN servers if you don’t have any, or figure out if the data centers you use in that region are good enough (they probably aren’t)

Cramming too many TURN servers to the iceServers configuration

Lets call this what it really is – a software bug on your part.

How do I know? Because Firefox will complain to the console that you have too many iceServers if you pass its WebRTC peer connection more than 3 or 4 servers…

Why is that?

Each TURN or STUN server you add to the list means that your WebRTC client will need to gather more ICE candidates. And after collecting these candidates, it will need to conduct more ICE connectivity checks because there are more candidates.

The end result? More messages going over the network, with more resources used on this thing, which is practically useless.

This isn’t the worst of it though…

Let’s say you have TURN servers in 10 regions. This means 30 iceServers to add – UDP, TCP and TLS for each region.

What happens if a user is kinda “in-between” regions? He may end up oscillating between them, replacing the TURN server he is connected through every few seconds or minutes. And yes. I’ve seen this happen in real life.

You don’t want to let WebRTC decide on the region to use with its internal logic – you want to do that via DNS geolocation.

Read this for some more tips on TURN servers 👉 We TURNed to see a STUNning view of the ICE

Placing WebRTC signaling servers in every region

Just don’t.

It is complex to achieve (you need a distributed database or at the very least a distributed in memory data grid to store the state of all active users).

And the gain? Too small.

Users might end up sending their signals 100ms or more faster, but it won’t affect actual media quality.

So just don’t.

Your deployment is likely not large enough for this one to be worth it.

Sending video at HD resolutions no matter what

HD video is great. It gives high quality video. While at it, one can go 4K at 60fps. Or higher still.

The bigger the better. Right?

Not so fast.

Sending higher video resolutions and frame rates theoretically improves media quality.

But it comes at a cost of CPU and network resources. You don’t have abundance in these two, so whenever you decide to go higher in your resolution, you need to make sure it is worth it.

Things I ask myself when assisting clients in this area:

• What’s the resolution and framerate of the camera being sent?
• What’s the resolution and framerate of the video being sent (from that camera)?
• On the receiving end, what’s the resolution of the window the video is going to be displayed on?
• How important is the content of that video to that specific interaction?

My view on this?

• Do the minimum that will get you good enough results
• The more you protect and care for CPU and network resources, the more stabilized the system is going to be. From there, you can experiment with increasing quality and sacrificing more CPU and network resources for it

Check out these two resources while at it:

👉 Tweaking WebRTC video quality: unpacking bitrate, resolution and frame rates

👉 8 ways to optimize WebRTC performance

Using simulcast (or SVC) for all of your calls

Simulcast and SVC are great technologies. That said, they have their place and uses – they shouldn’t be used at all times.

Why?

• Simulcast and SVC in general take up more bitrate than “normal” video compression techniques. Especially simulcast
• They both also take more CPU
• Oh, and SVC? It might not be supported by a hardware implementation of an encoder, which means you may end up going for a software codec

I care less about the hardware angle – I am more into taking up more CPU and bitrate. If at the end of the day, they take up more resources, I want to see some value out of it. But in 1:1 calls, there is usually less value (none with Simulcast, some with SVC due to improved resiliency to packet losses – but again, not something I’d go after for most services).

So no. don’t use these technologies all the time.

In general, don’t use them in 1:1 calls, which are likely the majority of the calls in your WebRTC application.

Aiming to use video hardware acceleration wherever possible

Hardware acceleration is great for video coding. Video compression consumes CPU resources and on mobile devices this also translates to battery life and device heat. When using hardware acceleration for video coding, you suffer less from these because they don’t happen anymore on the CPU – there’s a dedicated DSP chip that takes care of it for you.

Here’s the thing though – you shouldn’t always strive to use video hardware acceleration in WebRTC.

Why? Because of these “minor” inconveniences:

• Not all devices have video hardware acceleration for all video codecs you may need
• Devices that have hardware acceleration might not have it implemented properly or satisfactorily. Here are some examples from Chrome related WebRTC bugs:
  • Crashes on encoding video at very low bitrates
  • Poor quality on low resolutions
  • Inability to encode or decode specific streams from other devices
  • No support for SVC on certain encoders
  • Inability to run multiple encoders or decoders concurrently
  • Poor optimization for interactive video use cases

What should you do then?

• Figure out if hardware acceleration of video codecs is for you
• Assume it will require more QA resources and a lot more devices in your lab
• Aim for whitelisting devices for hardware acceleration use (versus blacklisting devices once your users bump into them)
• Err on the side of being cautious on this one

Opting for AV1 video codec for all calls

AV1 is the best video codec that WebRTC has to offer. It has the latest coding tools and the best compression rates. For a given bitrate, it is likely to produce the highest quality out of all other alternatives.

Why not always use it then? Because using AV1 usually takes up more CPU than the alternatives, so older devices can’t really support it at the resolutions and bitrates you may need it

If we end up choking the CPU to make use of AV1, the result is going to be poor media quality – the machine will be starved and will start throttling – missing packets and frames, heating up and freezing voice and video.

Deciding to use AV1?

• Prepare to use multiple video codecs in your application – dynamically
• Figure out when to use and when to skip the use of AV1

WebRTC optimizations the right way

Optimizing WebRTC applications is more than just a simple set of rules to follow. It ends up depending on your use case and your implementation.

It is a balancing act between the level of media quality you want to achieve (the highest) versus the available hardware and network resources you have at your disposal (an unknown that changes dynamically from one session to another and also within a session).

If you need help with this, know that this is what I do in many of my consulting projects. Just contact me and we can check together if I can help you out.