Recording WebRTC Sessions: client side or server side?

26/09/2016

Recording WebRTC? Definitely server side. But maybe client side.

Recording WebRTC sessions

This article is again taken partially from one of the lessons in my upcoming WebRTC Architecture Course. There, it is given in greater detail, and is recorded.

Recording is obviously not part of what WebRTC does. WebRTC offers the means to send media, but little more (which is just as it should be). If you want to record, you’ll need to take matters into your own hands.

Generally speaking, there are 3 different mechanisms that can be used to record:

  1. Server side recording
  2. Client side recording
  3. Media forwarding

Let’s review them all and see where that leads us.

#1 – Server side recording

This is the technique I usually suggest developers to use. Somehow, it fits best in most cases (though not always).

WebRTC server-side recording

What we do in server-side recording is route our media via a media server instead of directly between the browsrs. This isn’t TURN relay – a TURN relay doesn’t get to “see” what’s inside the packets as they are encrypted end-to-end. What we do is terminate the WebRTC session at the server on both sides of the call – route the media via the server and at the same time send the decoded media to post processing and recording.

What do I mean by post processing?

  • We might want to mix the inputs from all participants and combine it all to a single media file
  • We might want to lower the filesize that we end up storing
  • Change format (and maybe the codecs?), to prepare it for playback in other types of devices and mediums

There are many things that factor in to a recording decision besides just saying “I want to record WebRTC”.

If I had to put pros vs cons for server side media recording in WebRTC, I’d probably get to this kind of a table:

+
No change in client-side requirements Another server in the infrastructure
No assumptions on client-side capabilities or behavior Lots of bandwidth (and processing)
Can fit resulting recording to whatever medium and quality level necessary Now we must route media

#2- Client side recording

In many cases, developers will shy away from server-side recording, trying to solve the world’s problem on the client-side. I guess it is partially because many WebRTC developers tend to be Java Script coders and not full stack developers who know how to run complex backends. After all, putting up a media server comes with its own set of headaches and costs.

So the basics of client-side recording leans towards the following basic flow:

WebRTC client-side recording - part 1

We first record stuff locally – WebRTC allows that.

WebRTC client-side recording - part 2

And then we upload what we recorded to the server. Here’ we don’t really use WebRTC – just pure file upload.

Great on paper, somewhat less in reality. Why? There are a few interesting challenges when you record locally on machine you don’t know or control, via a browser:

  • Do you even know how much available storage do you have to use for the recording? Will it be enough for that full hour session you planned to do for your e-learning service?
  • And now that the session is done and you’re uploading a Gb of a file. Is the user just going to sit there and wait without closing his browser or the tab that is uploading the recording?
  • Where and what do you record? If both sides record, then how do you synchronize the recordings?

It all leads to the fact that at the end of the day, client side recording isn’t something you can use. Unless the recording is short (a few minutes) or you have complete control over the browser environment (and even then I would probably not recommend it).

There are things you can do to mitigate some of these issues too. Like upload fragments of the recording every few seconds or minutes throughout the session, or even do it in parallel to the session continuously. But somehow, they tend not to work that well and are quite sensitive.

Want the pros and cons of client side recording? Here you go:

+
No need to add a media server to the media flow Client side logic is complex and quite dependent on the use case
Requires more on the uplink of the user – or more wait time at the end of the session
Need to know client’s device and behavior in advance

#3 – Media forwarding

This is a lesser known technique – or at least something I haven’re really seen in the wild. It is here, because the alternative is possible to use.

The idea behind this one is that you don’t get to record locally, but you don’t get to route media via a server either.

WebRTC media forwarding

What is done here, is that media is forwarded by one or both of participants to a recording server.

The latest releases of Chrome allows to forward incoming peer connection media, making this possible.

This is what I can say further about this specific alternative:

+
No need to add a media server into the flow – just as an additional external recording server Requires twice the uplink or more
Do you want to be the first to try this technique?

Things to remember

Recording doesn’t end with how you record media.

There’s meta data to treat (record, playback, sync, etc).

And then there’s the playback part – where, how, when, etc.

There are also security issues to deal with and think about – both on the recording end and on the playback side.

These are covered in a bit more detail in the course.

What’s next?

If you are going to record, start by leaning towards server side recording.

Sit down and list all of your requirements for recording, archiving and playback – they are interconnected. Then start finding the solution that will fit your needs.

And if you feel that you still have gaps there, then why not enroll to the Advanced WebRTC Architecture course?

Responses

Alexey Aylarov says:
September 26, 2016

Btw, current media recording functionality in browsers don’t work with RTP stream, it gets data from renderer and encodes it one more time. I believe it should affects the quality of the result.

Reply
    Tsahi Levent-Levi says:
    September 26, 2016

    Alexey,

    Thanks. I believe there’s also a forwarder, where you can take inputs from an incoming peer connection and just forward them elsewhere.
    I still don’t recommend taking that route anyway.

    Reply
Aswath Rao says:
September 26, 2016

Can we not use Canvas element for media forwarding in Firefox?

Reply
    Tsahi Levent-Levi says:
    September 26, 2016

    Not sure as I haven’t tried it out Aswath.

    I remember Firefox doing stuff with Canvas elements and being able to encode them and send them out though.

    Reply
Warren McDonald says:
September 27, 2016

We have been using the forwarding method for audio streams from each participant for some time now. It is quite flexible and allows separation of conference concerns from media acquisition and processing.

We are about about to refactor to mix the audio at one primary end, with known performance, and then send a single steam to the server.

Reply
Daniel Hansen says:
September 28, 2016

I tried several html5 recording solutions on a production site, but without apple and ie webRTC support you have more disappointed user than you can imagine. atm server recording is your friend, but im looking forward to webrtc implementation in 2017…

Playing arround with localStorage,but does not seem to be working. 🙁

Reply
RealTimeWeekly | RealTimeWeekly #149 says:
October 3, 2016

[…] Recording WebRTC Sessions: client side or server side? […]

Reply
dan says:
October 9, 2016

your articles miss code and small details – this make them and your course not real

Reply
    Tsahi Levent-Levi says:
    October 9, 2016

    Dan,

    The course isn’t meant to help people code. There’s enough good references for that floating around out there.

    People who come to me usually need to decide what to code – what to pick as the tools they will be using and why should they pick one framework over another. To be able to do these things, exact code and API has less value than understanding the bigger picture and how things combine.

    The course may well not fit your needs.

    Reply
Philippe Thiers says:
April 17, 2017

Any tutorial for Media forwarding? Now I have some problems with video recording in the client, with long session gave me some memory leaks

Reply
    Tsahi Levent-Levi says:
    April 18, 2017

    Look at this sample: https://webrtc.github.io/samples/src/content/peerconnection/multiple-relay/

    My general suggestion would be to focus on a server side solution and not forwarding.

    Reply
      Philippe Thiers says:
      April 21, 2017

      Thanks!
      For your suggestion, Do you now any example any server side solution?

      Reply
        Tsahi Levent-Levi says:
        April 21, 2017

        Kurento, Janus and Jitsi all have the ability to also record. Each with its own strengths and limitations.

        Reply
          Marcus says:
          September 6, 2017

          Hi, thanks for this nice summary. I looked at Kurento, Janus, Jitsi quickly. At a first glance, it seems Kurento is the quickest and easiest to get up and running for reimplementing a WebRTC call that was peer-to-peer, into a WebRTC call that goes through the media server so it can be recorded. I was wondering if you could elaborate on the strengths and weaknesses of the alternatives of Janus and Jitsi.

          Reply
          Tsahi Levent-Levi says:
          September 6, 2017

          Marcus,

          There are different reasons to select different media servers, and I go over these in detail in my online course.

          Two quick things that come to mind here:

          1. This is also a religion of sorts. People tend to stick with what they know and like

          2. Kurento is great, but it is practically unchanged for a year now. This makes it a risky project at the moment, although there are some positive signs of movement lately

          Reply
Krishna Sai says:
May 5, 2017

Has anyone had any luck getting Jitsi recording to work?

Reply
kt says:
June 9, 2017

Is a TURN/STUN server required if WebRTC is used only to capture audio from a hosts microphone and transfer it to a database via a media server?

From what I’ve researched it doesn’t seem to be needed but I have found it really difficult to get concrete information on using WebRTC to record audio.

Reply
    Tsahi Levent-Levi says:
    June 9, 2017

    kt,

    It depends.

    If you plan on recording on the client side, without really using the PeerConnection, sending the recorded media over a websocket or in an HTTP message – then you have no need for TURN/STUN servers. These are only required if you make use of the PeerConnection and plan to record on the server side.

    Tsahi

    Reply
Nilesh Yadav says:
September 29, 2017

I am interested to know in which peer connection audio stream flowing from client to server. i need to know how can i get to know i mean by looking on which parameter on server side audio drop can be identified. Some times in session we cant hear participant. Please Reply.

Reply
    Tsahi Levent-Levi says:
    October 1, 2017

    Nilesh,

    I am not sure I understand the question. In most cases, a single peer connection will handle both the audio and the video will be sent and received together. In most cases, on the same SRTP connection.
    If the issues you are having are connectivity ones, then first look at your TURN configuration and try understanding your webrtc-internals better. This should be of help: https://testrtc.com/webrtc-internals-documentation/

    Reply
Ravi says:
August 20, 2019

I am facing issues while recording the stream that I am receiving from a Peer webrtc connection. Once I stop recording and try to play it I only see the last frame. I am using MediaRecorder API. Recording of local stream from getuser media seems to work fine. Any idea what may be the issue ?

Reply

Comment