SFU stands for Selective Forwarding Unit. Also known in the specifications as SFM (Selective Forwarding Middlebox).
At times, the term is used to describe a type of video routing device, while at other times it will be used to indicate the support of routing technology and not a specific device.
An SFU is a media server component capable of receiving multiple media streams and then deciding which of these media streams should be sent to which participants. Its main use is in supporting group calls and live streaming/broadcast scenarios.
Publishers and subscribers
In many cases, we associate roles to users/streams in an SFU. The publisher is the one publishing or contributing a stream by sending its media towards the media server. The subscriber is the one who wants to receive and view a stream that has been published towards the media server. For group meetings, all users are both publishers of their own media streams as well as subscribers to media streams published by other participants. In broadcasts, there are a small number of publishers and a larger number of subscribers.
The great thing about his approach is that not all users must publish and not all users must subscribe. Furthermore, each user has its own set of subscribed streams which can be different than those of others in the session. This leads to higher flexibility and personalization of the session – either due to network and device capabilities or simply the preference and role of the user in the session itself.
SFU performance
WebRTC SFUs are the most common media server architecture today when implementing large group meetings and live streaming services. The reason for that is that it gives the best return on investment. You will find these implementations in most video conferencing and group video meeting applications. In audio-only use cases they are a bit less popular, though there are a few that use them in these cases as well.
SFUs don’t process the media but rather route it around. As such, they consume considerably less CPU than their MCU alternative. Their performance relies heavily on network throughput.
When deploying SFU servers, it is recommended to place them as close as possible to the users that need to connect to them, spreading them geographically across the globe.
Technology used in SFUs
Common technologies used with an SFU include:
- Simulcast, enabling receiving multiple video streams at different bitrates and then selectively deciding which to forward to which viewer
- Temporal scalability used to create a video stream that has less dependencies between its frames, enabling dropping the frame rate on the bitstream when forwarding to certain viewers with less available bitrate (or need)
- SVC, used to send a single video stream from senders that has multiple layers in them that enable the SFU to strip some of these layers independently per viewer
- Insertable Streams, used to provide E2EE in group calls so that the media servers will not have access to the actual media they are processing
