WebCodecs is a browser API that provides low-level access to the browser’s built-in audio and video encoders and decoders. Unlike WebRTC’s media pipeline which manages encoding automatically, WebCodecs gives developers direct control over individual frames and encoded chunks.
How WebCodecs works
WebCodecs exposes four main interfaces:
- VideoEncoder / VideoDecoder – encode raw video frames to compressed data and vice versa
- AudioEncoder / AudioDecoder – encode raw audio samples to compressed data and vice versa
- EncodedVideoChunk / EncodedAudioChunk – containers for compressed media data
- VideoFrame / AudioData – containers for raw, uncompressed media
Developers feed raw frames in and get encoded chunks out (or vice versa), with full control over codec configuration, bitrate, keyframe placement, and SVC layers.
WebCodecs and WebRTC
WebCodecs complements WebRTC in several ways:
- Custom pipelines: Build media pipelines that bypass WebRTC’s opinionated encoding stack – useful for applications that need fine-grained codec control
- Server-side processing: Encode/decode media in workers for cloud rendering, transcoding, or recording without a full PeerConnection
- Insertable Streams integration: Process encoded frames from WebRTC using WebCodecs for decode, manipulation, and re-encode
- Codec support: Works with H.264, VP8, VP9, AV1, Opus, and other browser-supported codecs
WebCodecs is increasingly used alongside WHIP/WHEP and WebTransport to build streaming pipelines that don’t require a full WebRTC stack.
