Virtual background and noise cancellation for LiveKit
Add background blur and replacement, beautification, auto-framing and AI noise cancellation to LiveKit apps. Effects run on each participant's device before the track is published — no changes to your LiveKit server.
How it works
-
1
Wrap the track
Video: set an Effects SDK TrackProcessor on the camera track. Audio: run the microphone through the Audio Effects SDK.
-
2
Process on the device
The SDK segments and enhances frames or cleans audio in the browser or app — WebGPU, WebGL or WebAssembly on the web.
-
3
Publish as usual
LiveKit publishes the processed track. Every subscriber — people or agents — gets the result.
Code
LiveKit's track processors are the standard way to modify a local track. The sample implements one around the Web Video Effects SDK; the Flutter fork adds the same effects to LocalVideoTrack on Android, iOS and the web.
- Video: blur, image or video backgrounds, beautification, auto-framing, color correction, lower thirds
- Audio: noise suppression and room echo removal, speed / balanced / quality presets
- Change effects during the call without republishing
import { Track, TrackProcessor, VideoProcessorOptions } from 'livekit-client';
import { tsvb } from 'effects-sdk';
export class EffectsVideoProcessor implements TrackProcessor<Track.Kind.Video, VideoProcessorOptions> {
name = 'effects-sdk';
processedTrack?: MediaStreamTrack;
sdk = new tsvb('YOUR_CUSTOMER_ID');
constructor() {
this.sdk.config({ provider: 'webgpu' });
this.sdk.preload();
}
async init(opts: VideoProcessorOptions) {
this.sdk.clear();
this.sdk.useStream(new MediaStream([opts.track]));
this.processedTrack = this.sdk.getStream()?.getVideoTracks()[0];
await new Promise((ready) => (this.sdk.onReady = ready));
this.sdk.run();
}
async restart(opts: VideoProcessorOptions) {
await this.init(opts);
}
async destroy() {
this.sdk.stop();
this.sdk.clear();
this.processedTrack = undefined;
}
}import { createLocalVideoTrack } from 'livekit-client';
const processor = new EffectsVideoProcessor();
const cameraTrack = await createLocalVideoTrack();
await cameraTrack.setProcessor(processor);
await room.localParticipant.publishTrack(cameraTrack);
// Change effects at any time, without republishing
processor.sdk.setBlur(0.6);
processor.sdk.setBackground('https://example.com/office.jpg');import { atsvb } from 'audio-effects-sdk';
const sdk = new atsvb('YOUR_CUSTOMER_ID');
sdk.config({ preset: 'speed', sample_rate: 16000 });
sdk.preload();
const mic = await navigator.mediaDevices.getUserMedia({ audio: true });
sdk.useStream(mic);
sdk.onReady = async () => {
sdk.run();
const cleanTrack = sdk.getStream().getAudioTracks()[0];
// Publish the processed track instead of the raw microphone
await room.localParticipant.publishTrack(cleanTrack);
};// pubspec.yaml: livekit_client from github.com/EffectsSDK/livekit-client-sdk-flutter
// Web only: load tsvb-web.js in index.html and call this first
VideoEffectsSDKExt.initialize('YOUR_CUSTOMER_ID');
final videoTrack = await LocalVideoTrack.createCameraTrack(
CameraCaptureOptions(effectsSdkRequired: true),
);
await videoTrack.start();
// Android and iOS: authorize the track
final status = await videoTrack.auth('YOUR_CUSTOMER_ID');
if (status == AuthStatus.active) {
await videoTrack.setPipelineMode(PipelineMode.blur);
await videoTrack.setBlurPower(0.6);
}What you can add to a LiveKit room
Questions
Building on Flutter? See Flutter integration.
Does it work with LiveKit Cloud and self-hosted LiveKit?
Yes. Effects are applied on the participant's device before the track is published, so the server — LiveKit Cloud or your own — receives an already processed track. Nothing changes on the server side.
How is this different from LiveKit's built-in noise filter?
Effects SDK runs entirely on the end user's device on web, iOS, Android and desktop, works with any LiveKit deployment, and comes with video effects in the same SDK family. Pick the presets (speed, balanced, quality) that fit your users' hardware.
Can I change effects during a call?
Yes. Blur, backgrounds, beautification, framing and the other effects can be switched at any time without recreating or republishing the track.
Does it work for LiveKit voice agents?
Yes. Clean the user's microphone with the Audio Effects SDK before publishing, and your agent gets audio without background noise and room echo. See noise cancellation for voice AI agents.
Which LiveKit SDKs are supported?
livekit-client for the web (JavaScript/TypeScript) through a TrackProcessor, and LiveKit Flutter on Android, iOS and the web through our fork of the Flutter SDK.
Integrating with LiveKit?
Tell us about your LiveKit setup — web, Flutter or agents. We help with the integration.
Ready to build? Open the Developer Portal and start a free trial.