Effects SDK updates - June 2025

Web

  • Improved handling and reporting of network request issues (e.g., session authentication, model loading).
  • The SDK now performs a consistency check in Electron apps using reference data after initialization. If inconsistencies are detected, it will automatically fall back to the WASM backend.
  • Native inference modules are available for Electron, offering better performance and lower resource usage compared to the WASM backend. See documentation for more details.
  • Improved resource caching logic.
  • Enhanced precision in getMetrics, especially during the first second of execution
Effects SDK updates - May 2025

Web

  • Introduced classic Chroma Key support alongside ML-based segmentation
  • Restored backward compatibility in affected modules

iOS

  • Fixed Color Grading
  • Added ability to pass custom server URL.
Effects SDK updates - April 2025

Web

  • Automatically falls back to the CPU WASM provider if an error occurs in the WebGPU provider during runtime.
  • Added more details for onError callback
  • Introduced the ability to dynamically load effects into the pipeline.
  • A new method has been added to retrieve various metrics, including: Output FPS, Segmentation inference time, Full frame draw time (including segmentation inference)
  • Removed unnecessary allocations and copies for improved integration with Electron native modules.
  • Optimized the camera stream switch for faster performance.
  • Fixed the background positioning issue when rotating the device between portrait and landscape modes.
  • Resolved the problem with incorrect output stream resolution when SDK is not enabled (impacting Safari on mobile devices).

Android

  • Added offline authentication.
  • Fixed frame orientation for landscape mode.

iOS

  • Added offline authentication.
Effects SDK updates - March 2025

Web

  • Drawing Pipeline Optimization: Transitioned to a more straightforward frame-by-frame processing approach.
  • Introduced the onBackgroundSuccess callback, which triggers when the background is successfully changed.
  • Added handling for GPU uncaptured errors to improve debugging and error tracking.
  • Resolved the Low Light Effects issue.
  • Optimized SDK authentication, significantly improving loading times.
Effects SDK updates - February 2025

Web

  • Reduced CPU Overhead on WebGPU: Significantly optimized CPU usage during segmentation tasks when running on WebGPU, ensuring smoother performance and better resource efficiency.
  • isSupported API Endpoint: Introduced a new isSupported API to help developers easily check for compatibility with the SDK before initialization.
  • Multi-Instance GPU Segmentation Support: Added support for running multiple SDK instances with GPU-based segmentation on the same page, enabling more flexible use cases.
  • NPM Package Enhancements: Improved the NPM package structure by providing separate files for ECMAScript Modules (ESM) and CommonJS, making it easier to integrate into different project setups.
  • GPU Device Lost Handling: Implemented robust handling for GPU device loss scenarios, particularly addressing issues on older Intel Integrated Graphics drivers.
  • Dynamic Eval and CSP Compatibility: Fixed issues related to dynamic eval and script loading, improving compatibility with stricter Content Security Policies (CSP).
  • Worker-Timer Errors in Firefox: Resolved Firefox-specific worker-timer errors by moving them to the ErrorBus system instead of throwing exceptions, ensuring smoother error handling.
  • Background Blinking on Intel Integrated Graphics: Fixed an issue where background blinking occurred during parallel segmentation tasks across different tabs on Intel Integrated Graphics.
Effects SDK updates - December 2024

Android

  • Introduce SDK Session server support
  • Add timestamp for imagePipeline.process call
  • Perfomance improvements
  • Fix crashes during the release() process

Desktop

  • Web Authorization instead of watermark.
  • Added support of Neural Engine for Apple systems.
  • New supported CVPixelBuffer formats.
  • Faster Background Blur.
  • Support of metal compatible CVPixelBuffe.
  • Faster NV12 on MacOS.
Effects SDK updates - November 2024

Web

  • Resolved issue where the mask would float in certain edge cases.
  • Removed unsafe-eval from protobuf dependencies for enhanced security.
  • Fixed Webpack warning regarding dynamic require loading issues.

iOS

  • Publish Swift package for Video Effects SDK xcframework.
  • Added support of Neural Engine for iOS.
  • New supported CVPixelBuffer formats.
  • Faster Background Blur.
  • Support of metal compatible CVPixelBuffer.
Effects SDK updates - October 2024

Web

  • Reduced CPU usage by 30-50%, benefiting both the WebGPU and Wasm backends.
  • Resolved rendering issues when the browser tab is inactive, specifically impacting Safari and Firefox.
  • Improved the stability of FPS in the output stream when the browser tab is inactive.
Effects SDK updates - August 2024

Web

  • Person Extraction with Transparent Background: Added the ability to extract a person with a transparent background (available in canvas and MediaStream).
  • Mirroring Effect: Introduced a mirroring effect that mirrors the original stream and applies all subsequent effects to the mirrored stream, eliminating previous issues with orientation and alignment.
  • Beautification Effects: Optimized beautification effects to reduce GPU usage by 50%, enhancing performance.
  • Granular Effect Control: Added more detailed control over which effects are included in the main processing pipeline, providing better customization and stability.
  • Third-Party Library Optimization: Removed third-party libraries that used eval, leading to a more secure and optimized structure.
  • Memory Leak Resolution: Fixed a memory leak issue in frame bypass mode for Safari and Firefox, improving overall stability and performance on these browsers.
Effects SDK updates - May 2024

Web

  • Introduced a feature for video quality improvement. It makes videos sharper and clearer. You can control the power of the effect.
  • Adjustments for Chrome 125 and VideoFrame timestamps.

Desktop

  • Added Balanced Preset for 32-bit build (Win).
  • Fixed issue of Color Grading when used a high-res image in 32-bit build (Win).
  • Fixed issue when Denoise and Sharpness enabled same time (macOS).
  • Fixed issue when selected a high-res background image for Background Replacement while Denoise enabled (Win).
  • Fixed edge artifacts when enabled CPU backed Sharpness (Linux).
  • Fixed rare issue when Blur, Sharpness, and Color Correction effects enabled in same time (macOS).
  • Fixed performance issue of Beautification and Smart Zoom enabled in same time (Win)
Effects SDK updates - April 2024

Desktop

  • Add Sharpening Effect.
Effects SDK updates - March 2024

Web

  • Introduced the feature for executing segmentation on WebGPU
  • Transitioned all graphical rendering processes to WebGL
  • Implemented an automatic adjustment mechanism for stream resolution changes.
  • Introduced a double buffering system for incoming video frames
  • Converted global settings to instance-specific configurations

Android

  • Add sharpening option for frames
  • Use context for camera pipeline
  • Reduce demo app size
  • Performance fix
See Details
Effects SDK updates - February 2024

Web

  • Enhanced initial loading performance through the introduction of preload and cache functionalities.
  • Implemented a feature to pause the streaming content.
  • Expanded the selection of available background options (added Canvas and ImageBitmap).
  • Introduced the capability to turn off segmentation in layouts (show full camera frame or hide it).
  • Improved the handling of resizing for MediaStream in the background.
  • Resolved an issue where video backgrounds would pause when navigating away from the tab.
Effects SDK updates - January 2024

Web

  • Introduced support for portrait segmentation models.
  • Enhanced error and information reporting.
  • Optimized the internal pipeline for ML inference loading.
Effects SDK updates - November 2023

Web

  • New Feature: Low Light Correction - This enhancement, also known as low light adjustments in Zoom, significantly improves video quality in suboptimal lighting conditions or when using lower-quality cameras.
  • Enhancements in Stability and Speed - Various optimizations have been implemented to enhance the overall stability and speed of the software, ensuring a smoother and more reliable user experience.

Desktop

  • Low Light Adjustment
  • Other improvements

Android

  • Updated models for images in portrait orientation
  • Performance improvements
  • Bug fixes

Desktop

  • Low Light Adjustment
Effects SDK updates - October 2023

Web

  • Enhanced Balanced Segmentation Preset
  • Improved Drawing and Buffering Pipeline
  • New onFrame Callback
  • Updated Onnxruntime
  • Ready for XNNPACK Optimizations
See Details
Effects SDK updates - September 2023

Desktop

  • Publish the Video Effects SDK desktop sample
Effects SDK updates - March 2023

Web

  • Added auto-framing functionality.
  • Added automatic color correction functionality.
  • Optimized background blur.
  • Optimized segmentation boundary position.
  • Removed "halo" from person while blur enabled.
Effects SDK updates - February 2023

Android

  • Added Color Grading functionality.

Web

  • Introduced Flutter Integration (Web platform + flutter_webrtc).
See Details
Effects SDK updates - January 2023

Web

  • Introduced Electron Integration with Native Segmentation backend (Linux).
See Details
Effects SDK updates - December 2022

Web

  • Optimized drawing pipelene by moving to WebGL.
See Details
Effects SDK updates - November 2022

Web

  • Introduced speed/balanced/quality segmentation presets.

Windows/macOS/iOS

  • Added Auto-Framing functionality.

Android

  • Significantly improved segmentation quality for portrait mode.
See Details
Effects SDK updates - October 2022

Web

  • Introduced Electron Integration with Native Segmentation backend (Windows/macOS).

Windows/macOS/iOS

  • Added Automatic Color Correction functionality.
See Details

Take a look at our demo applications for Windows, macOS, iOS, Android and the Web. Get an understanding of how our AI effects work in real environments.

AI is changing the world around you

GET FREE TRIAL
svg
svg