Web

• Improved handling and reporting of network request issues (e.g., session authentication, model loading).
• The SDK now performs a consistency check in Electron apps using reference data after initialization. If inconsistencies are detected, it will automatically fall back to the WASM backend.
• Native inference modules are available for Electron, offering better performance and lower resource usage compared to the WASM backend. See documentation for more details.
• Improved resource caching logic.
• Enhanced precision in getMetrics, especially during the first second of execution

Web

• Introduced classic Chroma Key support alongside ML-based segmentation
• Restored backward compatibility in affected modules

iOS

• Fixed Color Grading
• Added ability to pass custom server URL.

Web

• Automatically falls back to the CPU WASM provider if an error occurs in the WebGPU provider during runtime.
• Added more details for onError callback
• Introduced the ability to dynamically load effects into the pipeline.
• A new method has been added to retrieve various metrics, including: Output FPS, Segmentation inference time, Full frame draw time (including segmentation inference)
• Removed unnecessary allocations and copies for improved integration with Electron native modules.
• Optimized the camera stream switch for faster performance.
• Fixed the background positioning issue when rotating the device between portrait and landscape modes.
• Resolved the problem with incorrect output stream resolution when SDK is not enabled (impacting Safari on mobile devices).

Android

• Added offline authentication.
• Fixed frame orientation for landscape mode.

iOS

• Added offline authentication.

Web

• Drawing Pipeline Optimization: Transitioned to a more straightforward frame-by-frame processing approach.
• Introduced the onBackgroundSuccess callback, which triggers when the background is successfully changed.
• Added handling for GPU uncaptured errors to improve debugging and error tracking.
• Resolved the Low Light Effects issue.
• Optimized SDK authentication, significantly improving loading times.

Web

• Reduced CPU Overhead on WebGPU: Significantly optimized CPU usage during segmentation tasks when running on WebGPU, ensuring smoother performance and better resource efficiency.
• isSupported API Endpoint: Introduced a new isSupported API to help developers easily check for compatibility with the SDK before initialization.
• Multi-Instance GPU Segmentation Support: Added support for running multiple SDK instances with GPU-based segmentation on the same page, enabling more flexible use cases.
• NPM Package Enhancements: Improved the NPM package structure by providing separate files for ECMAScript Modules (ESM) and CommonJS, making it easier to integrate into different project setups.
• GPU Device Lost Handling: Implemented robust handling for GPU device loss scenarios, particularly addressing issues on older Intel Integrated Graphics drivers.
• Dynamic Eval and CSP Compatibility: Fixed issues related to dynamic eval and script loading, improving compatibility with stricter Content Security Policies (CSP).
• Worker-Timer Errors in Firefox: Resolved Firefox-specific worker-timer errors by moving them to the ErrorBus system instead of throwing exceptions, ensuring smoother error handling.
• Background Blinking on Intel Integrated Graphics: Fixed an issue where background blinking occurred during parallel segmentation tasks across different tabs on Intel Integrated Graphics.

Android

• Introduce SDK Session server support
• Add timestamp for imagePipeline.process call
• Perfomance improvements
• Fix crashes during the release() process

Desktop

• Web Authorization instead of watermark.
• Added support of Neural Engine for Apple systems.
• New supported CVPixelBuffer formats.
• Faster Background Blur.
• Support of metal compatible CVPixelBuffe.
• Faster NV12 on MacOS.

Web

• Resolved issue where the mask would float in certain edge cases.
• Removed unsafe-eval from protobuf dependencies for enhanced security.
• Fixed Webpack warning regarding dynamic require loading issues.

iOS

• Publish Swift package for Video Effects SDK xcframework.
• Added support of Neural Engine for iOS.
• New supported CVPixelBuffer formats.
• Faster Background Blur.
• Support of metal compatible CVPixelBuffer.

Web

• Reduced CPU usage by 30-50%, benefiting both the WebGPU and Wasm backends.
• Resolved rendering issues when the browser tab is inactive, specifically impacting Safari and Firefox.
• Improved the stability of FPS in the output stream when the browser tab is inactive.

Web

• Person Extraction with Transparent Background: Added the ability to extract a person with a transparent background (available in canvas and MediaStream).
• Mirroring Effect: Introduced a mirroring effect that mirrors the original stream and applies all subsequent effects to the mirrored stream, eliminating previous issues with orientation and alignment.
• Beautification Effects: Optimized beautification effects to reduce GPU usage by 50%, enhancing performance.
• Granular Effect Control: Added more detailed control over which effects are included in the main processing pipeline, providing better customization and stability.
• Third-Party Library Optimization: Removed third-party libraries that used eval, leading to a more secure and optimized structure.
• Memory Leak Resolution: Fixed a memory leak issue in frame bypass mode for Safari and Firefox, improving overall stability and performance on these browsers.

Web

• Introduced a feature for video quality improvement. It makes videos sharper and clearer. You can control the power of the effect.
• Adjustments for Chrome 125 and VideoFrame timestamps.

Desktop

• Added Balanced Preset for 32-bit build (Win).
• Fixed issue of Color Grading when used a high-res image in 32-bit build (Win).
• Fixed issue when Denoise and Sharpness enabled same time (macOS).
• Fixed issue when selected a high-res background image for Background Replacement while Denoise enabled (Win).
• Fixed edge artifacts when enabled CPU backed Sharpness (Linux).
• Fixed rare issue when Blur, Sharpness, and Color Correction effects enabled in same time (macOS).
• Fixed performance issue of Beautification and Smart Zoom enabled in same time (Win)

Desktop

• Add Sharpening Effect.

Web

• Introduced the feature for executing segmentation on WebGPU
• Transitioned all graphical rendering processes to WebGL
• Implemented an automatic adjustment mechanism for stream resolution changes.
• Introduced a double buffering system for incoming video frames
• Converted global settings to instance-specific configurations

Android

• Add sharpening option for frames
• Use context for camera pipeline
• Reduce demo app size
• Performance fix

Web

• Enhanced initial loading performance through the introduction of preload and cache functionalities.
• Implemented a feature to pause the streaming content.
• Expanded the selection of available background options (added Canvas and ImageBitmap).
• Introduced the capability to turn off segmentation in layouts (show full camera frame or hide it).
• Improved the handling of resizing for MediaStream in the background.
• Resolved an issue where video backgrounds would pause when navigating away from the tab.

Web

• Introduced support for portrait segmentation models.
• Enhanced error and information reporting.
• Optimized the internal pipeline for ML inference loading.

Web

• New Feature: Low Light Correction - This enhancement, also known as low light adjustments in Zoom, significantly improves video quality in suboptimal lighting conditions or when using lower-quality cameras.
• Enhancements in Stability and Speed - Various optimizations have been implemented to enhance the overall stability and speed of the software, ensuring a smoother and more reliable user experience.

Desktop

• Low Light Adjustment
• Other improvements

Android

• Updated models for images in portrait orientation
• Performance improvements
• Bug fixes

Desktop

• Low Light Adjustment

Web

• Enhanced Balanced Segmentation Preset
• Improved Drawing and Buffering Pipeline
• New onFrame Callback
• Updated Onnxruntime
• Ready for XNNPACK Optimizations

Desktop

• Publish the Video Effects SDK desktop sample

Web

• Added auto-framing functionality.
• Added automatic color correction functionality.
• Optimized background blur.
• Optimized segmentation boundary position.
• Removed "halo" from person while blur enabled.

Android

• Added Color Grading functionality.

Web

• Introduced Flutter Integration (Web platform + flutter_webrtc).

Web

• Introduced Electron Integration with Native Segmentation backend (Linux).

Web

• Optimized drawing pipelene by moving to WebGL.

Web

• Introduced speed/balanced/quality segmentation presets.

Windows/macOS/iOS

• Added Auto-Framing functionality.

Android

• Significantly improved segmentation quality for portrait mode.

Web

• Introduced Electron Integration with Native Segmentation backend (Windows/macOS).

Windows/macOS/iOS

• Added Automatic Color Correction functionality.