Class tsvb

Constructors

  • Initiation of main SDK instance.

    Parameters

    • customer_id: string

      the unique customer identifier provided by SDK vendor.

    • inference: any = null

    Returns tsvb

Properties

components: any = ...
customer_id: string

Unique customer identifier provided by SDK vendor.

recorder: IRecorder
sdkOptions: Options

Accessors

  • set onFrame(f): void
  • Parameters

    • f: undefined | Function

    Returns void

  • set onReady(f): void
  • A callback is fired when the SDK is ready for frame processing. Effects configuration should only be applied after this callback has been triggered.

    Parameters

    • f: Function

    Returns void

Methods

  • Type Parameters

    • K extends "lowerthird_3" | "overlay_screen" | "watermark" | "lowerthird_1" | "lowerthird_2" | "lowerthird_4" | "lowerthird_5" | "stickers"

    Parameters

    • c: ClassType<K>
    • id: string

    Returns void

  • Preload ML resources and put it to named cache. Will preload all required models files based on the sdk.config(). Could be used on the app initialisation. Will speed up future SDK initialization.

    Parameters

    • clear: boolean = false

      pass true to delete cache objects

    Returns Promise<void>

  • Clear output stream.

    Returns boolean

  • Disable background effect. As a background will be shown original video.

    Returns boolean

  • Disable blur of the background.

    Returns boolean

  • Returns void

  • Returns void

    Deprecated

    renamed to clearCustomResolution

  • Ability to configure sdk execution environment

    Parameters

    • config: any

      configuration object

    Returns void

    Example

    General configuration options

    config = {
    api_url: 'url', // Enable custom URL configuration for SDK authentication, suitable for on-premises solutions.
    model_url: 'url', // Custom URL for the segmentation model; in most cases, this parameter does not require configuration.
    sdk_url: 'url', // This parameter specifies the URL to the SDK folder for cases where you host the model files yourself.
    preset: 'balanced', // You can set the default segmentation preset to one of the following options: quality, balanced, speed, or lightning.
    proxy: true/false, // The configuration specifies whether segmentation should operate in a separate worker thread (not in the main UI thread), with the default value set to true.
    provider: wasm/webgpu/auto, // Allow users to select where to execute the segmentation. In auto mode, the SDK will verify the availability of a GPU. If a GPU is not available, it will automatically fall back to using WASM
    stats: true/false, // To enable or disable the sending of statistics.
    models: {
    'colorcorrector': 'url', // The feature allows for the provision of a custom model name; if left empty, the feature will be disabled.
    'facedetector': 'url', // The feature allows for the provision of a custom model name; if left empty, the feature will be disabled.
    'lowlighter': 'url', // The feature allows for the provision of a custom model name; if left empty, the feature will be disabled.
    },
    wasmPaths: { // Currently, WASM files are loaded from the same directory where the SDK is placed, but custom URLs are also supported (for example, you can load them from CDNs).
    'ort-wasm.wasm': 'url',
    'ort-wasm-simd.wasm': 'url',
    'ort-wasm-threaded.wasm': 'url',
    'ort-wasm-simd-threaded.wasm': 'url'
    }
    }

    Example

    Example of how to change default segmentation preset

    config = {
    preset: 'lightning'
    }

    Example

    Example of how to disable colorcorrection and facedetection

    config = {
    models: {
    'colorcorrector': '',
    'facedetector': '',
    'lowlighter': ''
    }
    }

    Example

    Example of how to hot models on custom domain

    config = {
    sdk_url: 'https://domain.com/sdk/' // in this derectory should be subfolder models with all required models
    }
  • Type Parameters

    • K extends "lowerthird_3" | "overlay_screen" | "watermark" | "lowerthird_1" | "lowerthird_2" | "lowerthird_4" | "lowerthird_5" | "stickers"

    Parameters

    • arg: K extends keyof OptionsMap
          ? ComponentArguments<K<K>>
          : Omit<ComponentArguments<K>, "options">

    Returns ClassType<K>

  • Disable beautification effect.

    Returns boolean

  • Disable color-corrector effect.

    Returns boolean

  • Disable color-filter effect.

    Returns boolean

  • Disable Frame Skipping - segmentation will be running on every video frame. FrameSkipping disabled by default.

    Returns boolean

  • Disable LowLight effect.

    Returns boolean

  • Disable Mirroring Effect.

    Returns boolean

  • Disable Sharpness effect.

    Returns boolean

  • Disable smart-zoom effect.

    Returns boolean

  • Enable beautification effect.

    Returns boolean

  • Enable color-correction effect.

    Returns boolean

  • Enable color-filter effect.

    Returns boolean

  • Enable Frame Skipping - segmentation will be running on every second frame, this will increase FPS but brings some motion trail

    Returns boolean

  • Enable LowLight effect.

    Returns boolean

  • Enable Mirroring Effect.

    Returns boolean

  • Enable Sharpness effect.

    Returns boolean

  • Enable smart-zoom effect.

    Returns boolean

  • Freeze the stream

    Returns boolean

  • Type Parameters

    • K extends "lowerthird_3" | "overlay_screen" | "watermark" | "lowerthird_1" | "lowerthird_2" | "lowerthird_4" | "lowerthird_5" | "stickers"

    Parameters

    • key: K

    Returns {
        lowerthird_1: typeof LtLeftTextbox;
        lowerthird_2: typeof LtHorizontalMirror;
        lowerthird_3: typeof LtSlideBold;
        lowerthird_4: typeof LtDoubleSlideRect;
        lowerthird_5: typeof LtTwoSlideRects;
        overlay_screen: typeof OverlayScreen;
        stickers: typeof Stickers;
        watermark: typeof Watermark;
    }[K]

  • Get Customer ID provided by vendor.

    Returns string

  • Get the detected faces squares. Works only if SmartZoom is enabled. To get detected faces without applying SmartZoom effects, just use setFaceArea with the minimal parameter.

    Returns FaceCombiner[]

  • Return current active segmentation mode.

    Returns string

  • Get the output MediaStream object for further processing.

    Returns null | MediaStream

  • Hide fps and other stats on the stream.

    Returns boolean

  • Check the minimal requirements for SDK

    Returns boolean

  • Parameters

    • Optional f: (() => void)
        • (): void
        • Returns void

    Returns void

  • Parameters

    • Optional f: (() => void)
        • (): void
        • Returns void

    Returns void

  • Parameters

    • Optional f: ((id) => void)
        • (id): void
        • Parameters

          • id: string

          Returns void

    Returns void

  • pass onError callback to ErrorBus

    Parameters

    • f: ((e) => void)

      callback function that takes ErrorObject as its first argument.

        • (e): void
        • Parameters

          • e: ErrorObject

          Returns void

    Returns void

  • Parameters

    • Optional f: (() => void)
        • (): void
        • Returns void

    Returns void

  • Initialize all ML resources (models and inference objects). The initial configurations are obtained from sdk.config().

    Returns Promise<void>

  • Run the processing of frames.

    Returns boolean

  • Set media source of the background. Video sources will be played automatically from the beginning.

    Parameters

    • url: BackgroundSource

      the link to image/video of the server or one of the following objects: MediaStream, MediaStreamTrack, HTMLVideoElement, ImageBitmap, Canvas.

    Returns boolean

  • Set the background color for background 'color' mode.

    Parameters

    • color: number

      in hexadecimal format.

    Returns void

  • Control background fit/fill mode. Default value is fill mode.

    Parameters

    • mode: string

      the background fit mode, can be fit of fill

    Returns boolean

  • Control face beautification level.

    Parameters

    • level: number

      could be from 0 to 1. Higher number -> more visible effect of beautification.

    Returns boolean

  • Enable blur of the background and set the power of blur.

    Parameters

    • power: number

      of the blur, can be a number from 0 to 1. Higher number -> better blur. This value could affect the performance (CPU/GPU, FPS)

    Returns boolean

  • Parameters

    • level: number

    Returns boolean

    Deprecated

    Due to the update of segmentation models, there is no need for such a function. It will be removed in future versions.

  • Parameters

    • mode: string

      the boundary mode, can be smooth or strong

    Returns boolean

    Deprecated

    with updated models don't need anymore Control boundary mode smooth or strong. Default value is strong mode.

  • Set period in ms for cc-model working.

    Parameters

    • value: number

      can be a number from 0 to 5000 (default 1000)

    Returns boolean

  • Set power of color correction.

    Parameters

    • value: number

      can be a number from 0 to 1 (default 1)

    Returns boolean

  • set color-filter config.

    Parameters

    • config: Partial<ColorFilterConfig>

    Returns boolean

  • Set the layout with custom params

    Parameters

    • persent: {
          size?: number;
          xOffset?: number;
          yOffset?: number;
      }

      objects with the custom params

      • Optional size?: number
      • Optional xOffset?: number
      • Optional yOffset?: number

    Returns boolean

    Example

    persent = {
    xOffset?: number, // horizontal offset relative to center, value can be a number from -1 to 1
    yOffset?: number, // vertical offset relative to center, value can be a number from -1 to 1
    size?: number, mask size percentage// value can be a number from 0 to 1 *
    }
  • Parameters

    • size: ResizeSettings

    Returns void

  • Set the face-area proportion. Used by the smart-zoom effect to calculate frame scale value

    Parameters

    • value: number

      can be a number from 0.01 to 1 (default = 0.1)

    Returns boolean

  • Set the face detector accuracy.

    Parameters

    • value: number

      can be a number from 0.2 to 1 (default 0.75)

    Returns boolean

  • Set filter part for the color correction (dev feature).

    Parameters

    • value: number

      can be a number from 0 to 1 (default 1)

    Returns boolean

  • Show fps on the stream.

    Parameters

    • limit: number

    Returns boolean

  • Set the layout. Useful for presentations.

    Parameters

    • mode: string

      could be the one of the following: center, left-bottom, right-bottom

    Returns boolean

  • Set the layout mode. You can disable segmentation and show full camera frame or hide camera frame.

    Parameters

    • mode: LayoutMode

      could be the one of the following: 'segmentation' | 'full' | 'hide' | 'transparent' Segmentation - Process the segmentation and display the selected background. Full - Show the full original frame without segmentation. Hide - Hide the original frame completely (only the background will be visible). Transparent - Process the segmentation and return the segmented person with a transparent background.

    Returns boolean

  • Set LowLight effect config.

    Parameters

    • config: Partial<LowLightConfig>

      LowLight effect config.

    Returns boolean

  • Set LowLight effect power.

    Parameters

    • value: number

      number from 0 to 1 (default 1)

    Returns boolean

  • The default frame format is RGBA. Using this method, you can change the frame format to I420.

    Parameters

    • format: FrameFormat

    Returns void

  • Parameters

    • size: ResizeSettings

    Returns void

    Deprecated

    renamed to setCustomResolution

  • Set the segmentation mode. Segmentation mode allow to choose combination of quality and speed of segmentation. Balanced mode is enabled by default.

    Parameters

    • preset: string

      in string format. The values could be quality, balanced, speed, lightning.

    Returns Promise<boolean>

  • Set Sharpness effect config.

    Parameters

    • config: Partial<SharpnessConfig>

      Sharpness effect config.

    Returns boolean

  • Set period in ms for face detector reaction.

    Parameters

    • value: number

      can be a number from 0 to 1000 (default 100)

    Returns boolean

  • Set sensitivity for the smart-zoom rection. The set value means the difference between the new and old face-params for the smartzoom reaction

    Parameters

    • value: number

      can be a number from 0 to 1 (default 0.05)

    Returns boolean

  • Set count of the smart-zoom smoothing. The more steps, the higher the smoothing

    Parameters

    • steps: number

      can be a number from 0.01 to 1 (default 0.2)

    Returns boolean

  • Show fps on the stream.

    Returns boolean

  • Stop applying all effects. The original frames will be bypassed to the output stream.

    Returns boolean

  • Switch face-square-drawing mode.

    Parameters

    • isOn: boolean

      is a boolean argument (default false)

    Returns boolean

  • Switch preface-square-drawing mode. Draw face square before processing

    Parameters

    • isOn: boolean

      is a boolean argument (default false)

    Returns boolean

  • Set the canvas where will be rendered the processed frames.

    Parameters

    • canvas: HTMLCanvasElement

      the HTMLCanvasElement object.

    Returns void

  • Unfreeze the stream

    Returns boolean

  • Set the MediaStream object which will be the source of the video frames for processing.

    Parameters

    • stream: MediaStream

      the source MediaStream object.

    • Optional resize: ResizeSettings

    Returns void