Skip to main content
Version: 7.1

Video modifier

The video modifier feature allows you to modify local video frames before they are transmitted. This feature enables your application to apply real-time effects, filters, or transformations to the outgoing video stream.

Supported call typeMinimum SDK version
1-to-1 call, group call (conference)PlanetKit 7.0
Note
  • The video modifier only applies to local video captured from the camera.
  • Video modification processing should be designed to minimize computational cost and latency in order to prevent frame drops and quality degradation.

Use cases​

The main use cases for video modifiers are as follows.

  • Real-time video effects and filters
  • Beauty filters and facial enhancements
  • Color correction and adjustments

Implementation steps​

To implement the video modifier feature, follow these steps.

Implement a video modifier​

Create a class that conforms to PlanetKitVideoModifierDelegate to modify video frames.

import PlanetKit

class VideoEffectModifier: NSObject, PlanetKitVideoModifierDelegate {
    func videoOutput(_ videoBuffer: PlanetKitVideoBuffer) {
        // Access video buffer properties
        let sampleBuffer = videoBuffer.sampleBuffer  // CMSampleBuffer?: video sample buffer
        let timestamp = videoBuffer.timestamp        // CMTime: timestamp
        let rotation = videoBuffer.rotation          // PlanetKitVideoRotation: rotation angle
        let position = videoBuffer.position          // PlanetKitCameraPosition: camera position
        let source = videoBuffer.source              // PlanetKitVideoSource: video source

        // Extract pixel buffer from sample buffer
        guard let originalSampleBuffer = sampleBuffer,
              let pixelBuffer = CMSampleBufferGetImageBuffer(originalSampleBuffer) else {
            return
        }

        // Get video dimensions
        let width = CVPixelBufferGetWidth(pixelBuffer)
        let height = CVPixelBufferGetHeight(pixelBuffer)

        // Apply your custom effect and create a new sample buffer
        guard let modifiedSampleBuffer = applyVideoEffect(
            to: originalSampleBuffer,
            width: width,
            height: height
        ) else {
            // If effect fails, keep original buffer
            return
        }

        // Set the modified sample buffer back to the video buffer
        videoBuffer.sampleBuffer = modifiedSampleBuffer
    }

    func didEncounterError(_ error: PlanetKitVideoModifierError) {
        // Handle errors during video modification
        switch error {
        case .videoFrameRejectedByUpscaling:
            print("Error: Video frame was upscaled (resolution increased)")
        case .videoFrameRejectedByFormatChange:
            print("Error: Video frame pixel format was changed")
        @unknown default:
            print("Error: Unknown video modifier error")
        }
    }

    private func applyVideoEffect(
        to sampleBuffer: CMSampleBuffer,
        width: Int,
        height: Int
    ) -> CMSampleBuffer? {
        // Implement your video effect logic here
        // Note: This runs on the camera capture thread, keep processing lightweight
    }
}

Set the video modifier​

Register the modifier with the camera manager to start modifying video frames.

// Get camera manager instance
let cameraManager = PlanetKitCameraManager.shared

// Create and set the modifier
let videoModifier = VideoEffectModifier()
cameraManager.modifier = videoModifier

Clear the video modifier​

Unregister the modifier to stop modifying video frames.

let cameraManager = PlanetKitCameraManager.shared

// Clear the modifier by setting it to nil
cameraManager.modifier = nil

The APIs related to the video modifier feature are as follows.