Overview

Processing clips in bulk is a pretty common scenario that Lumeo makes easy to pull off. Typical use cases for bulk processing clips include video search and indexing, alerts and extracting metadata for reporting or dashboards.

There are 3 key steps to bulk process clips:

1. Build and test the pipeline that will process each clip
2. Setup one or more gateways in your workspace that will process these clips
3. Queue clips to be processed using the deployment_queues API or Universal Bridge

Build and Test Your Pipeline

Any Pipeline can process a clip, so this step is no different from building and testing a typical pipeline. See Getting Started Guide as a starting point

Setup Gateways

The preferred way to process a large number of clips is to setup Lumeo Gateways in the cloud as a Kubernetes cluster. You can also request Lumeo-managed Cloud Gateways in your account to save you from setting up your own.

This allows you to scale gateway capacity up and down easily to meet your processing volume needs. However, if you are just experimenting with this capability, note that it will work with any gateway in your workspace.

To setup Lumeo Gateways as a Kubernetes cluster, follow the Gateway Setup Guide or GCP - Kubernetes Guide. A guide for Azure is coming soon too.

📘

Recommendation: Setup a separate workspace for clip processing

We recommend setting up a separate workspace to house the Gateways to be used for bulk processing, so that you have complete control over which gateways are used for processing.

The deployment_queuesAPI used to queue clips for processing in the next step will pick the first available gateway in the workspace, irrespective of whether it is an edge device, or in the cloud and you generally would not want those clips to be routed to a gateway being used for other tasks.

You can control the maximum number of clips each Gateway processes concurrently by setting the Max deployments property in Gateway settings.

Process Clips using API

The last step is to queue clips to be processed when Gateway capacity is available. You can do this using the deployment_queues API. Each Workspace comes with a default deployment queue that will create a deployment from a queued entry on a first-in-first-out basis, on the least utilized gateway.

Pipelines that generate alerts will do so when the deployment runs. Other outputs such as clips generated using the Save Clip Node and Save Snapshot Node can be accessed using the files API, or via your own S3 bucket.

The Api Recipe below shows you how to Queue deployments and access any output files.

Process Clips using Universal Bridge

Lumeo's Universal Bridge lets you upload Clips to Lumeo's cloud using SMTP, FTP as well as via scripts.

Uploading clips using the Universal Bridge creates a virtual camera which can then be configured with a specific pipeline & camera-specific pipeline overrides using the Console. This pipeline is then deployed for each new clip that is uploaded.

Learn more : Universal Bridge

Billing and Performance Considerations

Clips that are uploaded to Lumeo's cloud, and any media artifacts generated by the pipeline count towards your Lumeo cloud storage usage.