Search

Lumeo's AI Search enables natural language queries for media recordings saved to Lumeo Cloud

Overview

Lumeo's AI Search makes it easy to find the event or video you are looking for using natural language descriptions. AI Search is available via Monitor -> Search section in the Lumeo Console.

📘

Availability

Search is available on Lumeo's Core and Enterprise plans with a Lumeo Cloud Storage Retention Add-on, and requires that video clips be uploaded to Lumeo Cloud using the Save Clip node.

Search for video using natural language

Search for video using natural language

After a media recordings have been indexed for relevant Data types, you can search for them with various types of metadata as outlined below:

Data typeExampleDescription
Objectsa car, a personSearch for object types and object attributes detected by AI Models.
Object Signaturesa red car , a person carrying a yellow bagSearch for objects whose signatures have been indexed by describing them
Text5XCZ66PSearch for text descriptions generated using OCR, OpenAI GPT4 Lookup, Caption Generator or similar nodes.
Eventslicense plate flaggedSearch for Events marked using Trigger an Event node.
Scene Signaturesa person lying on the floorSearch for specific objects or scenes using descriptions.

Filters

The following filters are available to limit your search :

  • Time range
  • Data type
  • Source
  • Gateway

Relevance, Quantity, and Sorting

When Scene search is enabled, you can adjust the Sensitivity slider to return more relevant results (tighter match) or more number of results (loose match). You should adjust the slider to relax or tighten search matching depending on your query and the number of results returned.

Results

Following these guidelines will help you make the most out of the AI Search capability:

  • When searching for objects or scenes, lead with nouns and pronouns where possible; prefix with "a". Start with the object type you are trying to find and then describe it. Example:
    • a red car
    • a person carrying a yellow object
    • a red car in a parking lot
  • Event, Text, and Object labels such as license plates can be searched by just providing the exact value you are looking for.
  • If your search yields too few or too many recordings, adjust the sensitivity slider ("Relevance" <---> "Quantity") to see more or fewer results.
  • If search takes too long to return results, try restricting the Time range, Source or Gateway before searching.

Indexing and Configuration

Search is available for media recordings that are generated by a pipeline which have a Create Search Index node followed by a Save Clip node. Currently, snapshots and GIFs are not indexed.

Create Search Index node lets you control the data types that are indexed. Indexing is performed on the Lumeo AI Gateway, and the results are uploaded to Lumeo Cloud along with the media. Note that indexing Object or Scene Signatures is computationally heavy.

To get a head start, look for Search enabled templates in the Solutions Gallery.