Overview

Use the Model inference node to run an AI model on the video stream. The node adds inference outputs (object bounding boxes, labels, probabilities) and raw tensor outputs from the Model to the metadata.

Use the Display Stream Info Node after the Inference Node to display the output on the video stream, and/or Function Node if you need to process the output and use the metadata in your app.

Labels for off-the-shelf models can be found under Model Labels

Detection Models

Detection Models identify objects in a stream, and provide you with the object label along with the coordinates for the object's bounding box. Detection Models can be run on an entire frame or on the output of another Detection Model.

Examples of Detection Models : Face detector, Traffic Detector

Classifier Models

Classifier Models (Such as a Mask Classifier, Vehicle Type Classifier, etc.) operate on the output of a Detector Model and identify the type of the object detected by the Detector.

In order to use a Classifer Model, add a 2nd Model Inference Node after the one that runs the Detector Model, and use the infer_on_node property to configure the 2nd node to operate on the output of the 1st one.

Inputs & Outputs

Inputs : 1, Media Format : Raw or Multiplexed Video
Outputs : Raw or Multiplexed Video (same as Input format)
Output Metadata : Objects, Bounding boxes, classes and Raw tensors.

Properties

Property	Description	Type	Default	Required
`model_id`	Model	`string`	`null`	Yes
`infer_on_node`	If set, run inference only on objects produced by the specified model inference node. Accepted node type: `model_inference`.	`string`	`null`	No
`inference_interval`	Infer on every Nth frame. 1 means infer every frame. Range: minimum `1`. Unit: frames.	`number`	`1`	Yes
`class_properties`	Optional map of class label -> properties. Use "*" for defaults applied to all classes.	`json`	`null`	No

ClassInferenceProperties Property	Value
`min_inference_threshold`	Minimum inference threshold, should be set on the [0.0, 1.0] interval. (optional field)
`eps`	Relative difference between sides of the rectangles to merge them into a group. Used in OpenCV groupRectangles function and DBSCAN algorithm. Increase the threshold to reduce potential duplicate detections of a single object. (optional field)
`object_min_size`	Minimum size (`Width x Height` format) in pixels to consider a detected object. (optional field)
`object_max_size`	Maximum size (`Width x Height` format) in pixels to consider a detected object. (optional field)

Output Metadata

The fields below are declared by this node's metadata schema; the JSON values are representative examples.

Path	Type	Description
`objects`	`array`	Value of objects.
`objects[].attributes`	`dictionary`	Object's classes, as identified by Classifier models chained to this Object Detection model. Contains: `label` : Class label, `probability` : Probability, `class_id` : Unique id for the class' label
`objects[].attributes[].class_id`	`int`	Unique id for this object's label.
`objects[].attributes[].label`	`string`	Label for this object as outlined in Model Labels
`objects[].attributes[].probability`	`float`	Probability of detection
`objects[].class_id`	`int`	Unique id for this object's label.
`objects[].id`	`long int, optional`	If present, a unique ID assigned to this object by a Track Objects Node
`objects[].label`	`string`	Label for this object as outlined in Model Labels
`objects[].probability`	`float`	Probability of detection
`objects[].rect.height`	`integer`	Value of height.
`objects[].rect.left`	`integer`	Value of left.
`objects[].rect.top`	`integer`	Value of top.
`objects[].rect.width`	`integer`	Value of width.

JSON example

{
  "objects": [
    {
      "attributes": [
        {
          "class_id": 10,
          "label": "white",
          "probability": 0.9437
        }
      ],
      "class_id": 0,
      "id": 5750484150146564100,
      "label": "car",
      "probability": 0.7,
      "rect": {
        "height": 25,
        "left": 656,
        "top": 201,
        "width": 47
      }
    }
  ]
}

Object labels and attributes

Object labels/classes added: Detection models add the selected model's object labels and class IDs; see Model Labels.
Object attribute labels/classes added: Classification models add the selected model's class labels and class IDs to upstream objects; see Model Labels.