Overview

The AI Model by Roboflow node allows for running a Roboflow AI Model inference on input frames, and adds detected objects to metadata. This functionality is useful for applications requiring object detection and recognition.

Inputs & Outputs

Inputs : 1, Media Format : Raw Video
Outputs : 1, Media Format: Raw Video
Output Metadata : Objects metadata

Properties

Property	Description	Type	Default	Required
`model_id`	Model alias or ID from Roboflow. Typical format is `{project_id}/{version}`.	`string`	`null`	Yes
`model_type`	Model type. Options: Detector (`detector`); Classifier (`classifier`).	`enum`	`detector`	Yes
`roboflow_api_key`	Roboflow Private API Key (for inference) can be found at Account -> Settings -> Workspace -> Roboflow API in your Roboflow account.	`string`	`null`	Yes
`inference_mode`	Inference mode. Options: Full frame (`full_frame`): Run inference on entire frame; Specific objects (`objects`): Run inference on specific object types, ex for classification.	`enum`	`full_frame`	Yes
`objects_to_infer`	ex. `car`,`person`,`car.red`. Object types and classes to include. Leave blank to include all. Conditional on `inference_mode` being `objects`.	model-labels	`null`	No
`interval`	Infer on every nth frame. 1 means infer every frame. Range: minimum `1`. Unit: frames.	`number`	`1`	Yes
`confidence_threshold`	Enable to override the default minimum inference threshold for all classes. Range: minimum `0`, maximum `1.0`. Step: `0.1`.	`float`	`0.5`	No
`iou_threshold`	Increase the threshold to reduce potential duplicate detections of a single object. Conditional on `model_type` being `detector`. Range: minimum `0`, maximum `1.0`. Step: `0.1`.	`float`	`0.5`	No
`min_object_size`	Minimum output object size. Conditional on `model_type` being `detector`.	`string`	`null`	No
`reclassify_size_threshold`	Reclassify a specific object if size changes by this percent. Only used when inference mode `objects`, and Track objects node is present before this node. Conditional on `model_type` being `classifier`. Range: minimum `1`. Unit: percent.	`number`	`50`	No
`max_classify_attempts`	Attempt classification per object this many times before giving up if it yields no class results. Only used when inference mode `objects`, and Track objects node is present before this node. Conditional on `model_type` being `classifier`. Range: minimum `1`.	`number`	`3`	No

Output Metadata

The fields below are declared by this node's metadata schema; the JSON values are representative examples.

Path	Type	Description
`objects`	`array`	Value of objects.
`objects[].attributes`	`dictionary`	Object's classes, as identified by Classifier models chained to this Object Detection model. Contains: `label` : Class label, `probability` : Probability, `class_id` : Unique id for the class' label
`objects[].attributes[].class_id`	`int`	Unique id for this object's label.
`objects[].attributes[].label`	`string`	Label for this object as outlined in Model Labels
`objects[].attributes[].probability`	`float`	Probability of detection
`objects[].class_id`	`int`	Unique id for this object's label.
`objects[].id`	`long int, optional`	If present, it is a unique ID assigned to this object by a Track Objects Node
`objects[].label`	`string`	Label for this object as outlined in Model Labels
`objects[].probability`	`float`	Probability of detection
`objects[].rect.height`	`integer`	Value of height.
`objects[].rect.left`	`integer`	Value of left.
`objects[].rect.top`	`integer`	Value of top.
`objects[].rect.width`	`integer`	Value of width.

JSON example

{
  "objects": [
    {
      "attributes": [
        {
          "class_id": 10,
          "label": "white",
          "probability": 0.9437
        }
      ],
      "class_id": 0,
      "id": 5750484150146564100,
      "label": "car",
      "probability": 0.7,
      "rect": {
        "height": 25,
        "left": 656,
        "top": 201,
        "width": 47
      }
    }
  ]
}

Object labels and attributes

Object labels/classes added: Detection models add their prediction labels and class IDs. Full-frame classification can add a full_frame object with class 44444.
Object attribute labels/classes added: Classification labels use the model's class IDs; multi-label classification uses class 44445.