Overview

The Face Recognition Edge node recognizes faces (by a match to faces provided by LDAP, JSON or CSV config) within a specified region in live video stream, and adds face metadata to the frame metadata.

To use this node, you specify face images via LDAP or a JSON/CSV configuration, and configure this node to perform lookup against those registered faces. You can also add metadata to those faces which this node will retrieve and display or color the face with.

This node makes it easy to build common use cases such as access control, customer experience improvements, etc.

This node requires a AI Model Node and a Track Objects Node before it in order to function properly.

Inputs & Outputs

Inputs : 1, Media Format : Raw Video
Outputs : 1, Media Format: Raw Video
Output Metadata : Face Information

Properties

Property	Description	Type	Default	Required
`display_roi`	Display ROI info on video.	bool	true	No
`display_faceinfo`	Display the Face and any related attributes on video. Options: `disabled`, `bottom_left`, `bottom_right`, `top_left`, `top_right`.	enum	bottom_left	No
`face_label_attributes`	Comma-separated list of attributes from JSON user data or LDAP attributes used to generate the Face's label.	string	null	No
`display_userdata`	Display any associated user metadata on the face.	bool	false	No
`rois`	Semicolon-separated list of areas in the video within which the Face Recognition is performed, each area identified by a set of normalized coordinates. Format: `x1,y1,x2,y2,x3,y3,x1,y1`.	string	null	No
`roi_labels`	Comma-separated list of labels for each ROI. Format: `label1, label2`. Conditional on `rois`.	string	null	No
`known_face_source`	Source from which to fetch images and metadata of known faces to lookup against. Options: `ldap`, `json`, `csv`.	enum	ldap	Yes
`json_source`	Raw JSON or URL from which to fetch a JSON containing metadata and URLs for known faces. Conditional on `known_face_source` being `json`.	string	null	Yes
`csv_source`	Raw CSV or URL from which to fetch a CSV file containing metadata and URLs for known faces. Conditional on `known_face_source` being `csv`.	string	null	Yes
`ldap_source_server`	LDAP Server Host + Port. Conditional on `known_face_source` being `ldap`.	string	null	Yes
`ldap_source_use_ssl`	Use SSL for LDAP. Conditional on `known_face_source` being `ldap`.	bool	true	Yes
`ldap_source_user`	LDAP Server User. Conditional on `known_face_source` being `ldap`.	string	null	No
`ldap_source_password`	LDAP Server Password. Conditional on `known_face_source` being `ldap`.	string	null	No
`ldap_cert`	LDAP Certificate, if needed. Conditional on `known_face_source` being `ldap`.	string	null	No
`ldap_cert_key`	LDAP Certificate Key, if needed. Conditional on `known_face_source` being `ldap`.	string	null	No
`ldap_search_base`	Search base to use when accessing LDAP. Conditional on `known_face_source` being `ldap`.	string	null	Yes
`ldap_attributes`	Extra Attributes to pick from LDAP as user metadata. Conditional on `known_face_source` being `ldap`.	string	null	No
`ldap_photo_attrib`	Attribute name that stores the photo or photo URL in your LDAP configuration. Conditional on `known_face_source` being `ldap`.	string	null	Yes
`ldap_photo_type`	Format of the LDAP Photo Attribute. Options: `raw`, `url`. Conditional on `known_face_source` being `ldap`.	enum	raw	Yes
`min_confidence`	Ignores face matches if they are below this threshold. Float, range: 0 - 1.	float	0.4	No
`min_face_size_pixels`	Minimum width & height of a face that we attempt to lookup.	int	64	No
`face_model`	Face Model to use. Options: `small` (5 point), `large` (68 point).	enum	small	No
`face_lookup_interval`	Number of frames between successive lookups of a particular face.	int	3	No
`max_lookups_per_face`	Maximum number of lookups for each new face before marking it unrecognized.	int	5	No

Metadata

Metadata Property	Description
`nodes.<node_id>`	Describes the ROIs monitored by this node, and their properties. Format: as defined in the table below. `<node_id>` for Face Recognition Nodes is of the form `face_recX` (ex. `face_rec1`)

Example

"nodes": {
    "face_rec1": {
        "type": "face_rec",
        "rois": {
            "<roi_name>": {
                "recognized_faces": [{
                    "id": 1230171012121,
                    "person_id": "1239-12381-23110-1213",
                    "confidence": 0.4,
                    "user_data" : {},
                    "label": "person name"
                  }],
                  "unrecognized_face_ids": [120398123,12397231,21312],
                  "recognized_face_count": 3,
                  "recognized_face_delta": 2,
                  "unrecognized_face_count": 3,          
                  "unrecognized_face_delta": 1          
            }
        }
    }
}

Format

Key	Type	Description
`recognized_faces`	Dictionary	Attributes for recognized faces
`recognized_faces.id`	String	Tracking ID for the face
`recognized_faces.person_id`	String	Azure Person ID associated with this face. You can use this to match against repeat occurences of this face.
`recognized_faces.confidence`	Float	Confidence of the match
`recognized_faces.user_data`	Dictionary	Any `user_data` stored with the Azure Face API
`recognized_faces.label`	String	Face Name as stored in the Azure Face API
`unrecognized_face_ids`	Array of Strings	Tracking IDs for unrecognized Faces.
`recognized_face_count`	Integer	Number of recognized faces in this frame
`recognized_face_delta`	Integer	Number of newly recognized faces in this frame
`unrecognized_face_count`	Integer	Number of unrecognized faces in this frame
`unrecognized_face_delta`	Integer	Number of newly unrecognized faces in this frame

JSON or CSV Format

When using known_face_source set to json or csv, the specified configuration must follow the format below.

Fields

Field	Requirement	Description
`person_id`	Mandatory	Unique Identifier for a face.
`image_url`	Mandatory	Must point to a Face image and be accessible from the internet. ex. https://assets.lumeo.com/media/demos/facerec/8672.jpg
`label`	Mandatory	Label to be displayed on the video for the face.
`user_data`		You can associate additional attributes along with each registered face/person (aka `user_data`). Lumeo can extract and display these attributes for matched faces. This node will use the following `user_data` attributes, if present. These (and any other) `user_data` properties will be added to `recognized_faces.user_data` field within Lumeo metadata.
`user_data.modifyTimestamp`	Optional	Node will use this to cache face signatures to speed up subsequent startup times if this value has not changed.

`user_data.color`	Optional	Node will display the recognized face with a bounding box in this color.
`user_data.aux_label`	Optional	Node will display this label along with the recognized face on the video
`user_data.<your_attribute_label>`	Optional

JSON

[
    {
        "image_url": "https://assets.lumeo.com/media/demos/facerec/8672.jpg",
        "person_id": "person1",
        "label": "person_name",
        "user_data": {
            "modifyTimestamp": "2022-01-02T12:12:12Z",
            "color": "00ffff",
            "aux_label": "FastPass Valid",
            "attrib1": 12345,
            "attrib2": "test"
        }
    },
    {
        "image_url": "https://assets.lumeo.com/media/demos/facerec/8674.jpg",
        "person_id": "person2",
        "label": "person_name2",
        "user_data": {
            "modifyTimestamp": "2022-01-02T12:12:12Z",
            "color": "00ffff",
            "aux_label": "FastPass Invalid",
            "attrib1": 12345,
            "attrib2": "test"
        }
    },
    {
       ...
    }
]

CSV

person_id,label,image_url,user_data.modifyTimestamp,user_data.aux_label,user_data.color,user_data.add_your_attributes_here
person1,person name,https://assets.lumeo.com/media/demos/facerec/8674.jpg,1/14/2023 8:53:22,access denied,0xff0000,testing

Google Sheets (CSV)

CSV template : https://docs.google.com/spreadsheets/d/1u4t1Fg3nMWQjP902xyLKk-H2ASw_DLNCr4nZ3tX8NAU/edit#gid=0

A neat trick is to enable Publish to Web feature in Google sheets which generates a live updating CSV link that you can configure in this node's settings. This lets you manage your face database in Google Sheets and have Lumeo automatically pull from it every time a deployment is started.

Here's how :

Go to File -> Share -> Publish to Web in Google Sheets menu

1340 — Go to File -> Share -> Publish to Web in Google Sheets menu

Select Link, Sheet and CSV options. Also enable "Automatically republish ... " checkbox.

1115 — Select Link, Sheet and CSV options.

Copy the resulting link into this node's properties.