Face Recognition Edge Node

Recognize faces using local processing

🚧

Beta Feature

Beta features are functional, but not yet intended for production release since they are undergoing testing.

Overview

The Face Recognition Edge node recognizes faces (by a match to faces provided by LDAP or a JSON config) within a specified region in live video stream, and adds face metadata to the frame metadata.

To use this node, you specify face images via LDAP or a JSON configuration, and configure this node to perform lookup against those registered faces. You can also add metadata to those faces which this node will retrieve and display or color the face with.

This node makes it easy to build common use cases such as access control, customer experience improvements, etc.

This node requires a AI Model Node and a Track Objects Node before it in order to function properly.

Inputs & Outputs

  • Inputs : 1, Media Format : Raw Video
  • Outputs : 1, Media Format: Raw Video
  • Output Metadata : Face Information

Properties

PropertyValue
display_roiBoolean. If true, ROI info will be drawn on video

Ex. true / false
display_faceinfoIf set, the Face and any related attributes is drawn on video. See below for face attributes format.

Options: disabled, bottom_left, bottom_right, top_left , top_right
face_label_attributesComma separated list of attributes from JSON user data or LDAP attributes used to generate the Face's label.
display_userdataBoolean. Display any associated user metadata on the face?

Ex true / false
roisSemicolon separated list of areas in the video within which the Face Recognition is performed, each area identified by a set of normalized coordinates.

If none is specified, the Face Recognition recognizes faces within the entire frame.

Format: x1,y1,x2,y2,x3,y3,x1,y1

Ex: 0,0,0.25,0.5,0.75,0.5,0,0
roi_labelsComma separated list of labels for each ROI in the rois list above.

Format: label1, label2

Ex: door, window
known_face_sourceSource from which to fetch images and metadata of known faces to lookup against.

Options: ldap, json
json_sourceRaw JSON or URL from which to fetch a JSON containing metadata and URLs for known faces. See below for format.

Ex. {...} or https://yoursite.com/lumeo/known-faces-employees.json

Used only if known_face_source is json.
ldap_source_serverLDAP Server Host + Port

Ex: ldap.google.com:636
ldap_source_use_sslUse SSL for LDAP

Ex. true or false
ldap_source_userLDAP Server User.
ldap_source_passwordLDAP Server Password.
ldap_certLDAP Certificate, if needed.
ldap_cert_keyLDAP Certificate Key, if needed.
ldap_search_baseSearch base to use when accessing LDAP.

Ex. dc=lumeo,dc=com
ldap_attributesExtra Attributes to pick from LDAP as user metadata.
ldap_photo_attribAttribute name that stores the photo or photo URL in your LDAP configuration
ldap_photo_typeFormat of the LDAP Photo Attribute.

Options: raw, url
min_confidenceIgnores face matches if they are below this threshold. Float, range : 0 - 1

Ex. 0.4

Note: This is not the percentage match, but a "face similarity metric" with 0.4 or higher yielding about 97% accuracy.
min_face_size_pixelsMinimum width & height of a face that we attempt to lookup.

Ex. 64
face_modelFace Model to use. 5 Point model is faster than 68 Point model, with minimal loss in accuracy.

Options: small (5 point), large (68 point)
face_lookup_intervalNumber of frames between successive lookups of a particular face.

Ex. 3
max_lookups_per_face Maximum number of lookups for each new face before marking it unrecognized.

Ex: 5

Metadata

Metadata PropertyDescription
nodes.<node_id>Describes the ROIs monitored by this node, and their properties.
Format : as defined in the table below.

<node_id> for Face Recognition Nodes is of the form face_recX (ex. face_rec1)

Example

"nodes": {
    "face_rec1": {
        "type": "face_rec",
        "rois": {
            "<roi_name>": {
                "recognized_faces": [{
                    "id": 1230171012121,
                    "person_id": "1239-12381-23110-1213",
                    "confidence": 0.4,
                    "user_data" : {},
                    "label": "person name"
                  }],
                  "unrecognized_face_ids": [120398123,12397231,21312],
                  "recognized_face_count": 3,
                  "recognized_face_delta": 2,
                  "unrecognized_face_count": 3,          
                  "unrecognized_face_delta": 1          
            }
        }
    }
}

Format

KeyTypeDescription
recognized_facesDictionaryAttributes for recognized faces
recognized_faces.idStringTracking ID for the face
recognized_faces.person_idStringAzure Person ID associated with this face. You can use this to match against repeat occurences of this face.
recognized_faces.confidenceFloatConfidence of the match
recognized_faces.user_dataDictionaryAny user_data stored with the Azure Face API
recognized_faces.labelStringFace Name as stored in the Azure Face API
unrecognized_face_idsArray of StringsTracking IDs for unrecognized Faces.
recognized_face_countIntegerNumber of recognized faces in this frame
recognized_face_deltaIntegerNumber of newly recognized faces in this frame
unrecognized_face_countIntegerNumber of unrecognized faces in this frame
unrecognized_face_deltaIntegerNumber of newly unrecognized faces in this frame

JSON Format

When using known_face_source set to json, the specified JSON configuration must follow the format below.

image_url is a mandatory property and must point to a Face image.
person_id is optional. Unique Identifier for a face.
label is optional. Label to be displayed on the video for the face.
user_data is optional. It is a JSON dictionary with arbitrary keys as needed by your application. See next section for details.

[{
    "image_url": "https://assets.lumeo.com/media/demos/facerec/8672.jpg",
    "person_id": "person1",
        "label": "person_name",
    "user_data": {
      "attrib1": 12345,
      "attrib2": "test"
    }
},
{
    "image_url": "https://assets.lumeo.com/media/demos/facerec/8674.jpg",
    "user_data": ""
},
{
    "image_url": "https://assets.lumeo.com/media/demos/facerec/8680.jpg",
    "user_data": ""
},
{
    "image_url": "https://assets.lumeo.com/media/demos/facerec/8682.jpg",
    "user_data": ""
},
{
    "image_url": "https://assets.lumeo.com/media/demos/facerec/8683.jpg",
    "user_data": ""
},
{
    "image_url": "https://assets.lumeo.com/media/demos/facerec/8684.jpg",
    "user_data": ""
},
{
    "image_url": "https://assets.lumeo.com/media/demos/facerec/8688.jpg",
    "user_data": ""
},
{
    "image_url": "https://assets.lumeo.com/media/demos/facerec/8698.jpg",
    "user_data": ""
},
{
    "image_url": "https://assets.lumeo.com/media/demos/facerec/8718.jpg",
    "user_data": ""
}]

User Data

You can associate additional attributes along with each registered face/person (aka user_data). Lumeo can extract and display these attributes for matched faces. This node will use the following user_data attributes, if present:

  • color: Node will display the recognized face with a bounding box in this color.
  • aux_label : Node will display this label along with the recognized face on the video

These (and any other) user_data properties will be added to recognized_faces.user_data field within Lumeo metadata.

{
  "color":"00ffff",
  "aux_label":"FastPass Valid"
}