Face Recognition Azure

Lookup detected faces using Azure Face Recognition API

Overview

The Face Recognition (Azure) node uses Azure Face API to recognize faces (by a match to your private repository) within a specified region in live video stream, and adds face metadata to the frame metadata.

To use this node, you register faces directly with Azure Face API and configure this node to perform lookup against those registered faces. You can also add metadata to those faces using the Azure Face API, which this node will retrieve and display or color the face with.

This node makes it easy to build common use cases such as access control, customer experience improvements, etc.

This node requires a AI Model Node and a Track Objects Node before it in order to function properly.

Inputs & Outputs

  • Inputs : 1, Media Format : Raw Video
  • Outputs : 1, Media Format: Raw Video
  • Output Metadata : Face Information

Properties

PropertyDescriptionTypeDefaultRequired
roi_labelsRegions of interest labelsstringNo
roisLook up faces only in this region. Conditional on roi_labels. Format: comma-separated normalized x,y coordinate pairs; separate multiple polygons with semicolons (for example, 0.1,0.1,0.9,0.1,0.9,0.9).stringnullNo
azure_endpointAzure endpointstringnullYes
azure_api_keyAzure API KeystringnullYes
person_group_idID of the Azure face API large person group to look up faces from.stringnullYes
display_roiDisplay ROI on video?booltrueNo
display_faceinfoDisplay face info on video? Options: Disabled (disabled); Bottom left (bottom_left); Bottom right (bottom_right); Top left (top_left); Top right (top_right).enumbottom_leftNo
face_lookup_intervalNumber of frames between successive lookups of a particular face. Unit: frames.numbernullNo
max_lookups_per_faceMax. lookups per face before marking it unrecognized. Unit: lookups.number5Yes
min_confidenceFace match confidence thresholdfloat0.6Yes
min_face_size_pixelsMin. width and height of a face to attempt looking up.number64Yes
debugLog debugging information?boolfalseNo

Output Metadata

The fields below are declared by this node's metadata schema; the JSON values are representative examples.

PathTypeDescription
nodes.<node_id>.rois.<roi_label>.recognized_face_countintegerNumber of recognized faces in this frame
nodes.<node_id>.rois.<roi_label>.recognized_face_deltaintegerNumber of newly recognized faces in this frame
nodes.<node_id>.rois.<roi_label>.unrecognized_face_countintegerNumber of unrecognized faces in this frame
nodes.<node_id>.rois.<roi_label>.unrecognized_face_deltaintegerNumber of newly unrecognized faces in this frame
nodes.<node_id>.rois.<roi_label>.coordsarrayArray of pixel coordinates defining the configured ROI or line.
nodes.<node_id>.rois.<roi_label>.recognized_faces[].idintegerTracked face object ID.
nodes.<node_id>.rois.<roi_label>.recognized_faces[].person_idstringIdentifier of the matched person.
nodes.<node_id>.rois.<roi_label>.recognized_faces[].confidencenumberConfidence of the face match.
nodes.<node_id>.rois.<roi_label>.recognized_faces[].user_dataany JSON valueJSON-compatible value containing user data stored with the matched identity.
nodes.<node_id>.rois.<roi_label>.recognized_faces[].labelstringDisplay label of the matched identity.
nodes.<node_id>.rois.<roi_label>.recognized_faces[].descriptionstringHuman-readable summary of the face match.
nodes.<node_id>.rois.<roi_label>.unrecognized_face_idsarray of stringsTracking IDs for unrecognized Faces.
nodes.<node_id>.rois.<roi_label>.objects_of_interest_keysarrayArray of metadata keys that contain object IDs relevant to downstream integrations.
nodes.<node_id>.rois.<roi_label>.descriptionstringHuman-readable summary of the node’s current result.
nodes.<node_id>.rois.<roi_label>.recognized_faces[].user_data.examplestringValue of example.
nodes.<node_id>.typestringIdentifies the node type that produced this metadata.

JSON example

{
  "nodes": {
    "face_rec_azure1": {
      "rois": {
        "roi1": {
          "coords": [],
          "description": "value",
          "objects_of_interest_keys": [],
          "recognized_face_count": 3,
          "recognized_face_delta": 2,
          "recognized_faces": [
            {
              "confidence": 0.4,
              "description": "value",
              "id": 1230171012121,
              "label": "person name",
              "person_id": "1239-12381-23110-1213",
              "user_data": {
                "example": "value"
              }
            }
          ],
          "unrecognized_face_count": 3,
          "unrecognized_face_delta": 1,
          "unrecognized_face_ids": [
            120398123,
            12397231
          ]
        }
      },
      "type": "face_rec_azure"
    }
  }
}

Object labels and attributes

  • Object labels/classes added: None.
  • Object attribute labels/classes added: Recognized face/person label (10200) and Azure person ID (10201).

The following information is added to the detected object's "attributes" array:

"class_id" field"label" field"probability" field
10200The face label - usually the person name (string)The face recognition confidence
10201The face ID (string)The face recognition confidence

Azure Face API

Lumeo will lookup an Azure LargePersonGroup for faces, trained using these attributes:

  • detectionModel : detection_03
  • recognitionModel : recognition_04

Create an Azure Face API Endpoint

Follow the steps outlined under Prerequisites in the Azure Face API Quickstart to obtain your Face API Endpoint and API Key to configure within this node.

Create and Train a Person Group

You will register your faces directly with the Azure API, and then Lumeo will use the specified group to lookup unknown faces against that group in real time.

Helpful resources:

📘

Azure Face API UI

For a quick and easy way to get started, we've created a sample app using the Azure Face API that lets you create Person Groups, Persons and register Face images using a web app. Check it out here.

The code samples below show you how to train a Azure large person group to work with Lumeo. Start by installing the client library:

pip install --upgrade azure-cognitiveservices-vision-face
import glob, os, sys, time
from urllib.request import urlopen
from azure.cognitiveservices.vision.face import FaceClient
from azure.cognitiveservices.vision.face.models import TrainingStatusType, FaceAttributeType
from msrest.authentication import CognitiveServicesCredentials

ENDPOINT = "<YOUR_AZURE_FACEAPI_ENDPOINT>"
KEY = "<YOUR_AZURE_FACEAPI_KEY>"

# Name your person group. Must be lowercase, alphanumeric, or using dash or underscore.
# We use a uuid to avoid name collisions.
PERSON_GROUP_ID = "lumeo-demo"

# Create a client
face_client = FaceClient(ENDPOINT, CognitiveServicesCredentials(KEY))

# Create and train a person group and add a person group person to it.
def build_person_group(client, person_group_id, pgp_urls,create_group=True):
    print('Create and build a person group...')
    # Create empty Person Group. Person Group ID must be lower case, alphanumeric, and/or with '-', '_'.
    print('Person group ID:', person_group_id)
    if create_group:
        client.large_person_group.create(large_person_group_id = person_group_id, name=person_group_id, recognition_model= "recognition_04")

    for image_url in pgp_urls:
        try:
            pgp_name = image_url.split("/")[-1].split(".")[0]
            new_person = client.large_person_group_person.create(person_group_id, pgp_name)
            client.large_person_group_person.add_face_from_stream(person_group_id, new_person.person_id, urlopen(image_url),detection_model="detection_03")
        except Exception as e:
            print("Error downloading face image from url ({}) : {}".format(image_url,str(e)))

    # Train the person group, after a Person object with many images were added to it.
    client.large_person_group.train(person_group_id)

    # Wait for training to finish.
    while (True):
        training_status = client.large_person_group.get_training_status(person_group_id)
        print("Training status: {}.".format(training_status.status))
        if (training_status.status is TrainingStatusType.succeeded):
            break
        elif (training_status.status is TrainingStatusType.failed):
            client.large_person_group.delete(person_group_id=PERSON_GROUP_ID)
            sys.exit('Training the person group has failed.')
        time.sleep(5)

# Train a person group
person_face_image_urls = ['https://image1.jpg','https://image2.jpg']
build_person_group(face_client,PERSON_GROUP_ID,person_face_image_urls,True)

Set Person User Data

Azure Face API allows you to associate additional attributes along with each registered face/person (aka user_data). Lumeo can extract and display these attributes for matched faces from the Azure Face API. This node will use the following user_data attributes, if present from Azure API:

  • color: Node will display the recognized face with a bounding box in this color.
  • aux_label : Node will display this label along with the recognized face on the video

These (and any other) user_data properties will be added to recognized_faces.user_data field within Lumeo metadata.

{
  "color":"00ffff",
  "aux_label":"FastPass Valid"
}

Here's a code snippet to set this Person User Data using the Azure API

import glob, os, sys, time
from urllib.request import urlopen
from azure.cognitiveservices.vision.face import FaceClient
from azure.cognitiveservices.vision.face.models import TrainingStatusType, FaceAttributeType
from msrest.authentication import CognitiveServicesCredentials

ENDPOINT = "<YOUR_AZURE_FACEAPI_ENDPOINT>"
KEY = "<YOUR_AZURE_FACEAPI_KEY>"

# Assumes that a Large Person Group with this Person Group ID was already created.
PERSON_GROUP_ID = "lumeo-demo"

# Assumes that there is a person in this group with this person id.
PERSON_ID = "421cac29-2b6c-418a-af6c-79181daa92d4"

# Create a client
face_client = FaceClient(ENDPOINT, CognitiveServicesCredentials(KEY))

# Update Attrributes of the person with the Lumeo format attributes
face_client.large_person_group_person.update(
		PERSON_GROUP_ID,
    PERSON_ID,
    user_data='{"level":"fastpass","color":"00ffff","aux_label":"FastPass Valid"}')

Did this page help you?