Face Recognition Azure
Lookup detected faces using Azure Face Recognition API
Overview
The Face Recognition (Azure) node uses Azure Face API to recognize faces (by a match to your private repository) within a specified region in live video stream, and adds face metadata to the frame metadata.
To use this node, you register faces directly with Azure Face API and configure this node to perform lookup against those registered faces. You can also add metadata to those faces using the Azure Face API, which this node will retrieve and display or color the face with.
This node makes it easy to build common use cases such as access control, customer experience improvements, etc.
This node requires a AI Model Node and a Track Objects Node before it in order to function properly.
Inputs & Outputs
- Inputs : 1, Media Format : Raw Video
- Outputs : 1, Media Format: Raw Video
- Output Metadata : Face Information
Properties
| Property | Description | Type | Default | Required |
|---|---|---|---|---|
roi_labels | Regions of interest labels | string | — | No |
rois | Look up faces only in this region. Conditional on roi_labels. Format: comma-separated normalized x,y coordinate pairs; separate multiple polygons with semicolons (for example, 0.1,0.1,0.9,0.1,0.9,0.9). | string | null | No |
azure_endpoint | Azure endpoint | string | null | Yes |
azure_api_key | Azure API Key | string | null | Yes |
person_group_id | ID of the Azure face API large person group to look up faces from. | string | null | Yes |
display_roi | Display ROI on video? | bool | true | No |
display_faceinfo | Display face info on video? Options: Disabled (disabled); Bottom left (bottom_left); Bottom right (bottom_right); Top left (top_left); Top right (top_right). | enum | bottom_left | No |
face_lookup_interval | Number of frames between successive lookups of a particular face. Unit: frames. | number | null | No |
max_lookups_per_face | Max. lookups per face before marking it unrecognized. Unit: lookups. | number | 5 | Yes |
min_confidence | Face match confidence threshold | float | 0.6 | Yes |
min_face_size_pixels | Min. width and height of a face to attempt looking up. | number | 64 | Yes |
debug | Log debugging information? | bool | false | No |
Output Metadata
The fields below are declared by this node's metadata schema; the JSON values are representative examples.
| Path | Type | Description |
|---|---|---|
nodes.<node_id>.rois.<roi_label>.recognized_face_count | integer | Number of recognized faces in this frame |
nodes.<node_id>.rois.<roi_label>.recognized_face_delta | integer | Number of newly recognized faces in this frame |
nodes.<node_id>.rois.<roi_label>.unrecognized_face_count | integer | Number of unrecognized faces in this frame |
nodes.<node_id>.rois.<roi_label>.unrecognized_face_delta | integer | Number of newly unrecognized faces in this frame |
nodes.<node_id>.rois.<roi_label>.coords | array | Array of pixel coordinates defining the configured ROI or line. |
nodes.<node_id>.rois.<roi_label>.recognized_faces[].id | integer | Tracked face object ID. |
nodes.<node_id>.rois.<roi_label>.recognized_faces[].person_id | string | Identifier of the matched person. |
nodes.<node_id>.rois.<roi_label>.recognized_faces[].confidence | number | Confidence of the face match. |
nodes.<node_id>.rois.<roi_label>.recognized_faces[].user_data | any JSON value | JSON-compatible value containing user data stored with the matched identity. |
nodes.<node_id>.rois.<roi_label>.recognized_faces[].label | string | Display label of the matched identity. |
nodes.<node_id>.rois.<roi_label>.recognized_faces[].description | string | Human-readable summary of the face match. |
nodes.<node_id>.rois.<roi_label>.unrecognized_face_ids | array of strings | Tracking IDs for unrecognized Faces. |
nodes.<node_id>.rois.<roi_label>.objects_of_interest_keys | array | Array of metadata keys that contain object IDs relevant to downstream integrations. |
nodes.<node_id>.rois.<roi_label>.description | string | Human-readable summary of the node’s current result. |
nodes.<node_id>.rois.<roi_label>.recognized_faces[].user_data.example | string | Value of example. |
nodes.<node_id>.type | string | Identifies the node type that produced this metadata. |
JSON example
{
"nodes": {
"face_rec_azure1": {
"rois": {
"roi1": {
"coords": [],
"description": "value",
"objects_of_interest_keys": [],
"recognized_face_count": 3,
"recognized_face_delta": 2,
"recognized_faces": [
{
"confidence": 0.4,
"description": "value",
"id": 1230171012121,
"label": "person name",
"person_id": "1239-12381-23110-1213",
"user_data": {
"example": "value"
}
}
],
"unrecognized_face_count": 3,
"unrecognized_face_delta": 1,
"unrecognized_face_ids": [
120398123,
12397231
]
}
},
"type": "face_rec_azure"
}
}
}Object labels and attributes
- Object labels/classes added: None.
- Object attribute labels/classes added: Recognized face/person label (
10200) and Azure person ID (10201).
The following information is added to the detected object's "attributes" array:
| "class_id" field | "label" field | "probability" field |
|---|---|---|
| 10200 | The face label - usually the person name (string) | The face recognition confidence |
| 10201 | The face ID (string) | The face recognition confidence |
Azure Face API
Lumeo will lookup an Azure LargePersonGroup for faces, trained using these attributes:
- detectionModel :
detection_03 - recognitionModel :
recognition_04
Create an Azure Face API Endpoint
Follow the steps outlined under Prerequisites in the Azure Face API Quickstart to obtain your Face API Endpoint and API Key to configure within this node.
Create and Train a Person Group
You will register your faces directly with the Azure API, and then Lumeo will use the specified group to lookup unknown faces against that group in real time.
Helpful resources:
- Adding Faces to LargePersonGroup
- Azure : Best Practices for Adding user to a Face Service
- Azure : Building a React App to Enroll Users
Azure Face API UIFor a quick and easy way to get started, we've created a sample app using the Azure Face API that lets you create Person Groups, Persons and register Face images using a web app. Check it out here.
The code samples below show you how to train a Azure large person group to work with Lumeo. Start by installing the client library:
pip install --upgrade azure-cognitiveservices-vision-face
import glob, os, sys, time
from urllib.request import urlopen
from azure.cognitiveservices.vision.face import FaceClient
from azure.cognitiveservices.vision.face.models import TrainingStatusType, FaceAttributeType
from msrest.authentication import CognitiveServicesCredentials
ENDPOINT = "<YOUR_AZURE_FACEAPI_ENDPOINT>"
KEY = "<YOUR_AZURE_FACEAPI_KEY>"
# Name your person group. Must be lowercase, alphanumeric, or using dash or underscore.
# We use a uuid to avoid name collisions.
PERSON_GROUP_ID = "lumeo-demo"
# Create a client
face_client = FaceClient(ENDPOINT, CognitiveServicesCredentials(KEY))
# Create and train a person group and add a person group person to it.
def build_person_group(client, person_group_id, pgp_urls,create_group=True):
print('Create and build a person group...')
# Create empty Person Group. Person Group ID must be lower case, alphanumeric, and/or with '-', '_'.
print('Person group ID:', person_group_id)
if create_group:
client.large_person_group.create(large_person_group_id = person_group_id, name=person_group_id, recognition_model= "recognition_04")
for image_url in pgp_urls:
try:
pgp_name = image_url.split("/")[-1].split(".")[0]
new_person = client.large_person_group_person.create(person_group_id, pgp_name)
client.large_person_group_person.add_face_from_stream(person_group_id, new_person.person_id, urlopen(image_url),detection_model="detection_03")
except Exception as e:
print("Error downloading face image from url ({}) : {}".format(image_url,str(e)))
# Train the person group, after a Person object with many images were added to it.
client.large_person_group.train(person_group_id)
# Wait for training to finish.
while (True):
training_status = client.large_person_group.get_training_status(person_group_id)
print("Training status: {}.".format(training_status.status))
if (training_status.status is TrainingStatusType.succeeded):
break
elif (training_status.status is TrainingStatusType.failed):
client.large_person_group.delete(person_group_id=PERSON_GROUP_ID)
sys.exit('Training the person group has failed.')
time.sleep(5)
# Train a person group
person_face_image_urls = ['https://image1.jpg','https://image2.jpg']
build_person_group(face_client,PERSON_GROUP_ID,person_face_image_urls,True)Set Person User Data
Azure Face API allows you to associate additional attributes along with each registered face/person (aka user_data). Lumeo can extract and display these attributes for matched faces from the Azure Face API. This node will use the following user_data attributes, if present from Azure API:
color: Node will display the recognized face with a bounding box in this color.aux_label: Node will display this label along with the recognized face on the video
These (and any other) user_data properties will be added to recognized_faces.user_data field within Lumeo metadata.
{
"color":"00ffff",
"aux_label":"FastPass Valid"
}Here's a code snippet to set this Person User Data using the Azure API
import glob, os, sys, time
from urllib.request import urlopen
from azure.cognitiveservices.vision.face import FaceClient
from azure.cognitiveservices.vision.face.models import TrainingStatusType, FaceAttributeType
from msrest.authentication import CognitiveServicesCredentials
ENDPOINT = "<YOUR_AZURE_FACEAPI_ENDPOINT>"
KEY = "<YOUR_AZURE_FACEAPI_KEY>"
# Assumes that a Large Person Group with this Person Group ID was already created.
PERSON_GROUP_ID = "lumeo-demo"
# Assumes that there is a person in this group with this person id.
PERSON_ID = "421cac29-2b6c-418a-af6c-79181daa92d4"
# Create a client
face_client = FaceClient(ENDPOINT, CognitiveServicesCredentials(KEY))
# Update Attrributes of the person with the Lumeo format attributes
face_client.large_person_group_person.update(
PERSON_GROUP_ID,
PERSON_ID,
user_data='{"level":"fastpass","color":"00ffff","aux_label":"FastPass Valid"}')Updated 18 days ago
