Face Recognition Azure
Lookup detected faces using Azure Face Recognition API
Overview
The Face Recognition (Azure) node uses Azure Face API to recognize faces (by a match to your private repository) within a specified region in live video stream, and adds face metadata to the frame metadata.
To use this node, you register faces directly with Azure Face API and configure this node to perform lookup against those registered faces. You can also add metadata to those faces using the Azure Face API, which this node will retrieve and display or color the face with.
This node makes it easy to build common use cases such as access control, customer experience improvements, etc.
This node requires a AI Model Node and a Track Objects Node before it in order to function properly.
Inputs & Outputs
- Inputs : 1, Media Format : Raw Video
- Outputs : 1, Media Format: Raw Video
- Output Metadata : Face Information
Properties
Property | Value |
---|---|
azure_endpoint | Azure endpoint for cognitive services. ex. https://lumeo.cognitiveservices.azure.com/ |
azure_api_key | Azure API Key for your endpoint ex. asoir2983271sdhaoa |
person_group_id | ID of the Azure Face API Large Person Group to lookup faces from. ex. fastpass-users |
display_roi | Boolean. If true, ROI info will be drawn on video Ex. true / false |
display_faceinfo | If true, the Face and any related attributes is drawn on video. See below for face attributes format. |
rois | Semicolon separated list of areas in the video within which the Face Recognition is performed, each area identified by a set of normalized coordinates. If none is specified, the Face Recognition recognizes faces within the entire frame. Format: x1,y1,x2,y2,x3,y3,x1,y1 Ex: 0,0,0.25,0.5,0.75,0.5,0,0 |
roi_labels | Comma separated list of labels for each ROI in the rois list above. Format: label1, label2 Ex: door, window |
max_lookups_per_face | Maximum number of lookups for each new face before marking it unrecognized. |
min_confidence | Ignores face matches if they are below this threshold. |
min_face_size_pixels | Minimum width & height of a face that we attempt to lookup. |
Metadata
Metadata Property | Description |
---|---|
nodes.<node_id> | Describes the ROIs monitored by this node, and their properties. Format: as defined in the table below. <node_id> for Face Recognition Nodes is of the form face_rec_azureX (ex. face_rec_azure1 ) |
Example
"nodes": {
"face_rec_azure1": {
"type": "face_rec_azure",
"rois": {
"<roi_name>": {
"recognized_faces": [{
"id": 1230171012121,
"person_id": "1239-12381-23110-1213",
"confidence": 0.4,
"user_data" : {},
"label": "person name"
}],
"unrecognized_face_ids": [120398123,12397231,21312],
"recognized_face_count": 3,
"recognized_face_delta": 2,
"unrecognized_face_count": 3,
"unrecognized_face_delta": 1
}
}
}
}
Format
Key | Type | Description |
---|---|---|
recognized_faces | Dictionary | Attributes for recognized faces |
recognized_faces.id | String | Tracking ID for the face |
recognized_faces.person_id | String | Azure Person ID associated with this face. You can use this to match against repeat occurences of this face. |
recognized_faces.confidence | Float | Confidence of the match |
recognized_faces.user_data | Dictionary | Any user_data stored with the Azure Face API |
recognized_faces.label | String | Face Name as stored in the Azure Face API |
unrecognized_face_ids | Array of Strings | Tracking IDs for unrecognized Faces. |
recognized_face_count | Integer | Number of recognized faces in this frame |
recognized_face_delta | Integer | Number of newly recognized faces in this frame |
unrecognized_face_count | Integer | Number of unrecognized faces in this frame |
unrecognized_face_delta | Integer | Number of newly unrecognized faces in this frame |
Objects metadata augmentation
The following information is added to the detected object's "attributes"
array:
"class_id" field | "label" field | "probability" field |
---|---|---|
10200 | The face label - usually the person name (string) | The face recognition confidence |
10201 | The face ID (string) | The face recognition confidence |
"objects": [{
"id": 5750484150146564100,
"label": "face",
"class_id": 0,
"probability": 0.98,
"rect": {
"width": 47,
"top": 201,
"left": 656,
"height": 25.
},
"attributes": [{
"label": "person name",
"class_id": 10200,
"probability": 1.0,
},
{
"label": "1230171012121",
"class_id": 10201,
"probability": 1.0,
}]
}]
Azure Face API
Lumeo will lookup an Azure LargePersonGroup
for faces, trained using these attributes:
- detectionModel :
detection_03
- recognitionModel :
recognition_04
Create an Azure Face API Endpoint
Follow the steps outlined under Prerequisites in the Azure Face API Quickstart to obtain your Face API Endpoint and API Key to configure within this node.
Create and Train a Person Group
You will register your faces directly with the Azure API, and then Lumeo will use the specified group to lookup unknown faces against that group in real time.
Helpful resources:
- Adding Faces to LargePersonGroup
- Azure : Best Practices for Adding user to a Face Service
- Azure : Building a React App to Enroll Users
Azure Face API UI
For a quick and easy way to get started, we've created a sample app using the Azure Face API that lets you create Person Groups, Persons and register Face images using a web app. Check it out here.
The code samples below show you how to train a Azure large person group to work with Lumeo. Start by installing the client library :
pip install --upgrade azure-cognitiveservices-vision-face
import glob, os, sys, time
from urllib.request import urlopen
from azure.cognitiveservices.vision.face import FaceClient
from azure.cognitiveservices.vision.face.models import TrainingStatusType, FaceAttributeType
from msrest.authentication import CognitiveServicesCredentials
ENDPOINT = "<YOUR_AZURE_FACEAPI_ENDPOINT>"
KEY = "<YOUR_AZURE_FACEAPI_KEY>"
# Name your person group. Must be lowercase, alphanumeric, or using dash or underscore.
# We use a uuid to avoid name collisions.
PERSON_GROUP_ID = "lumeo-demo"
# Create a client
face_client = FaceClient(ENDPOINT, CognitiveServicesCredentials(KEY))
# Create and train a person group and add a person group person to it.
def build_person_group(client, person_group_id, pgp_urls,create_group=True):
print('Create and build a person group...')
# Create empty Person Group. Person Group ID must be lower case, alphanumeric, and/or with '-', '_'.
print('Person group ID:', person_group_id)
if create_group:
client.large_person_group.create(large_person_group_id = person_group_id, name=person_group_id, recognition_model= "recognition_04")
for image_url in pgp_urls:
try:
pgp_name = image_url.split("/")[-1].split(".")[0]
new_person = client.large_person_group_person.create(person_group_id, pgp_name)
client.large_person_group_person.add_face_from_stream(person_group_id, new_person.person_id, urlopen(image_url),detection_model="detection_03")
except Exception as e:
print("Error downloading face image from url ({}) : {}".format(image_url,str(e)))
# Train the person group, after a Person object with many images were added to it.
client.large_person_group.train(person_group_id)
# Wait for training to finish.
while (True):
training_status = client.large_person_group.get_training_status(person_group_id)
print("Training status: {}.".format(training_status.status))
if (training_status.status is TrainingStatusType.succeeded):
break
elif (training_status.status is TrainingStatusType.failed):
client.large_person_group.delete(person_group_id=PERSON_GROUP_ID)
sys.exit('Training the person group has failed.')
time.sleep(5)
# Train a person group
person_face_image_urls = ['https://image1.jpg','https://image2.jpg']
build_person_group(face_client,PERSON_GROUP_ID,person_face_image_urls,True)
Set Person User Data
Azure Face API allows you to associate additional attributes along with each registered face/person (aka user_data
). Lumeo can extract and display these attributes for matched faces from the Azure Face API. This node will use the following user_data
attributes, if present from Azure API:
color
: Node will display the recognized face with a bounding box in this color.aux_label
: Node will display this label along with the recognized face on the video
These (and any other) user_data
properties will be added to recognized_faces.user_data
field within Lumeo metadata.
{
"color":"00ffff",
"aux_label":"FastPass Valid"
}
Here's a code snippet to set this Person User Data using the Azure API
import glob, os, sys, time
from urllib.request import urlopen
from azure.cognitiveservices.vision.face import FaceClient
from azure.cognitiveservices.vision.face.models import TrainingStatusType, FaceAttributeType
from msrest.authentication import CognitiveServicesCredentials
ENDPOINT = "<YOUR_AZURE_FACEAPI_ENDPOINT>"
KEY = "<YOUR_AZURE_FACEAPI_KEY>"
# Assumes that a Large Person Group with this Person Group ID was already created.
PERSON_GROUP_ID = "lumeo-demo"
# Assumes that there is a person in this group with this person id.
PERSON_ID = "421cac29-2b6c-418a-af6c-79181daa92d4"
# Create a client
face_client = FaceClient(ENDPOINT, CognitiveServicesCredentials(KEY))
# Update Attrributes of the person with the Lumeo format attributes
face_client.large_person_group_person.update(
PERSON_GROUP_ID,
PERSON_ID,
user_data='{"level":"fastpass","color":"00ffff","aux_label":"FastPass Valid"}')
Updated 3 months ago