Universal Bridge, Labels, Solink and Rhombus integrations, 360 People models, OSHA compliance and a host of other goodies packed into a fun action-packed spring release!

Hot off the press, here are the highlights from the latest release:

Platform

• 🍀 Universal Bridge : New cloud-based service that lets you add AI to any IP camera by configuring it to send clips to Lumeo's cloud on motion or at periodic intervals. Process these clips using any custom analytic designed in the Lumeo cloud. Read our press release here.
• 🍀 Labels : Simplify management of Cameras, Gateways, Streams using labels. Labels are hierarchical (nest them all you like), and provide a filtered view of your deployed infrastructure.

• A brand new Console UX that makes it easier to navigate and get common tasks done!

Nodes

Integrations

• 🍀 Trigger Solink Event Node : Send Events to Solink when an event occurs in Lumeo. Read more about the Solink integration here
• 🍀 Trigger Rhombus Event : Send Events to
  Rhombus
  when an event occurs in Lumeo.

Recognitions & Rules

• 🍀 Scene Change Detection : Detect scene changes in the camera view. Useful for OSHA compliance (Box stacking, pathways and fire extinguishers unblocked, and more).
• 🍀 Aggregate Metadata Node : Aggregate metadata with Wildcards and output a single trigger. Simplify your Trigger Conditions.
• 🍀 Merge Objects Node : Combine objects from multiple models (such as multiple people detection models) when they overlap. Makes it easy to build pipelines using different models such as day and night people models.
• 🍀 Time Trigger Node : Trigger at certain times of the day, periodic intervals, etc.
• Face Recognition Edge Node : Speed up face loading for edge face rec by 10x
• Webhook Node : New static key format for Webhooks, also image and snapshot POSTS

Models

• 🍀 Person and Vehicle Reidentification Models
• 🍀 360 (Fisheye) camera people detection Models (Premium)
• 🍀 2D Bodypose estimation model

Gateway (v 1.10.10)

• Support for Nvidia Orin devices
• Use your own S3 buckets to store clips and snapshots. Configure in the Save Clip Node and Save Snapshot Node.
• Collect and report runtime metrics to an OpenTelemetry compatible receiver. Configure under Settings -> Metrics
• Bug fixes and performance updates.

Platform

• 🍀 GCP - Kubernetes Guide
• 🍀 Deployment limits per gateway : limit how many deployments can run in parallel on a single gateway
• 🍀 Console UX enhancements: enhanced JSON log viewer, Snapshot previews in streams, incremental loading for large number of cameras/deployments/resources.

Nodes

• 🍀 Webhook Node updates to push snapshots and clips with Form multi-part POST.
• 🍀 Camera calibration Node support
• 🍀 Occupancy Monitor updated to add exited_objects and objected_entered_timestamp to metadata.
• 🍀 Face Recognition Edge Node for Face recognition via the edge (local) processing.
• 🍀 Blur Objects Node now supports conditional blurring

Gateway (v 1.6.22)

• 🍀 Deepstream 6.1 support, Jetson requires Jetpack 5.0, DGPU requires drivers 515.x+, Balena will work as is
• 🍀 gateway_id and gateway_name, pipeline_id, pipeline_name available in function node parameters
• 🍀 Set "Tensor::source_node_id" and add "source" to object and object attribute meta so you can easily tell which node/model generated that object type and attributes.
• 🍀 Support for processing models trained using Azure Custom Vision Service (https://customvision.ai)
• 🍀 [Balena] Auto assign gateway name from BALENA_DEVICE_NAME_AT_INIT env var if present
• 🍀 Add extra metadata to video source node
• 🐛 Reduce MQTT traffic when there are no subscribers to logs topic
• 🐛 On Jetson and dGPU, local snapshot nodes save every frame. This has been fixed.

Platform

• 🍀 Usage limits : Set limits on the number of deployments, cloud storage, video minutes and live streams are processed to control spend.
• 🐛 Improved logging when analytic pipelines encounter errors
• 🐛 Performance improvements, Scalability and Bug fixes.

Nodes

• 🍀 Edit existing lines and ROIs by dragging vertices.
• 🍀 Custom Parser support in Custom Function nodes to process models for which Lumeo doesn't have parsers. See Custom Model Parser for more details.
• 🍀 Trigger Immix Alarm Node
• 🍀 Trigger Email Alert Node

Models

• 🍀 Vehicle and People ReIdentification Models

Gateway (v 1.3.29)

(All connected Gateways will auto update and restart any running deployments)

• 🍀 Balena support for Jetson NX
• 🍀 Torch and Torchvision pre-installed in Gateway container images, use via Custom Functions
• 🍀 Object tensors now available in custom function nodes
• 🐛 Fix for missing cameras during Discovery when a large number of cameras are present

File support for offline processing, Simplified BYO model experience, Pre-buffer for Clips, Homekit Streaming node, Docker containers for Jetson devices and Unattended install mode

Hot off the press, here are the highlights from the latest release:

Platform

• 🍀 File support : Upload files to Lumeo or link a self-hosted file to process it using any gateway. See Files section in left nav to start. See Stream page for details.
• ​🍀 Simplified BYO Model experience: A more structured way to import your models into Lumeo. See Analytics Library -> My AI Models to start.
• ​🍀 Export pipelines to JSON and import them into a new workspace.
• 🐛 Fix Bulk updating deployments causes ROI coordinates to be lost​​

Nodes

• 🍀 Save Clip Node Pre-buffer support. Save clips starting from a time before a trigger occurs.
• 🍀 Homekit Streaming node : Publish Lumeo output streams as Apple Homekit Cameras for quick viewing and trigger alerts based on any analytic condition.
• 🐛 Fix flickering when viewing square aspect ratio video snapshots in Line and ROI nodes.​

Gateway (v 0.4.11)

(All connected Gateways will auto update and restart any running deployments)

• 🍀 Docker container support for Jetson devices, defaults to docker for new installs. See Setting Up a Gateway section for details.
• 🍀 Unattended install support. Specify LUMEO_APP_ID and LUMEO_API_KEY environment variables to the installer to automatically provision a gateway. See Unattended Install section for details.
• 🐛 Fix so that linked ONVIF cameras' streams update automatically. Fixes an issue with new streams not showing up automatically in the Milestone ONVIF Bridge.
• 🐛 Fix inferencing slowdowns for the full YOLO models. Typically yields a 2-3x speedup for YOLO models.
  ​For more details, check out the Release notes.

Reorganized node navigation, Bug fixes and reliability updates

Hot off the press, here are the goodies from the latest release:

Nodes

🍀 Reorganized Node categories in the Node library to make it easier to find the one you need.
🍀 Proximity Detection node lets you choose the tracking point other than the centroid
🐛 Fix duplicate triggers and counts that did not respect the Occupancy Monitor node's object presence buffer

Lumeo Gateway (v 0.3.6)

(All connected Gateways will auto update and restart any running deployments)
🍀 Restart deployments automatically in more situations where the source is unresponsive of stops streaming
🐛 Fix inability to capture camera snapshots for some camera models

Multi-stream and Local License Plate Recognition pipelines, Send snapshots to Elasticsearch, Autoconfigure Milestone Camera IDs, and new nodes for Profiling Performance and Filter Objects

Core Platform

🔥 Pipelines

• New template for Multi-stream pipelines that let you process more streams with a given GPU
• New template for Local License Plate Recognition

Performance, Stability & Security

• 30% runtime performance improvement for pipelines with a large number of objects
• Fixed processing slow down when ingesting variable frame rate streams

Console

• Support double-click in multiline and polygon to finish or close a shape
• Auto generate default URI stream name from URI/file + allow editing in creation form

Models

• 🔥 Models : Local License Plate Recognition so you can read license plates on the edge without making a trip to the cloud!

Pipeline Nodes

Rules

• Line Counter Node Occupancy Monitor - select alternate tracking points to improve performance when Centroid tracking doesn't work well enough.
• Occupancy Monitor -
• Proximity Detector - Optionally add detected Groups as new "objects" so that you can build complex use cases like detect Group presence or speed.

Integrations

• Publish to Elasticsearch Node - send application name, deployment name and snapshots to Elasticsearch.
• Trigger Milestone Event Node - Automatically configures camera ID when you use Milestone ONVIF / Open Network Bridge as the source, making it even easier to associate alerts and events with the correct camera in Milestone.

Recognitions

• License Plate Recognition Node : New "Presence Buffer" property to reduce duplicate triggers.

Other

• 🔥 Profile Performance Node : Capture CPU, GPU utilization, memory and FPS along with system information to profile your pipelines.
• 🔥 Filter Objects Node : Filter out objects within or outside a given ROI
• Function Node - add, modify and delete object metadata using the Function node, for advanced use cases such as filtering out objects or creating new ones.

Docker container support for DGPU installs, New Models: Forklift, Face gear, Crowd People, New Integrations: Elasticsearch, SMS, Pushover, Azure Face Recognition and LPR enhancements.

Lumeo Core Platform

Pipelines

• New, ready-to-use templates for more use cases

Performance, Stability & Security

• Docker container support for DGPU installations, for ease of setup and deployment management
• Installer enhancements to reduce memory utilization, free up disk space and support alternate disk installation for Jetson devices
• Significant runtime performance and stability improvements in light of network issues.

Billing

• View your invoices and update payment methods from the Lumeo Console.

Models

• New Models : Forklift detection, Face gear, Crowd People Body detection (high density)

Lumeo Pipeline Nodes

Integrations (read more here)

• Publish to Elasticsearch Node - publish any metadata to ELK stack to enable search and dashboards via Kibana
• Trigger SMS Alert Node - Use Twilio to send SMS alerts when certain conditions are met.
• Trigger Pushover Notification Node

Recognitions

• Face Recognition Azure Node : Recognize Faces using Azure Face Recognition API. Read more here.
• License Plate Recognition Node : Lookup Make/model/color only when License plates are not legible, and support for zoomed in license plate view modes.

Jetpack 4.6 updates, New VMS Integration Nodes, New Rules: Proximity, Speed, Multi-line, and New Video transformations - Tile video, Multiplex/Demultiplex Nodes. Platform updates to enable easier ROI drawing, and Trigger Conditions for conditional actions.

Lumeo Core Platform

Pipelines

• ROI Drawing Tools : Draw Lines and ROIs on video stream previews.
• Trigger Metadata support: Most Lumeo Nodes now generate metadata that can be used in "Trigger Conditions" within other nodes to generate conditional actions (like take a snapshot when an object crosses a line).
• Trigger Conditions : New Trigger Condition builder in Node properties makes it easier to build trigger conditions with drop down and autocompletion.

Performance, Stability & Security

• Upgrade to Nvidia Jetpack 4.6 / Deepstream 6.0 (more models support with high accuracy & real-time performance).

  This requires you to update any Jetson devices to Jetpack 4.6 to receive Lumeo software updates moving forward. Instructions here.

• Auth updates : We've migrated to a new authentication system for console.lumeo.com. Use the "Forgot Password" link on https://console.lumeo.com to reset your password if you have trouble logging in.

Lumeo Pipeline Nodes

Integrations (read more here)

Send events and alarms to existing VMS installations.

• Trigger Genetec Alarm Node
• Trigger Milestone Event Node
• Trigger HikCentral Alarm Node

Rules

• Multi-Segment Line support for Line Counter Node : Draw multi segment lines to improve accuracy for line counters.
• Proximity Detector : Detect objects that are close to or far away from other objects.
• Speed Estimation : Estimate speeds of objects in a ROI and set triggers based on high/low thresholds.
• Occupancy Monitor : Enhancements to enable duplicate object rejection, and other miscellaneous performance tweaks.

Transformations

• Multiplex Video Node and Demultiplex Streams Node - allow you to combine and split video streams into and from a single stream to improve performance
• Tile Video in Grid - allows you to tile multiple streams of video in a grid, to compare different streams or different algorithms side by side.

Metadata format for the LPR, Line Counter, Presence Counter and Queue nodes has been updated to reduce duplicate information, and introduce a uniform structure with new properties that can be used to trigger snapshots, video clip capture, webhooks, etc.

Summary

Metadata format for the following nodes has been updated to reduce duplicate information, and introduce a uniform structure with new properties that can be used to trigger snapshots, video clip capture, webhooks, etc. :

• License Plate Detection Node
• Line Counter Node
• Presence Counter Node
• Queue Length Counter

These updates take effect when you create new deployments or update existing ones. Starting and stopping existing deployments will not be impacted.

If you are expecting the old(er) format metadata, you can turn on legacy metadata option in the node property to keep the older format while you transition to the new format.

What's changed

Metadata format updates

With this update, these nodes insert their metadata inside the root field nodes, while previously those nodes used top-level fields like lpr, lines, presence, queue. The contents of those fields have now been moved into nodes.<node_id> as per below:

• lpr -> nodes.<node_id>.license_plates
• lines -> nodes.<node_id>.lines
• presence -> nodes.<node_id>.rois
• queue -> nodes.<node_id>.rois

To provide for backward compatibility during this transition, the Line Counter Node and License Plate Detection Node have the option to turn ON the legacy metadata format in node properties. Enabling that option will revert the metadata for those deployments to the older format.

New metadata properties

These nodes now add new metadata to make it easier to trigger webhooks, snapshots, etc. based on the difference in counts between previous and current frame. This makes it easier to trigger when a new car is detected, or when a new object crosses a line, etc.

License Plate Recognition

nodes.annotate_lpr1.license_plates_entered_delta
nodes.annotate_lpr1.license_plates_exited_delta
nodes.annotate_lpr1.license_plates_count

Line Counter

nodes.annotate_line_counter1.lines.<line_label>.total_objects_crossed_delta
nodes.annotate_line_counter1.lines.<line_label>.dir1_objects_crossed_delta
nodes.annotate_line_counter1.lines.<line_label>.dir2_objects_crossed_delta

Presence Counter

nodes.annotate_presence1.rois.<roi_label>.objects_entered_delta
nodes.annotate_presence1.rois.<roi_label>.objects_exited_delta
nodes.annotate_presence1.rois.<roi_label>.objects_above_max_time_threshold_count

Queue Length Counter

nodes.annotate_queue1.rois.<roi_label>.objects_entered_delta
nodes.annotate_queue1.rois.<roi_label>.objects_exited_delta
nodes.annotate_queue1.rois.<roi_label>.objects_above_max_waiting_time_count

For specifics of the new format, see node documentation.

What you need to do

• If you have code that parses Lumeo metadata, either in a Custom Function, or sent to you via webhooks, or from a saved clip, you will need to update it to account for the format changes.
• If you are using the Publish to Google Sheets node, you will need to update the Metadata to Publish property to look for metadata in the new format. Here's an example of the new property value:

video*,nodes.*.dir1_objects_crossed,nodes.*.dir2_objects_crossed,nodes.*.total_objects_crossed,nodes.*.diff_objects_crossed

• While you make those changes, you can turn on "Legacy Metadata" format from node properties to retain the old format.

Spring is (almost) here! And with it, we bring you new nodes, bug fixes and (pretty awesome, if we may say so ourselves) performance improvements. Our focus for the upcoming release is to address issues we are seeing with clip storage, adding trigger conditions for clips & snapshots to make working with streaming metadata easier, and improve Console's responsiveness. Ah, and a new feature that we are sure you'll love.

What's new

• New nodes :
  -- Presence Detector Node : Count and track how long objects have been present in a specific region of interest.
  -- Line Counter Node : Count the number of objects that have crossed the line in one direction or the other.
  -- Webhook Node : Send metadata to a webhook.

• New models : Face detection (for infrared cameras), Vehicle Detection (for Dash cam perspective)

• Updated models : Vehicle Detection, License plate detection, Vehicle type classification.

• Deployment management UX has a new look, with a split pane view that leads to less clicking and easier access to the deployments you are looking for.

• x86 + DGPU support : You can now run the lumeo gateway installer on x86 linux (Ubuntu 18.04) with discrete Nvidia GPUs. For those times when Arm just won't do :). This means you can install Lumeo in AWS, Azure, or Google Cloud GPU-based VMs, or on local linux boxes.

Fixes

• Major under the hood revisions to our MQTT infrastructure, making connections from lumeod to our cloud much more reliable in the face of local network issues.
• Starting, stopping and updating Deployments is much more reliable now. Deployments auto start when created. When gateway auto updates or restarts, previously running pipelines will now auto start. When you update a running deployment, it will automatically restart to pick up new parameters - all the stuff you'd not want to think twice about.
• Lumeo can discover and link ONVIF enabled NVRs. We also fixed a pesky issue that resulted in some NVRs locking out the Lumeo host IP completely.
• Camera status is now accurately reflected in the Console.
• Pipelines with WebRTC and RTSP output streams now work as expected.
• And many more.