Setup Alerts

A model in production rarely fails loudly: the images change, the latency creeps up, the performances drop little by little. Picsellia lets you define Alerts on a Deployment, so that you are notified as soon as something goes wrong, without having to watch the Dashboard every day.

These Alerts are defined on metrics that are available in the Dashboard: for instance, when the images sent for inference are drifting, or when the latency is abnormally high.

1. The Alerts section

From a given Deployment, reach the Settings tab and open Alerts, in the Notification section of its menu.

The section lists the Alerts already defined on the current Deployment, which you can edit, and lets you create new ones. They are divided into two categories, depending on the type of metric they supervise:

• Unsupervised alerts, which supervise an Unsupervised metric, for instance avg_latency.
• Supervised alerts, which supervise a Supervised metric, for instance distribution.

Each Alert is displayed as a card with its name and its type (for instance VALUE), the fields that define it, an Enabled switch, and a Save button.

2. Create an Alert

The Create a new alert area, at the bottom of the section, lets you create an Alert:

1. Select the metric to monitor in the Key dropdown. The available keys are latency, avg_latency, is_outlier, outlier_score, is_drift, stats.ap_50_95, and distribution.
2. Define the Type, the Operator (<, <=, =, >=, or >), and the Threshold that, when reached, generates a notification. For an Alert on the distribution key, you define a Percentage instead.
3. Click on Create.

Most keys relate to the metrics of the Dashboard: the latency (latency and avg_latency) to the Latency, is_outlier and outlier_score to the Outlier score, is_drift to the KS Drift and stats.ap_50_95 to the Average Precision.

3. Edit an Alert

From the Alerts section, you can edit an existing Alert:

• Use its Enabled switch to deactivate or reactivate it. Once deactivated, an Alert cannot be triggered.
• Modify its Operator and its Threshold (or its Percentage).

Do not forget to click on Save after each modification of an existing Alert: it is saved card by card.

4. Visualize triggered Alerts

Once your Alerts are configured, the Dashboard tab tells you quickly if any of them has recently been triggered: the Alerts card, next to the Overview at the top of the Dashboard, lists the Alerts triggered on the Deployment, or displays No alert triggered.