Predictions overview

The Predictions tab of a Deployment is where a Computer Vision team looks at what a ModelVersion really does in production: every PredictedAsset and its Prediction logged on the Deployment, which you can browse, sort, filter, select, and send to review. It follows the same philosophy as the other image overviews of Picsellia, such as the Assets overview, and offers the same three views:

• The Grid view, to scan many PredictedAsset at a glance.
• The Table view, to compare their properties.
• The Details view, to inspect one PredictedAsset with its Prediction.

The number next to the Predictions tab is the number of PredictedAsset logged on the Deployment. How they get logged is detailed in Inferences.

1. PredictedAsset VS Data

As explained in the glossary, a Data is an element of the Datalake that represents an image physically stored through a Storage Connector. A PredictedAsset is an element local to a Deployment, the same way an Asset is local to a DatasetVersion. Asset and PredictedAsset are two similar objects in terms of philosophy, structure, and use.

However, a PredictedAsset is always related to a unique Data of the Datalake. Logging an image on a Deployment creates this Data in the target Datalake, and the PredictedAsset you visualize in the Predictions tab is linked to it. This means you can add this Data to a DatasetVersion, which generates a new Asset: this is how the images met in production feed your next training.

The Metadata of a PredictedAsset inherits from the Data ones, and a PredictedAsset also has Metadata related to its Prediction. A PredictedAsset usually has a Prediction, whereas a Data or an Asset never does.

2. PredictedAsset visualization

Use the three buttons at the top left of the tab to switch from one view to another.

The header of the page displays the name of the Deployment, and thumbnails are generated to keep the navigation fluid whatever the weight of your images.

A. Visualize Prediction and related objects

On each view, the Shape contained in the Prediction of a PredictedAsset are displayed, whatever the Version Type of the ModelVersion: Classification, bounding boxes, polygons, lines, or points. A Prediction can only be related to a PredictedAsset, never to a Data or an Asset.

Three controls above the images choose what you see:

• x / y Labels chooses which Label have their Shape displayed. Hiding a Label hides its Shape but never removes a PredictedAsset from the list.
• Confidence Score filter chooses which Shape are displayed according to their Confidence score, from a threshold (Greater than or Less than), a range (Between), or an exact value (Equal to). It works as a display setting too: the number of PredictedAsset does not change. For example, hide every Shape below 0.6 to see how the ModelVersion behaves on the predictions you would act on.
• Layer chooses the Shape to display: Default, Review (the Shape corrected by a human during the review), Champion, or Shadow. The Shadow layer displays the Prediction of the Shadow Model, when there is one, so you can compare it with the Champion one on the same image.

B. Grid view

The Grid view lets you visualize many PredictedAsset at a glance. Adjust the zoom to change the number of images per row, and the aspect ratio (Square, Landscape, Portrait, or 16:9) to match the format of your images.

Each thumbnail carries two pieces of information:

• A score in the bottom-left corner, which is the Outlier score of the PredictedAsset, as detailed here. Use it to spot the images that differ the most from your Training Data.
• A green check in the bottom-right corner when the PredictedAsset has been reviewed.

C. Table view

The Table view focuses on the properties of each PredictedAsset. It displays:

• The preview, the filename, the width and the height of the image.
• Who created the PredictedAsset (created by) and when (created at).
• The AE outlier score, as detailed here.
• The status in the review workflow: Pending or Reviewed.
• The number of Shape of the Champion and of the Shadow Prediction (champion shapes and shadow shapes), which tells at a glance where the two ModelVersion disagree.

The displayed properties are related to the PredictedAsset, not to the linked Data, and the Metadata and Custom Metadata are only visible in the Details view.

D. Details view

The Details view displays everything about one PredictedAsset and its Prediction. As in the Grid view, you can choose the aspect ratio, zoom in or out, select the Layer to display, and filter the displayed Shape by Label or Confidence score.

On the right side, the properties, the tags, and the Metadata of the PredictedAsset are displayed, among which its Status and its AE Outlier Score. Two buttons sit at the top of this panel: Review opens the Prediction Review tool on the PredictedAsset, and Select adds it to your selection.

At the bottom of the view, the gallery displays the thumbnails of the previous and next PredictedAsset, each with its Outlier score in the bottom-left corner. Use the left and right arrow keys for a smooth navigation across the images.

3. PredictedAsset ordering

From any view, use the Order by dropdown to sort the PredictedAsset by:

• Created at.
• Created by.
• Filename.
• AE Outlier Score, which brings the images that look the most unusual to the top (or the bottom) of the list.

The arrows button on its left switches between ascending and descending order, and in the Table view you can also sort from the header of some columns.

4. PredictedAsset filtering

On a Deployment that has been running for months, you rarely look at every image: you look for the ones that matter, such as the ones still to review, the least confident ones, or the ones of a given tag. Several tools, next to the Search Bar, narrow the list down. They can be combined, and the number of PredictedAsset displayed above the list follows.

A. Search Bar

The Search Bar filters the PredictedAsset on criteria such as their status or the number of Shape of their Prediction, with the same Query Language as the Data overview and the Assets overview, detailed here. For example, predictions.status = "Reviewed" keeps the PredictedAsset that have been reviewed.

B. Date

Click the Select Date button to open a Date picker and filter the PredictedAsset on their creation date.

C. Tags

The Select Tags button keeps only the PredictedAsset that carry given tags, without writing a query. It works the same way as the tag filter of the Asset of a DatasetVersion, detailed here: tick the tags, then choose how they are combined with AND, OR, or NOT. Tags are detailed in Tag your PredictedAsset.

D. Custom Metadata and Attribute

Add filter opens the filters that are not part of the Query Language. It proposes only the sections that apply to your Deployment, such as Custom metadata, Reviewed shapes attributes, and Predicted shapes attributes, to filter on the Custom Metadata of the PredictedAsset and on the Attribute values carried by the Shape of its Prediction or of its review. Attribute filtering is detailed here.

E. Score and Label

As seen above, the x / y Labels button and the Confidence Score filter only change which Shape are displayed. To keep only the PredictedAsset predicted with a given Label, or above a given score, use the Search Bar.

F. Campaign filters

When a Prediction Review Campaign exists on the Deployment, a Campaign filters button next to the view buttons displays a row to filter the PredictedAsset on their Task: its Step, its Assignee, and its status (All, In Progress, In Rework, Done, or Skipped). It is the quickest way to see what a given annotator still has to review.

5. PredictedAsset selection

Once you can visualize, sort, and filter the PredictedAsset, you can select one or many of them to act on them. From any view, the selection can be done:

1. Manually, by checking the tickbox of each PredictedAsset (available on all views), or with the Select button of the Details view.

2. In bulk, with the dropdown next to the first tickbox, which proposes Select all (Cmd/Ctrl + A), Select followed by a number of PredictedAsset (250 by default) to select the first ones displayed, and Clear (Esc) to empty the selection.

To select every reviewed PredictedAsset, filter on predictions.status = "Reviewed" in the Search Bar, then use Select all. The number of selected PredictedAsset against the total number currently displayed is shown above the list, and action buttons appear:

• Tag attaches or detaches tags, as detailed below.
• Submit to Feedback Loop sends the reviewed Prediction to the Pipeline, as detailed in Pipeline.
• Delete deletes the selection, as detailed below.

6. Tag your PredictedAsset

A Deployment accumulates production images continuously, and the ones worth acting on are rarely retrievable by Metadata alone. Like the Data of a Datalake and the Asset of a DatasetVersion, the PredictedAsset of a Deployment can carry tags you name yourself, the PredictedAssetTag: you create them from the Tags page of the Deployment Settings, attach and detach them from a selection, see them in the Details view, and filter on them. All of it is detailed in Tagging system.

7. Embeddings and visual search

Browsing production data by Metadata and Prediction only takes you so far: when a ModelVersion starts behaving oddly, what you usually want is the images that look like this one. Embeddings can now be computed for the PredictedAsset of a Deployment, exactly as they are for the Data of a Datalake (detailed here).

The computation is activated from the Data Embeddings page of the Deployment Settings, where you can also follow its progress, see how many PredictedAsset were indexed and which ones failed, Retry the ones that failed, and Deactivate the computation, which deletes the embeddings already computed.

Once the computation is over, select a PredictedAsset and use Find Similar, next to the Search Bar, to look for the ones that resemble it, or Text Search to look for the ones that match a text prompt, the same way as in a Datalake.

8. PredictedAsset deletion

To delete PredictedAsset from a Deployment, select them and click on Delete.