Dataset - Query language

Now that we are able to visualize, version, and annotate the Asset of a DatasetVersion, the next step is to retrieve them in an efficient way.

When a DatasetVersion holds thousands of Asset, manually browsing is not an option: the Query Language lets you combine properties, Metadata, AssetTag, and Annotation-related fields into a single query, so you can instantly carve out a precise, reproducible subset of Asset, for instance every Asset annotated with a given Label but still Pending review, to prioritize annotation work, launch a targeted Processing, or check a specific edge case.

1. Search Bar

To do so, the best tool is the Search Bar and its associated Query Language, used by default in the Query Language exploration mode detailed here.

2. Query Language

The Query Language is used in the Search Bar to filter among the Asset of a DatasetVersion, based on their properties, their AssetTag, or their Annotation.

It allows users to access all Asset properties and sub-properties to search on them and find the Asset satisfying the query.

A. Objects & Metadata available for searching

The Search Bar of a DatasetVersion lets you search on all the objects, properties, and fixed Metadata schema associated with the underlying Data, as well as DatasetVersion-specific fields such as annotations, nb_annotations, tags (i.e. AssetTag), or nb_tags:

Please note that in this list, some fields are other Picsellia objects linked to the Asset or its Annotation. It means that from the DatasetVersion Search Bar, you can go deeper into the Picsellia Object Schema to filter on a property of an object linked to the Asset.

In particular, data is the object linking an Asset back to its underlying Datalake Data. It lets you search on any Data property directly from the DatasetVersion, using data.<property> (for instance data.filename, data.source, data.content_type, data.file_size, data.width / data.height, data.latitude / data.longitude, data.datalake, or data.tags / data.nb_tags for the DataTag of the Data). The fixed Metadata schema of the Data is searchable the same way, through data.metadata.<key>.

For instance, we can retrieve all the Asset from the DatasetVersion whose underlying Data comes from a specific Source, and that have a car annotated as a Rectangle, with this query:

data.source = "AWS_bucket" and annotations.rectangles.label.name = "car"

Depending on the Version Type of your DatasetVersion, the object to query changes with the Shape type: for instance annotations.masks.label.name and annotations.nb_masks for a Mask DatasetVersion, or annotations.keypoints.label.name and annotations.nb_keypoints for a Keypoint one.

To browse through the Query Language across the Picsellia Object Schema, it is highly recommended to use the auto-completion proposed by the Search Bar.

In the end, the last element you are filtering on must be a property of either Asset or of an object linked to it.

In the previous example, annotations and data are the objects linked to the Asset, while rectangles.label.name and source are the properties you are filtering on within each. Most Asset properties, including filename, actually live on the linked Data rather than directly on the Asset itself, so they need the data. prefix:

data.filename = "FileNameOfMyData"

B. Comparison

Depending on the format of the object property you are filtering on, several comparison signs are proposed by the auto-completion engine:

• For strings:

• For integers or floats:

C. Logical connector

In order to perform the most accurate filtering possible among your DatasetVersion, you can create complex queries leveraging the and & or logical connectors.

At the end of the first query element, after leaving one space, the auto-completion engine will propose a logical connector for your query.

3. Asset satisfying the query

After having written your query, you just need to press the Enter key to run the query. The Asset from your DatasetVersion satisfying the query will then be displayed.

It means that after having filtered through the Search Bar, the only Asset displayed in your Assets overview are the ones matching the query. The number of Asset matches is displayed on top of the DatasetVersion as shown below: