Overview

The Datalake is a shared place to access, and organize your images and videos. It provides a simple way to view and manage your images, videos, and their associated information.

For Computer Vision teams, being able to quickly browse, sort, and filter large volumes of raw images and videos is what makes it possible to spot corrupted files, duplicates, or outliers, and to curate the right subset of Data before it ever reaches a Dataset or a training run.

A Datalake is composed of three tabs: Data (covered on this page), Charts, and Settings.

Your Organization can have several Datalake. You can switch to another one using the dropdown next to the current Datalake name at the top of the page, or click on Datalakes to access the full list of Datalake available in your Organization.

With Picsellia, you have three different viewing options within your Datalake:

• Grid View: A visual layout that displays multiple images or videos at once.
• Table View: A view that focuses on the information associated with each image or video.
• Details View: A comprehensive view that shows all the elements of a specific image or video, including the file itself and its related details.

You can easily switch between these views using the keyboard shortcut ("G", "T", or "D" if not customized)

1. Understanding Data in Picsellia

Data refers to an image or a video that is part of your Datalake. Any image or video used within any Picsellia feature is always associated with Data stored in the Datalake. For example, the Asset composing a Dataset or the Predictions made within a Deployment are always linked to Data stored in the Datalake.

Every Data is physically linked to a file stored on the Cloud Storage integrated with the current Datalake. This link is represented by the object_name property, which is visible from the Details view, and queryable from the Query Language.

The Data in Picsellia's Datalake includes both the image or video and its Properties. These Properties can be viewed using the Table button, which opens the Table view, detailed here.

In addition to these Properties, each Data can carry Metadata. Two systems coexist:

• A fixed Metadata schema: the same predefined list of keys is available to every Data (for instance GPS coordinates or acquisition date).
• A Custom Metadata system: each user can define their own key/value fields.

Unlike Properties, neither is visible from the Table view, only from the Details view. Creating, editing, and deleting Metadata (fixed or custom) is detailed here.

2.Data visualization

Inside the Datalake, you can switch between views using the appropriate button or shortcut

A.Grid View

The Grid View lets you visualize multiple images from the Datalake at once. You can adjust the zoom and change the aspect ratio (Square, Landscape, Portrait, 16:9) to match your images' format.

The Showing dropdown also allows you to display a given Projection name in the grid instead of the thumbnail created for the original image; you must first register the Projection name in the Organization settings.

B.Table View

The Table View primarily focuses on displaying all the information associated with each image from the Datalake.

The Table view provides access to the following Properties:

• preview: thumbnail of your image
• filename: complete filename of your Data
• Width/Height/Area: dimensions and total area in pixels of your Data
• source: source of your Data which can be manually prompted through the SDK during the import (e.g., direct_upload, AWS_bucket, serving, etc.)
• tags: DataTag associated with the Data (more about tags here)
• created at: creation date of the Data on Picsellia
• created by: Picsellia user who created the Data

C.Details View

The Details view aims to show all the elements of a specific image from the Datalake. You can choose the aspect ratio (Square, Landscape, Portrait, 16:9), zoom in/out, or reset the zoom (keyboard shortcuts are available for these actions; the key binding will be displayed by hovering over the associated button)

A floating menu on the right of the Details view lets you switch between two panels:

• The Details panel, showing the Properties and Metadata described above.

  

• The Data panel, letting you switch between the Thumbnail and Original quality, and select which Projection of the current Data to visualize. Projections are detailed here.

  

3. Exploration modes and filtering

By default, the Datalake is in Query Language mode: the main area shows the Grid, Table, or Details view described above, and the search bar lets you filter Data using the Query Language, detailed here. This search bar stays available regardless of the exploration mode selected.

In addition to the Query Language, Data can also filter by date using the Select Date button, or by Custom Metadata using the Add filter button.

Next to the search bar, a mode selector lets you switch to two other exploration modes, which replace the main view:

• Embeddings: mainly displays a UMAP projection of your Data embeddings, with clustering, letting you explore and visualize different sets of points, in addition to similarity or text-to-image search. Detailed here.

• Charts: build custom charts about your Data (for instance, a histogram of file sizes, or a breakdown by DataTag), detailed here. Charts you create are also accessible anytime from the Charts tab at the top of the Datalake.

  

4.Data ordering

From any Datalake view, you can select the sorting button to access more sorting options, and the arrow button next to it to switch between ascending and descending order.

You can sort the images and videos in the Datalake by:

• Creation Date
• File Name
• File Size
• Source

5.Data selection and actions

To select images or videos from the Datalake for further actions, you can:

1. Manually select by checking the tick box on each Data from any view

2. Select all or the first x displayed Data from your Datalake.

The number of selected Data against the total number of Data currently displayed/filtered in the Datalake will be shown.

Once Data are selected, action buttons let you:

• Attach or detach DataTag, detailed here.
• Launch a Processing, detailed here.
• Add the selected Data to a Dataset, detailed here.
• Delete the selected Data, detailed below.

6.Data deletion

To delete images or videos from your Datalake, select the Data to be deleted, then click on More > Delete data.

7. Settings

From the Datalake Settings tab, you can:

• General: rename the Datalake.
• Tags: manage the DataTag of the Datalake, detailed here.
• Access > Members: visualize members who can access the current Datalake with their level of rights and provide Direct Access (details here)
• Embeddings > Visual Search: activate Visual Search for the Datalake, detailed here.