Comparison

The purpose of any Project on Picsellia is to give you an AI Lab in which you can perform several Experiment, evaluate their performance, and select the best one to create the most performing ModelVersion. To help you select the best Experiment in a Project, Picsellia lets you compare them.

1. Compare Experiment

The comparison of Experiment starts in the Project overview, more precisely in the Experiments tab.

From this view, you can select two or more Experiment to compare and click on the Compare button.

The Comparison view will then open. This view puts side-by-side each Metric available in the Logs tab of the selected Experiment.

To ensure the consistency of the comparison, you can only compare Metrics that share the same Metric type and the same Metric name — this way you're sure you're not comparing, for instance, a loss with a recall.

In the screenshot below, one Metric of type Line and name val_cls_loss is shared by the two compared Experiment.

The Comparison view displays all the Metrics for both Experiment that share the same Metric type and Metric name.

Depending on the Metric type, the visualization can differ: for Line, Multi-line, Table, and Bar, the values of the compared Experiment are combined under a single item, whereas for Single Value, Image, and Heatmap, the Metric values are displayed side-by-side.

As you might expect, comparing Experiment is most valuable when the compared Experiment use the same training script, or at least training scripts that log the same kind of Metrics to the Experiment Tracking dashboard.

You can compare as many Experiment as you want, but above a certain number, the visualization in the Comparison view might be impacted. If you want to remove an Experiment from the Comparison view, click on its name in the top right corner.

You now have all the insights you need to properly compare two Experiment and assess which one suits your needs.