Experiment - Comparison

The purpose of any Project on Picsellia is to allow having an AI Lab in which users can perform several Experiment, evaluate their performances, and select the best one to create the most performing ModelVersion. To select the best Experiment in a Project, Picsellia allows to compare them to facilitate your decision.

1. Compare Experiment

The comparison of Experiment starts in the Project overview, more precisely in the Experiments tab.

From this view, you can select two or more Experiment to be compared and click on the Compare button.

The Comparison view will then open. This view basically allows putting side-by-side each Metric available in the Log tab of the selected Experiment.

In order to ensure the consistency of the comparison, you will be able to compare only Metric that have the same Metric type and the same Metric name. This way you are sure that you are not comparing a loss with a recall for instance.

In the below screenshot, one Metric with Line Type and the name val_cls_loss shared by the two compared Experiment.

The comparison view displays all the Metrics for both Experiment that share the same Metric Type and Metric name.

Depending on the Metric Type the visualization can differ, indeed for Line, Multi-line, Table, and Bar the Metric values of compared Experimentare combined under a single item whereas for Single value, Image, and Heatmap, the Metric values are displayed side-by-side.

As you might have understood, the comparison of Experiment is really valuable when compared Experiment are using the same training script or at least training scripts that are logging the same kind of Metric on the Experiment Tracking dashboard.

You can compare as many Experimentas you want but above a certain amount of Experiment compared the visualization in the Comparison view might be impacted. This is why if you want to remove an Experiment from the Comparison view, you just need to click on its name in the top right corner.

You now have all the insights you need to compare properly two Experiment and assess which one suits your needs.