The purpose of any
Project on Picsellia is to allow having an AI Lab in which users can perform several
Experiment, evaluate their performances, and select the best one to create the most performing
ModelVersion. To select the best
Experiment in a
Project, Picsellia allows to compare them to facilitate your decision.
The comparison of
Experiment starts in the Project overview, more precisely in the Experiments tab.
From this view, you can select two or more
Experiment to be compared and click on the Compare button.
The Comparison view will then open. This view basically allows putting side-by-side each Metric available in the Log tab of the selected
In order to ensure the consistency of the comparison, you will be able to compare only Metric that have the same Metric type and the same Metric name. This way you are sure that you are not comparing a loss with a recall for instance.
In the below screenshot, one Metric with Line Type and the name val_cls_loss shared by the two compared
The comparison view displays all the Metrics for both
Experiment that share the same Metric Type and Metric name.
Depending on the Metric Type the visualization can differ, indeed for Line, Multi-line, Table, and Bar the Metric values of compared
Experimentare combined under a single item whereas for Single value, Image, and Heatmap, the Metric values are displayed side-by-side.
As you might have understood, the comparison of
Experiment is really valuable when compared
Experiment are using the same training script or at least training scripts that are logging the same kind of Metric on the Experiment Tracking dashboard.
You can compare as many
Experimentas you want but above a certain amount of
Experiment compared the visualization in the Comparison view might be impacted. This is why if you want to remove an
Experiment from the Comparison view, you just need to click on its name in the top right corner.
You now have all the insights you need to compare properly two
Experiment and assess which one suits your needs.
Updated 3 months ago