Experiment - Comparison
The purpose of any Project
on Picsellia is to allow having an AI Lab in which users can perform several Experiment
, evaluate their performances, and select the best one to create the most performing ModelVersion
. To select the best Experiment
in a Project
, Picsellia allows to compare them to facilitate your decision.
1. Compare Experiment
Experiment
The comparison of Experiment
starts in the Project overview, more precisely in the Experiments tab.
From this view, you can select two or more Experiment
to be compared and click on the Compare button.
The Comparison view will then open. This view basically allows putting side-by-side each Metric available in the Log tab of the selected Experiment
.
In order to ensure the consistency of the comparison, you will be able to compare only Metric that have the same Metric type and the same Metric name. This way you are sure that you are not comparing a loss with a recall for instance.
In the below screenshot, one Metric with Line Type and the name val_cls_loss shared by the two compared Experiment
.
The comparison view displays all the Metrics for both Experiment
that share the same Metric Type and Metric name.
Depending on the Metric Type the visualization can differ, indeed for Line, Multi-line, Table, and Bar the Metric values of compared Experiment
are combined under a single item whereas for Single value, Image, and Heatmap, the Metric values are displayed side-by-side.
As you might have understood, the comparison of Experiment
is really valuable when compared Experiment
are using the same training script or at least training scripts that are logging the same kind of Metric on the Experiment Tracking dashboard.
You can compare as many Experiment
as you want but above a certain amount of Experiment
compared the visualization in the Comparison view might be impacted. This is why if you want to remove an Experiment
from the Comparison view, you just need to click on its name in the top right corner.
You now have all the insights you need to compare properly two Experiment
and assess which one suits your needs.
Updated 10 months ago