Pull and Patch Logs & Artifacts
Here we will learn how to interact with the artifacts and logs that you have
saved to the platform.
Working with Logs assets
List all your Logs assets
If you want to retrieve all the data assets you saved for one experiment, here is how:
from picsellia import Client
api_token = 'YOUR TOKEN'
project_name = 'My awesome Project'
client = Client(api_token)
project = client.get_project(project_name)
experiment = project.get_experiment(
name='my_new_experiment'
)
experiment.list_logs()
Delete all your logs assets
If you think that your experiments need a bit of cleansing, you can delete all the data assets you saved at once like this:
experiment.delete_all_logs()
Retrieve an asset
If you want to retrieve a particular log asset, let's say the parameters of your last training, here is how you can do it:
You will only return what is stored in the data field of the data asset, not all the information about the asset.
experiment.get_log('parameters')
{'steps': 200000,
'nb_gpu': 1,
'batch_size': 8,
'learning_rate': 0.005,
'annotation_type': 'classification'}
Updating an asset
Now let's say that you want to change the value list of parameters for this training, here is how:
log = experiment.get_log('parameters')
parameters = {
'steps': 5e6,
'nb_gpu': 8,
'batch_size': 64,
'learning_rate': 0.0055,
'annotation_type': 'detection'
}
log.update(name='parameters', data=parameters)
The method returns the updated object.
{
'id': 72,
'date_created': '2021-02-09T12:32:18.293746Z',
'last_update': '2021-02-09T12:32:18.293556Z',
'name': 'parameters',
'data': {'steps': 5000000.0,
'nb_gpu': 8,
'batch_size': 64,
'learning_rate': 0.0055,
'annotation_type': 'detection'},
'type': 'table'
}
Delete an asset
If you want to completely remove a data asset from all your visualisations, here is how:
log = experiment.get_log('parameters')
log.delete('parameters')
Working with Artifacts
List all your Artifacts
If you want to retrieve all the file assets you saved for one experiment, here is how:
from picsellia import Client
api_token = 'YOUR TOKEN'
project_name = 'My awesome Project'
client = Client(api_token)
project = client.get_project(project_name)
experiment = project.get_experiment(
name='my_new_experiment'
)
experiment.list_artifacts()
The method returns the list of artifacts.
[{'id': 22,
'date_created': '2021-02-09T12:32:18.022694Z',
'last_update': '2021-02-09T12:32:18.022465Z',
'large': False,
'name': 'config',
'object_name': '9a141ede-03dc-4e6c-a695-38661d9a97c3/pipeline.config'},
{'id': 23,
'date_created': '2021-02-09T12:32:18.068900Z',
'last_update': '2021-02-09T12:32:18.068723Z',
'large': True,
'name': 'model-latest',
'object_name': '9a141ede-03dc-4e6c-a695-38661d9a97c3/0/saved_model.zip'},
{'id': 24,
'date_created': '2021-02-09T12:32:18.112582Z',
'last_update': '2021-02-09T12:32:18.112410Z',
'large': True,
'name': 'checkpoint-data-latest',
'object_name': '9a141ede-03dc-4e6c-a695-38661d9a97c3/ckpt-0.data-00000-of-00001'},
{'id': 25,
'date_created': '2021-02-09T12:32:18.156945Z',
'last_update': '2021-02-09T12:32:18.156767Z',
'large': False,
'name': 'checkpoint-index-latest',
'object_name': '9a141ede-03dc-4e6c-a695-38661d9a97c3/ckpt-0.index'}]
Delete all your artifacts
If you think that your experiments need a bit of cleansing, you can delete all the artifacts you saved at once like this:
experiment.delete_all_artifacts()
Retrieve and download only one artifacts
If you only need to retrieve information about a file, you can use the following method:
experiment.get_artifact('config')
{'id': 22,
'date_created': '2021-02-09T12:32:18.022694Z',
'last_update': '2021-02-09T12:32:18.022465Z',
'large': False,
'name': 'config',
'object_name': '9a141ede-03dc-4e6c-a695-38661d9a97c3/pipeline.config'}
If you need to download the artifact, you can use the following method:
config = experiment.get_artifact('config')
config.download('config')
Dealing with large artifacts
If the size of the artifacts exceeds 5Mb or if you get errors while using
download(), you might need to set thelargeparameter toTrue.
config = experiment.get_artifact('config')
config.download('config', large=True)
Alternatively, you can download the artifacts to a specified folder, just set the path parameter to the path of the folder (must be an existing folder).
experiment.download('config', path='training')
Update an artifact
The update function is there if you need to perform an update on artefact information such as its name.
config = experiment.get_artifact('config')
config.update('config', name='new-config')
Be careful
If you call the store method again, it will automatically erase the old file and replace it with the new one.
Delete an artifact
If you want to remove a file asset from one of your experiments, here is how to do it:
from picsellia import Client
api_token = 'YOUR TOKEN'
project_name = 'My awesome Project'
client = Client(api_token)
project = client.get_project(project_name)
experiment = project.get_experiment(
name='my_new_experiment'
)
artifact = experiment.get_artifact('config')
artifact.delete()
Updated 4 months ago