Pull and Patch Logs & Artifacts

Here we will learn how to interact with the artifacts and logs that you have
saved to the platform.

Working with Logs assets

List all your Logs assets

If you want to retrieve all the data assets you saved for one experiment, here is how :

from picsellia import Client

api_token = 'YOUR TOKEN'
project_name = 'My awesome Project'

client = Client(api_token)
project = client.get_project(project_name)

experiment = project.get_experiment(
    name='my_new_experiment'
    )
experiment.list_logs()

Delete all your logs assets

If you think that your experiments need a bit of cleansing, you can delete all the data assets you saved at once like this :

experiment.delete_all_logs()

Retrieve an asset

If you want to retrieve a particular log asset, let's say the parameters of your last training, here is how you can do it :

πŸ“˜

You will only return what is stored in the data field of the data asset, not all the information about the asset.

experiment.get_log('parameters')
{'steps': 200000,
 'nb_gpu': 1,
 'batch_size': 8,
 'learning_rate': 0.005,
 'annotation_type': 'classification'}

Updating an asset

Now let's say that you want to change the value list of parameters for this training, here is how :

parameters = {
 'steps': 5e6,
 'nb_gpu': 8,
 'batch_size': 64,
 'learning_rate': 0.0055,
 'annotation_type': 'detection'
}
experiment.update_log(name='parameters', data=parameters)

The method returns the updated object

{
 'id': 72,
 'date_created': '2021-02-09T12:32:18.293746Z',
 'last_update': '2021-02-09T12:32:18.293556Z',
 'name': 'parameters',
 'data': {'steps': 5000000.0,
  'nb_gpu': 8,
  'batch_size': 64,
  'learning_rate': 0.0055,
  'annotation_type': 'detection'},
 'type': 'table'
 }

Delete an asset

If you want to completely remove a data asset from all your visualizations, here is how :

experiment.delete_log('parameters')

Working with Artifacts

List all your Artifacts

If you want to retrieve all the file assets you saved for one experiment, here is how :

from picsellia import Client

api_token = 'YOUR TOKEN'
project_name = 'My awesome Project'

client = Client(api_token)
project = client.get_project(project_name)

experiment = project.get_experiment(
    name='my_new_experiment'
    )
experiment.list_artifacts()

The method returns the list of artifacts

[{'id': 22,
  'date_created': '2021-02-09T12:32:18.022694Z',
  'last_update': '2021-02-09T12:32:18.022465Z',
  'large': False,
  'name': 'config',
  'object_name': '9a141ede-03dc-4e6c-a695-38661d9a97c3/pipeline.config'},
 {'id': 23,
  'date_created': '2021-02-09T12:32:18.068900Z',
  'last_update': '2021-02-09T12:32:18.068723Z',
  'large': True,
  'name': 'model-latest',
  'object_name': '9a141ede-03dc-4e6c-a695-38661d9a97c3/0/saved_model.zip'},
 {'id': 24,
  'date_created': '2021-02-09T12:32:18.112582Z',
  'last_update': '2021-02-09T12:32:18.112410Z',
  'large': True,
  'name': 'checkpoint-data-latest',
  'object_name': '9a141ede-03dc-4e6c-a695-38661d9a97c3/ckpt-0.data-00000-of-00001'},
 {'id': 25,
  'date_created': '2021-02-09T12:32:18.156945Z',
  'last_update': '2021-02-09T12:32:18.156767Z',
  'large': False,
  'name': 'checkpoint-index-latest',
  'object_name': '9a141ede-03dc-4e6c-a695-38661d9a97c3/ckpt-0.index'}]

Delete all your artifacts

f you think that your experiments need a bit of cleansing, you can delete all the artifacts you saved at once like this :

experiment.delete_all_artifacts()

Retrieve and download only one artifacts

If you only need to retrieve information about a file you can use the following method

experiment.get_artifact('config')
{'id': 22,
 'date_created': '2021-02-09T12:32:18.022694Z',
 'last_update': '2021-02-09T12:32:18.022465Z',
 'large': False,
 'name': 'config',
 'object_name': '9a141ede-03dc-4e6c-a695-38661d9a97c3/pipeline.config'}

If you need to download the artifact you can use the following method

experiment.download('config')

πŸ“˜

Dealing with large artifacts

If the size of the artifact exceeds 5Mb or if you get errors while using download(), you might need to set the large parameter to True

experiment.download('config', large=True)

Alternatively, you can download the artifacts to a specified folder, just set the path parameter to the path of the folder (must be an existing folder)

experiment.download('config', path='training')

Update an artifact

The update function is there if you need to perform an update on artifact information such as its name.

experiment.update_file('config', name='new-config')

❗️

Be careful

If you call the store method again, it will automatically erase the old file and replace it with the new one

Delete an artifact

If you want to remove a file asset from one of your experiments, here is how to do it.

from picsellia import Client

api_token = 'YOUR TOKEN'
project_name = 'My awesome Project'

client = Client(api_token)
project = client.get_project(project_name)

experiment = project.get_experiment(
    name='my_new_experiment'
    )

experiment.delete_artifact('model-latest')
```'


[block:callout]
{
  "type": "danger",
  "body": "Be aware that this will also permanently delete  the file from our storage.",
  "title": "Danger Zone"
}
[/block]

{% hint style="danger" %}
**** 
{% endhint %}

 

Did this page help you?