Changelog

Picsellia SDK Python is a library that allows users to connect to Picsellia backend.

All notable changes to this project will be documented in this file.

[6.21.6] - 2025-05-14

Changed

• On requests.ConnectionError, all api calls will retry up to 4 times with an exponential backoff

[6.21.5] - 2025-04-17

Added

• DatasetVersion.compute_shapes_embeddings() will wait for job to end by default
• DatasetVersion.activate_visual_search() will wait for job to end by default

[6.21.4] - 2025-04-15

Experimental

• Features for shape embeddings on DatasetVersion were added

Added

• Datalake.embeddings_computation_status() to fetch status of embeddings computation on Datalake
• DatasetVersion.embeddings_computation_status() to fetch status of embeddings computation on DatasetVersion
• DatasetVersion.compute_shapes_embeddings() to start computation of shape embeddings on DatasetVersion
• DatasetVersion.delete_shape_embeddings() to delete computation of shape embeddings on DatasetVersion
• DatasetVersion.get_shapes_embeddings_status() to fetch status of shape embeddings computation on DatasetVersion
• DatasetVersion.list_rectangles_embeddings() to fetch rectangle embeddings on DatasetVersion
• DatasetVersion.list_polygons_embeddings() to fetch polygon embeddings on DatasetVersion

[6.21.3] - 2025-04-09

Added

• To prepare next release of picsellia-agents, we need some internal methods

[6.21.2] - 2025-03-31

Changed

• Upload of large files will be retried 3 times on Read Timeout
• Better handling of large annotation files on all import_annotations_file methods.

[6.21.1] - 2025-02-24

Fixed

• An annotation type hint was causing bug with python >= 3.11

Docs

• Some minor doc fixes

[6.21.0] - 2025-02-20

Some features around Visual Search !

Changed

• Remove detection_boxes in SegmentationPredictionFormat

Added

• Datalake.visual_search() can be called to search similar data
• Datalake.text_search() can be called to search data that match a text query
• Datalake.count_embeddings() can be called to count number of data indexed
• Datalake.list_embeddings() can be called to list embeddings of in your datalake
• DatasetVersion.activate_visual_search() can be called to activate visual search in your dataset
• DatasetVersion.deactivate_visual_search() can be called to deactivate visual search in your dataset
• DatasetVersion.count_embeddings() can be called to count number of data indexed in your dataset
• DatasetVersion.list_embeddings() can be called to list embeddings of in your dataset
• Data.embeddings property can be used to retrieve stored embeddings
• Asset.embeddings property can be used to retrieve stored embeddings

Fixed

• Deployment attribute _review_campaign_id was not well-defined in some cases

[6.20.0] - 2025-01-20

Happy new year you all !
Some new features around campaigns, some refactoring, some breaking changes in this version as deprecated code was removed.

Breaking

Behavior has changed for users that initialize Client() without giving organization_name or organization_id

• Users that belong to only one organization will be connected to this organization.

• Users that belong to multiple organizations but own one will be connected to owned organization.

• Users that belong to multiple organizations but do not own any organization won't be able to initialize an unspecified Client.

• AnnotationCampaign.list_steps() will return an AnnotationCampaignStep instead of a dict!

• Model.type and Model.framework were deprecated and are removed from now.

• Datalake.find_all_datas() was deprecated and is removed from now. You should use list_data() instead.

• DatasetVersion.find_all_assets() was deprecated and is removed from now. You should use list_assets() instead.

• ModelContext.retrieve_experiment() was deprecated and is removed from now. You should use get_experiment() instead.

• Deployment.check_feedback_loop_status() was deprecated and is removed from now.

• Deployment.disable_feedback_loop() was deprecated and is removed from now.

• Deployment.setup_continuous_training() will not use deprecated parameter dataset_version anymore.

• Experiment.publish() was deprecated and is removed from now. You should use export_as_model_version() instead

• ModelVersion.update_thumbnail was deprecated and is removed from now. You should use the web app instead.

Added

• AnnotationCampaignStep model, to manage your campaign step
• Client.get_processing() to find a processing by its name
• AnnotationCampaign.sync_assets() can be called to add assets that are not assigned as task in your campaign
• AnnotationCampaign.update() can be called with default_entry_step and annotated_entry_step parameter
• AnnotationCampaignStep.redistribute_assignments() can be called to add assets that are not assigned as task in your campaign
• AnnotationCampaignStep.update() can be called to update a step
• AnnotationCampaignStep.delete() can be called to add assets that are not assigned as task in your campaign
• Client.list_annotation_campaigns() can be called to list annotation campaigns of your organization
• Client.list_review_campaigns() can be called to list review campaigns of your organization
• Worker class has now a type : DATASET or DEPLOYMENT
• Worker.user_id property
• Deployment.list_workers() can be used to list workers of a deployment
• ReviewCampaign and ReviewCampaignStep models can be used to handle campaigns on Deployment ! Methods are more or less the same, except for some edge case behaviors.
• Client.create_processing() can be called with description parameter
• DatasetVersion.copy_assets_to() can be called to copy assets from a dataset to another

Changed

• DatasetVersion.list_assets() should behave better when a list of parameter is given
• Datalake.list_data() should behave better when a list of parameter is given
• DatasetVersion.add_data() will add data by batch, so it won't fail with 413 error when more than 300k assets are given
• DatasetVersion.add_data() will now wait for job to end by default
• DatasetVersion.fork() will now wait for job to end by default
• AnnotationCampaign.create_step() can be called with sample_rate parameter

Removed

• AnnotationCampaign do not require name anymore

Fixed

• ModelVersion str method was not displaying well its origin name
• Handle 413 errors (when payload or uri are too large)
• Some bad generated documentation should be fine now
• Some methods were not reading filenames correctly on Windows

[6.19.2] - 2024-11-06

Added

• DatasetVersion.find_asset() can be called with id parameter to find an asset by its id.

[6.19.1] - 2024-10-31

Happy Halloween !

Added

• Deployment.find_predicted_asset() to find a PredictedAsset from different criteria

[6.19.0] - 2024-10-30

We dropped support of python3.8, goodbye old friend, you will be missed.

Added

• Deployment.predict_shadow() to predict with shadow model on an already processed PredictedAsset
• Deployment.monitor_shadow() and Deployment.monitor_shadow_from_oracle_prediction_id() to monitor with shadow model on an already processed PredictedAsset
• Added support of python3.13
• Bumped some packages
• While dropping python3.8, we could get rid of old syntaxes
• Better handling of conflict when monitoring already processed data
• A GitHub workflow so we can send to our Slack channel a release note

Changed

• We welcomed ruff as a new pre commit linter
• As we do not allow multi campaign on the same dataset, we changed DatasetVersion.get_campaign()

Fixed

• We don't warn anymore if you have a future version of the SDK

Deprecated

• We dropped support of python3.8

[6.18.3] - 2024-10-01

Added

• New processing types allowed on the platform : data auto tagging, model compression, model conversion

Fixed

• Deployment.predict() and Deployment.predict_bytes() were failing when sending metadata

[6.18.2] - 2024-09-17

Fixed

• We refactored upload of large file to allow upload very large file on Google cloud storage

[6.18.1] - 2024-09-05

Fixed

• Added Asset.content_type, needed in Asset.to_data_schema()

[6.18.0] - 2024-09-04

Added

• Deployment.predict_cloud_image() can be called to predict onto an image stored on your object storage
• Deployment.predict_data() can be called to predict onto a data stored on your target datalake
• Deployment.monitor_cloud_image() can be called to monitor an image stored on your target datalake
• Deployment.monitor_data() can be called to monitor a data stored on your target datalake
• All Deployment.predict() can be called with metadata dict parameter. It will be stored on Data after being processed.
• Deployment.monitor() can be called with metadata dict parameter aswell.
• All Deployment.predict() can be called with monitor boolean parameter. It allows you to only call inference service, it won't call our monitoring service.

[6.17.1] - 2024-08-22

Fixed

• On DatasetVersion.build_coco_file_locally() area of segmentation will now be computed properly

[6.17.0] - 2024-07-29

Added

• Experiment.launch() now return a Job
• PredictedAsset class can be used to manage predictions of a Deployment.
• PredictedAsset.add_review class can be used to add review to assets of a Deployment.
• Deployment.list_predicted_assets() can be used to list assets of a Deployment.
• Processing class can be used to manage your processings
• Client.list_processings() can be used to list your processings.
• DatasetVersion.launch_processing() can be used to launch a processing on your DatasetVersion.

Fixed

• On Datalake.upload_data() with parameter fill_metadata=True could not pass validation of our api in some case, so we need to cast values.
• DatasetVersion.list_assets() with parameter filename_startswith was not working
• DatasetVersion.build_coco_file_locally() will only use the LAST annotation created on each asset. In campaign mode it caused some problems

Changed

• Some apis were deprecated on the backend side so we changed it on the SDK. These apis will be removed in the future : SDK versions prior to 6.17.0 might break.
• Allowed version of Pillow is now ">=9.2.0, <11"

[6.16.0] - 2024-06-05

Added

• DatasetVersion.list_assets() can now be called with a list of data ids
• DatasetVersion.import_annotations_coco_file() can now be called with use_id. If you exported your coco file with use_id, you can use import it now.
• DatasetVersion.import_annotations_coco_video_file() can be called to import video annotations in your dataset. You could export it from picsellia with the SDK since the 6.13.0
• Rectangle, Classification, Polygon, Line, Point have now text field that you can use as an OCR feature. You can call update on shape to update text field.
• DatasetVersion.import_annotations_coco_file and its video counterpart will read utf8_string field of json coco file and import it into the platform.

Fixed

• On Datalake.upload_data() with parameter fill_metadata=True could not pass validation of our api in some case, so we need to cast values.
• DatasetVersion.list_assets() with parameter filename_startswith was not working
• DatasetVersion.build_coco_file_locally() will only use the LAST annotation created on each asset. In campaign mode it caused some problems

Changed

• We upgraded pydantic dependency : we now depends on its version 2
• Default max pagination is reduced to 1000. It was 10000, it's a little bit too much
• We reduced to 1000 the max chunk_size on load_annotations()
• We refactored tests and their files

Deprecated

• We dropped support of python3.7
• All stuff related to Scan have been removed

[6.15.0] - 2024-05-06

Added

• Datalake.import_bucket_objects() can now be called with tags and source, that will be given to created data.

Fixed

• Typo on get campaign log
• wait_for_upload() had 2 beartype decorators
• Some tests were failing

[6.14.2] - 2024-04-19

Changed

• We now use session object to download and upload from presigned url. This should speed up a little bit, and not create enormous amount of connection.

[6.14.1] - 2024-04-16

Added

• Datalake.list_data() can be called with ids to fetch data from their ids.

Fixed

• When listing data and assets, filtering with tags and filenames was not working
• Downloadable objects such as Data, Asset and Artifact, were syncing before downloading, we don't want to do that anymore
• Type hint problem on AnnotationCampaign
• Documentation on AnnotationCampaign
• Tests on AnnotationCampaign

[6.14.0] - 2024-03-19

Added

• DatasetVersion.create_campaign() can be used to create an AnnotationCampaign.
• AnnotationCampaign.add_step() can be used to add a step to an AnnotationCampaign.
• AnnotationCampaign.launch() can be used to add assignments to all assets on this campaign of your DatasetVersion.
• AnnotationCampaign.update() can be used to update an AnnotationCampaign.
• DatasetVersion.get_campaign() can be used to retrieve the AnnotationCampaign of this DatasetVersion.
• DatasetVersion.export_annotation_file() can be called with parameter use_id=True. Generated files will use asset id insted of filename.
• Datalake.import_bucket_objects() can be used to import bucket objects of your S3 that are not yet on the platform.

Fixed

• Calling MultiAsset.as_multidata() with more than 10k assets was bugged
• Some dependencies were fixed on pyproject.toml, such as pyyaml, beartype and orjson. Python 3.11 and Python 3.12 can now be used with the sdk

[6.13.0] - 2024-02-26

Added

• Datalake.upload_data() can now handle unreadable images (like hdf5) and video ! By default, it will wait for data to be full processed by our services
• Datalake.create_projection(data) can be used to create a projection on a data : you have to give a name for this projection, it will be of type CUSTOM.
• DatasetVersion.export_annotation_file() can be called with boolean parameter export_video to only export video in annotation file

Fixed

• Default content type assigned to data we cannot read mime type is now application/octet-stream

[6.12.0] - 2024-01-17

Added

• Data, MultiData, Asset, MultiAsset can be downloaded with download(use_id=True) to use id as filename. Extension of base filename will be used.
• DatasetVersion.build_coco_file_locally() can now be called with parameter use_id=True to build a coco file with file_name keys as <id>.<extension>
• When initializing a Client, you can pass parameter session with your own requests.Session object, that will be used on each request done by the sdk. It allows configuring a proxy or custom headers.

[6.11.0] - 2024-01-10

Happy new year everyone !
Some minor features and fixes in this version

Added

• Data.update_metadata() can be called to update metadata of a data.
• DatasetVersion.load_annotations() can now be called with parameter assets to build a json coco file with only given assets.

Fixed

• When downloading a file, there was a race condition when creating parent directories of downloaded file.
• Storing an artifact with do_zip=True was not creating expected archive name when a dot was present in filename.
• On upload, large file discriminant is now 25MiB and not 5MiB

Changed

• Major changes in docstrings, a lot of fixes!

[6.10.3] - 2023-09-22

Fixed

• PredictionFormat will not raise an exception if one of its list is empty. Deployment.monitor() can now be called with empty predictions.

[6.10.2] - 2023-09-06

Fixed

• Deployment.predict_bytes() needs parameter filename to work. It was sending prediction with filename media.

[6.10.1] - 2023-08-31

Added

• DatasetVersion.build_coco_file_locally() now adds area of bounding boxes on generated files for Object Detection.

[6.10.0] - 2023-08-21

Some deprecation on privacy object creation, it is not possible to create a public dataset, project or model from the sdk anymore.
Attach dataset version to Feedback Loop and Continuous Training settings of your deployments.

Added

• Deployment.attach_dataset_to_feedback_loop() can now be called to attach a dataset to Feedback Loop settings of your deployment. Attached datasets can be filled with reviewed prediction from your deployment.
• Deployment.detach_dataset_from_feedback_loop() allows detaching dataset from Feedback Loop.
• Deployment.list_feedback_loop_datasets() will list attached datasets of your Feedback Loop.
• Deployment.attach_dataset_to_continuous_training() can now be called to attach a dataset to Continuous Training settings of your deployment. Attached datasets will be added to your experiment created when Continuous Training is triggered.
• Deployment.detach_dataset_from_continuous_training() allows detaching dataset from Continuous Training.

Deprecated

• Parameter dataset_version of Deployment.setup_feedback_loop()
• Parameter dataset_version and model_version of Deployment.setup_continuous_training()
• Parameter private of Client.create_model()
• Parameter private of Model.update()
• Parameter private of Client.create_project()
• Parameter private of Project.update()
• Parameter private of Client.create_dataset()
• Parameter private of Dataset.update()

[6.9.0] - 2023-07-10

Added

• Datalake.upload() can now be called with parameter fill_metadata. On upload, set this parameter to True to read exif metadata flags and push metadata to Picsellia. By default, fill_metadata is False.
• Client.get_datalake() can now be called with parameter id or name to retrieve the Datalake you want. If nothing is given, this method will retrieve your default datalake.
• Deployment.predict_bytes() can now be called if you want to send image as bytes on Serving service
• ModelVersion.update() and Model.create_version() can now be called with docker_tag parameter

[6.8.0] - 2023-06-12

Added

• Datalake.upload_data() now accept parameter metadata which is a dict (or a list of dict if there are multiple filepaths) matching common metadata allowed by Picsellia.
• DatasetVersion.fork() can now be called with parameters with_annotations and with_labels as web application
• DatasetVersion.export_annotation_file() can now be called with AnnotationFileType.YOLO to retrieve a zip with yolo files
• Dataset.create_model() and DatasetVersion.update() have parameter description to update description of a ModelVersion

Fixed

• When adding evaluation, allow empty list to be sent as a list of shape

[6.7.1] - 2023-05-15

Some changes on monitor() following an update of our monitoring stack.

Added

• Property type of a deployment.
• Method Deployment.monitor_bytes() can be called if you have an image as bytes to send to monitoring.

Changed

• Deployment.monitor() do not use parameters model_version and shadow_model_version anymore
• Parameter shadow_raw_predictions of Deployment.monitor() will be replaced by parameter shadow_prediction in the future
• You can add content_type parameter to monitor() if you don't want sdk to infer it with mimetypes. It will be checked with common content types of the mimetypes library
• content_type of monitor_bytes() is mandatory and should be a SupportedContentType enum or string. Current common type supported are "image/png" and "image/jpeg"

[6.7.0] - 2023-05-05

Added

• Add Client.create_deployment() to create a deployment. Allow user to create it without using Picsellia Serving
• Add DatasetVersion.retrieve_stats() to retrieve stats of your dataset version
• Add DatasetVersion.train_test_val_split() to split a dataset version into 3 different multi assets
• Add DatasetVersion.split_into_multi_assets() to split a dataset version into N multi assets and return their label repartition
• Add MultiData.split() and MultiAsset.split() to split from a given ratio a multi asset
• Add User-Agent with picsellia version in headers of requests

Changed

• Framework and type of Model are now configurable into ModelVersion
• Get by id now use parent API to ensures object are in the same organization as the one connected
• Methods manipulating tags on client are now calling other routes
• Some minor fixes on documentation

Fixed

• Segmentation format used in monitor() was not supported by monitoring service

[6.6.0] - 2023-04-06

Added

• list_data and list_assets have a new parameter q that can be used the same way the query language is used in the web platform
• Deployment has new methods: set_training_data, check_training_data_metrics_status and disable_training_data_reference, that can be used for monitoring and unsupervised metrics.
• as_multidata of MultiAsset can now be called with parameter
• Artifact, Data, Asset, LoggingFile, ScanFile, ModelFile are now inheriting from Downloadable, and have url property that can be used to download files. These urls are presigned and expired at some point in the future.
• Methods add_evaluation, list_evaluations and compute_evaluations_metrics of Experiment can be used to add, list and compute evaluation of an Experiment

Changed

• Deployment Feedback Loop dataset is now only used as a recipient for new assets submitted after review in the predictions dashboard
• bbox of COCO annotation cannot be a tuple anymore

[6.5.0] - 2023-03-15

Added

• Jobs are now handled differently in Picsellia platform
• get_dataset_version_by_id can be done in Client
• get_model_version_by_id can be done in Client
• get_experiment_by_id can be done in Client
• Import yolo files with import_annotations_yolo_files
• upload_data can be called with an ErrorManager to retrieve errors after upload.

Fixed

• Pascal VOC files parsing allows some fields to not be set
• 502 errors were not handled correctly
• Uploading images that were transposed now correctly send width and height on the platform

Changed

• Downloading files has a new retry policy
• When importing YOLO, COCO or Pascal VOC files, you need to set type of dataset before.
• Some refactor on import annotations, it should be faster now !

[6.4.2] - 2023-03-03

Changed

• Return line after logging chapter
• Allow parameter enforced_ordered_categories on build_coco_file_locally() to enforce categories of built coco file.

Fixed

• Do a paginated call in dataset version train_test_split when loading assets
• Method set_type of dataset version was failing when logging result if a string was given

[6.4.1] - 2023-02-17

Fixed

• Some methods were using types not compatible with python older than 3.8

[6.4.0] - 2023-02-16

Added

• DatasetVersion list_assets() will call XGET method when filtering on filenames or object_names, to not have the limit size query error.

Fixed

• Datalake upload_data() now allow effectively a Datasource
• Import annotations was limited to 10 000 annotations because find_assets() was used. It is now using list_assets() which is paginated.

Changed

• Deployment update() cannot change active anymore.

Deprecated

• DatasetVersion find_all_assets()

Experimental

• DatasetVersion build_coco_file_locally() will create a COCOFile object that can be written in a json file

[6.3.2] - 2023-01-31

Added

• Experiment.export_in_existing_model() method will create a version of given model when exporting experiment.
• Datalake list_data() can be used to filter on object_names.

Fixed

• ModelVersion.store() was not using replace parameter correctly, it will now delete existing model file if it exists

Changed

• All max_workers parameters are now None by default, to use cpu count + 4 as default value
• Datalake list_data() will call XGET method when filtering on filenames or object_names, to not have the limit size query error.

Removed

• AnnotationStatus.REVIEWED status was never really used and is now removed from Picsellia.

[6.3.1] - 2023-01-20

This is a patch to fix download dataset on huge datasets

Fixed

• as_multidata now uses xget to retrieve all datas with given list of ids

Changed

• as_multidata is not called in download from DatasetVersion

[6.3.0] - 2023-01-19

Happy new year from the Picsellia Team !
This minor version add some useful methods and fix specified COCOFile format.

Added

• List public models with list_public_models() from the Picsellia Hub in Client.
• Find public model with find_public_model() from the Picsellia Hub in Client.
• Convert MultiAsset into MultiData with as_multidata() and as_list_of_data()
• Allow asset tags to be added when adding data to a dataset with add_data()
• Method __repr__ has been added to MultiObjects
• Property labels in ModelVersion objects
• Add get_dataset in ModelContext to get DatasetVersion from its attached name

Fixed

• breaking: COCOFile was wrongly specified: in annotations, only one segmentation was allowed for each shape,
  but COCOFile allows multiple polygons. Now you need to give a list of list of coordinates instead of only one list of coordinates.
  import_annotations_coco_file() should now work fine with COCO files with segmentation.
• Add AuthenticationError again in exceptions, which was used by picsellia-tf2

Changed

• may be breaking: Default value of force_create_label is now True in import_annotations_coco_file and import_annotation_voc_file.
• Rename retrieve_experiment into get_experiment in ModelContext
• retrieve_datasets in ModelContext now return a dict with attached name as key and dataset version as value
• download() a DatasetVersion is not calling assets extended endpoints which was taking a lot of time on huge datasets.
• Imports order with pre-commit cln tool
• Doc generation with all properties docstring

[6.2.1] - 2022-27-12

Fixed

• Fixed .predict() behavior by removing mandatory parameters tags and source

[6.2.0] - 2022-27-12

Added

• Add docker env variables support on ModelVersion
• Add an experiment.log_parameters() to use experiment.log() with parameters more easily
• Add possibility to attach base parameters when experiment.attach_model()
• Add feedbackloop method check status
• Add __repr__ methods to DAO objects, for better UX representation
• Add possibility to set tags and source when sending a prediction to serving
• Add get_or_create_datasource() in client

Changed

• Rename experiment.publish() into experiment.export_as_model()
• Some changes on feedbackloop methods
• Method list_data and list_assets can now use parameter intersect_tags to find objects that have all tags given
• Allow Path object in every method that accept path as string
• Some exceptions were renamed, some useless were removed

[6.1.2] - 2022-11-10

Fixed

• Use of typing.List instead of list to support python < 3.9

[6.1.1] - 2022-11-10

Added

• Add convert_tags_to_classification() in dataset version
• Add get_data_tags() on asset

Changed

• Github workflow

[6.1.0] - 2022-11-01

Added

• Add ONXX in Framework enums
• Possibility to use strings instead of enums in methods

Changed

• Make duration optional in annotation creation/update/overwrite
• Make duration int compatible
• log() method now create, update or append a log, in only one call to backend
• Remove create_log method
• Optional type hint is now used everywhere
• Prevent user from updating log image via log.update(), only experiment.log can be used for this case

Fixed

• In log() method, image type data stored bad values

[6.0.2] - 2022-10-17

Added

• Logging File for an Experiment

Changed

• A lot of typos, variable naming, minor formatting
• String .format() into f-strings
• Old package dependencies
• train_test_split() return MultiAsset instead of List[Asset]

Fixed

• Regeneration of JWT when expired
• When downloading file, open file only if response is ok

[6.0.1] - 2022-10-04

Added

• CHANGELOG file.

Changed

• Fixed test_train_split with breaking changes of query language in list assets.
• Documentation of core functions, minor typos fixes.
• Workflow for testing on staging when push on development and main.

[6.0.0] - 2022-10-03

Added

• Annotation are now objects storing Rectangle, Polygon, Classification, Point and Line for an Asset
• Artifact are now Experiment files stored
• DatasetVersion are now used as versions of Dataset
• Datalake objects
• Datasource objects
• Job objects, to wait for tasks to end
• ModelVersion are now used as versions of Model
• ScanFiles are now Scan files stored

Changed

• Pictures renamed to Asset that are Data objects in a DatasetVersion
• Multithreading usage
• Tags
• Annotations with objects