User Guides
Python SDK Reference
Python SDK Reference

Data

Properties

  • url Url
    This is generated by backend and expires after 1 hour. So this property might be out of date.
    Calling sync() method will retrieve a new url when expired.
  • id Id
  • object_name Object name
  • filename Filename
  • large If true, this (Object) has a large size

  • datalake_id UUID of Datalake where this Data is

  • object_name Object name of this Data

  • content_type Content type of this Data

  • filename Filename of this Data

  • large If true, this Data file is considered large

  • type Type of this Data

  • width Width of this Data

  • height Height of this Data

  • duration This property is no longer supported

  • metadata Metadata of this Data. Can be None

  • upload_status Status of upload of this Data. You can only use your data if this value is DONE.

Methods

reset_url

reset_url()

Description

Reset url of this object

download

download(
   target_path: Union[str, Path] = './', force_replace: bool = False,
   use_id: bool = False
)

Description

Download this object into given target_path

Examples

data = clt.get_datalake().list_data(limit=1)
data.download('./data/')

Arguments

  • target_path (str, optional) : Target path where data will be downloaded. Defaults to './'.

  • force_replace (bool, optional) : Replace an existing file if exists. Defaults to False.

  • use_id (bool, optional) : If true, will download file with id and extension as file name. Defaults to False.

add_tags

add_tags(
   tags: Union[Tag, list[Tag]]
)

Description

Add some tags to an object.
It can be used on Data/MultiData/Asset/MultiAsset/DatasetVersion/Dataset/Model/ModelVersion.

You can give a Tag or a list of Tag.

Examples

tag_bicycle = client.create_tag("bicycle", Target.DATA)
tag_car = client.create_tag("car", Target.DATA)
tag_truck = client.create_tag("truck", Target.DATA)

data.add_tags(tag_bicycle)
data.add_tags([tag_car, tag_truck])

remove_tags

remove_tags(
   tags: Union[Tag, list[Tag]]
)

Description

Remove some tags from an object (can be used on Data/Asset/DatasetVersion/Dataset/Model/ModelVersion)

You can give a Tag or a list of Tag.

Examples

data.remove_tags(tag_bicycle)
data.remove_tags([tag_car, tag_truck])

reset_url

reset_url()

Description

Reset url property of this Data by calling platform.

Returns

A url as a string of this Data.

is_ready

is_ready()

Description

wait_for_upload_done

wait_for_upload_done(
   blocking_time_increment: float = 1.0, attempts: int = 20
)

Description

get_tags

get_tags()

Description

Retrieve the tags of your data.

Examples

tags = data.get_tags()
assert tags[0].name == "bicycle"

Returns

List of Tag objects.

get_datasource

get_datasource()

Description

Retrieve DataSource of this Data if it exists. Else, will return None.

Examples

data_source = data.get_datasource()
assert data_source is None

Returns

A DataSource object or None.

delete

delete()

Description

Delete data and remove it from datalake.

⚠️ DANGER ZONE: Be very careful here!

Remove this data from datalake, and all assets linked to this data.

Examples

data.delete()

update_metadata

update_metadata(
   metadata: Union[None, dict, list[dict]]
)

Description