There are only 2 chunks because we don't split large files into multiple chunks

Hi @<1654294820488744960:profile|DrabAlligator92> ,

Regarding the model metadata, you don't need to actually construct the types, just use a list of dictionaries, and they will be casted automatically, for example:

client.models.add_or_update_metadata(model="<model-id>", metadata=[{"key": "foo", "type": "int", "value": "1"}])

Regarding reporting scalars, you can, but it won't be as easy as using the SDK calls - basically you can send a scalar event referencing the model (i.e. using client.events.add() )

Regarding the compression, you need to use a zipfile constant, not a string, import it using from zipfile import ZIP_DEFLATED

Hi @<1523701435869433856:profile|SmugDolphin23> Thanks for response.
As shared above i am initializing the chunk_size = 10 (i.e. 10MB)

dataset.upload(show_progress=True,chunk_size=10)

Uploading 2 files :
file1.npz -> 1.63 MB
file2.npz -> 315.01 MB

In logs i get below.
File compression and upload completed: total size 316.64 MiB, 2 chunk(s) stored (average size 158.32 MiB)

Now as per the explanation the chunk size is set to 10MB
First chunk will fill upto 10MB (file1.npz is 1.63 MB + Include file2.npz).
Second chunk will fill upto 10MB.
and so on.....
So here there should be more than 2 chunks right ?

@<1523701087100473344:profile|SuccessfulKoala55> Thanks for your response.
Currently we are targeting the APICLIENT for creating a model and adding scalar value.

from clearml.backend_api.session.client import APIClient
client = APIClient()
model_instance = client.models.create(name="sample_model",project="model_id",uri="sample_url")

The above code will create and return the model id.
It would be helpful if we can get a sample documentation or sample code snippet how we can use None
to add scalar value for above created model.

Hi @<1654294820488744960:profile|DrabAlligator92> ! The way chunk size works is:
the upload will try to obtain zips that are smaller than the chunk size. So it will continuously add files to the same zip until the chunk size is exceeded. If the chunk size is exceeded, a new chunk (zip) is created. The initial file in this chunk is the file that caused the previous size to be exceeded (regardless of the fact that the file itself might exceed the size).
So in your case: am empty chunk is created. file1.npz is added to the chunk. The size is not exceeded. Then file2.npz would be added to the same zip, but the file is too large, so the chunk size would be exceeded. A new chunk is then created, containing file2.npz