Okay I confirm having default parameters fixes that issue, but kinda sad to have lost 3 days into that super weird behavior

Can reproduce on Pro SaaS deployment on Firefox 105.0.3

The new 1.7.2 is still in release candidates so nothing new since 20 days ago

THe image OS and the runner OS were both Ubuntu 22 if I remember

You can specify default storage string on projects pointing to for instance a S3 bucket

@<1523701205467926528:profile|AgitatedDove14> Yup I tested to no avail, a bit sad that there is no working integration with one of the leading time series framework...

I'm looking for the github issue so I can check the PR fixing that, do you have any info on that ?

Seems pretty critical tho, SuccessfulKoala55 is the 1.7.2 releasing soon ? ( I see the rc2 has been built 3 days ago)

Is there an example of this somewhere ? Cause I'm training a YOLOv5 model which already has ClearML intergration built-in but it seems to be hardcoded to attach its task to a Yolov5 project and upload .pt file as artifact while I want to upload a converted .onnx weights with custom tags to my custom project

AgitatedDove14 Got that invalid region error on the set_upload_destination() while the region ( aws-global ) I specified in my agent config worked fine to retrieve a dataset from the same bucket
` 2022-11-04 15:05:40,784 - clearml.storage - ERROR - Failed testing access to bucket XXXXX: incorrect region specified for bucket XXXX (detected region eu-central-1)
Traceback (most recent call last):
File "/root/.clearml/venvs-builds/3.8/lib/python3.8/site-packages/clearml/model.py", l...

And after your modifications are made you can use . https://clear.ml/docs/latest/docs/references/sdk/dataset/#datasetsquash to squash your modified subset with the main dataset if you want to re-integrate it in your flow. But I don't remember if squash requires the both datasets to be present locally or not...

Okay, turns out the output_uri in the constuctor was overriding the .setup_aws_upload() and not the other way arround

@<1523701087100473344:profile|SuccessfulKoala55> I had already bumped boto3 to its latest version and all the files I added to the dataset were pickle binary files

Turns out the bucket param expected was expecting the bucket name without the s3:// protocol specification, but now that this issue is fixed i still have the same incorrect region specified error ,

` task = Task.init(
project_name='XXXX',
task_name=f'Training-{training_uuid}',
task_type=Task.TaskTypes.training,
output_uri=f's3://{constants.CLEARML_BUCKET}'
)
task.setup_aws_upload(
bucket=constants.CLEARML_BUCKET,
regi...

ClearML package version used: 1.9.1
ClearML Server: SaaS - Pro Tier

When running with PipelineDecorator.run_locally() I get the legitimate pandas error that I fixed by specifying the freq param in the pd.date_range(.... line in the component:
Launching step [generate_dataset] ClearML results page: `
[STEP 1/4] Generating dataset from autocut logs...
Traceback (most recent call last):
File "/tmp/tmp2jgq29nl.py", line 137, in <module>
results = generate_dataset(**kwargs)
File "/tmp/tmp2jgq29nl.py", line 18, in generate_dataset
...

So it seems to be an issue with the component parameter called in:
` @PipelineDecorator.pipeline(
name="VINZ Auto-Retrain",
project="VINZ",
version="0.0.1",
pipeline_execution_queue="Quad_VCPU_16GB"
)
def executing_pipeline(start_date, end_date):
print("Starting VINZ Auto-Retrain pipeline...")
print(f"Start date: {start_date}")
print(f"End date: {end_date}")

window_dataset_id = generate_dataset(start_date, end_date)

if name == 'main':
PipelineDec...

(currently I am a SaaS customer in Pro tier)

CostlyOstrich36 Should I start a new issue since I pinpointed the exact problem given than the beginning of this one was clearly confusing for both of us ?

Hey, I'm a SaaS user in PRO tier and I was wondering if it was a feature available on the auto-scaler apps so I could improve the cost-efficiency of my provisionned GCP A100 instances

I mean, if deleting tags in finalized datasets is possible in the GUI, it should be too in the SDK but I don't see the method

The pipeline log indicate the same version of Pandas ( 1.5.0 ) is installed, I really don't know what is happening

I was launching a pipeline run, but I don't remember having set the autoscaler to use spot instances (I believe the GCP terminology for spot instance is "preemptible" and I set it to false)

I suppose you cannot reproduct the issue from your side ?
Maybe it has to do that the faulty code was initially defined as a cached component

@<1523701087100473344:profile|SuccessfulKoala55> here you go

Component's prototype seems fine:
@PipelineDecorator.component( return_values=['dataset_id'], cache=False, task_type=TaskTypes.data_processing, execution_queue='Quad_VCPU_16GB', ) def generate_dataset(start_date: str, end_date: str, input_aws_credentials_profile: str = 'default'):

Well its not working, this params seems to be used to override the repo to pull since it has a str type annotation anyway, ClearML still attempted to pull the repo

(if for instance in wanna pull a yolov5 repo in the retraining component)

Linked github issue: None

Oh, it's a little strange the comment lines about it were in the agent section