As specified in the initial message, the instance type used is e2-standard-4

Hey, did you checked that out ? None

Try to spin-up an auto-scaler app or any other Pro-only features

But the task appeared with the correct name and outputs in the pipeline and the experiment manager

THe image OS and the runner OS were both Ubuntu 22 if I remember

Hey @<1523701087100473344:profile|SuccessfulKoala55> this is a fairly small dataset with a linear hierarchy of ~300 version and a size of ~2GBs

In the meantime is there some way to set a retention policy for the dataset versions ?

Thanks a lot @<1523701435869433856:profile|SmugDolphin23> ❤

Oh, it's a little strange the comment lines about it were in the agent section

I suppose you cannot reproduct the issue from your side ?
Maybe it has to do that the faulty code was initially defined as a cached component

Nice, that's a great feature! I'm also trying to have a component executing Giskard QA test suites on model and data, is there a planned feature when I can suspend execution of the pipeline, and display on the UI that this pipeline "steps" require a human confirmation to go on or stop while displaying arbitrary text/plot information ?

Okay, turns out the output_uri in the constuctor was overriding the .setup_aws_upload() and not the other way arround

When running with PipelineDecorator.run_locally() I get the legitimate pandas error that I fixed by specifying the freq param in the pd.date_range(.... line in the component:
Launching step [generate_dataset] ClearML results page: `
[STEP 1/4] Generating dataset from autocut logs...
Traceback (most recent call last):
File "/tmp/tmp2jgq29nl.py", line 137, in <module>
results = generate_dataset(**kwargs)
File "/tmp/tmp2jgq29nl.py", line 18, in generate_dataset
...

No I was was pointing out the lack of one, but turns out on some model the iteration is so slow even on GPUs when training on a lots of time serie that you have to set the pytorch lightning trainer argument log_every_n_steps to 1 (default 50 ) to prevent the ClearML iteration logger from timing-out

So it seems to be an issue with the component parameter called in:
` @PipelineDecorator.pipeline(
name="VINZ Auto-Retrain",
project="VINZ",
version="0.0.1",
pipeline_execution_queue="Quad_VCPU_16GB"
)
def executing_pipeline(start_date, end_date):
print("Starting VINZ Auto-Retrain pipeline...")
print(f"Start date: {start_date}")
print(f"End date: {end_date}")

window_dataset_id = generate_dataset(start_date, end_date)

if name == 'main':
PipelineDec...

ClearML package version used: 1.9.1
ClearML Server: SaaS - Pro Tier

Turns out the bucket param expected was expecting the bucket name without the s3:// protocol specification, but now that this issue is fixed i still have the same incorrect region specified error ,

` task = Task.init(
project_name='XXXX',
task_name=f'Training-{training_uuid}',
task_type=Task.TaskTypes.training,
output_uri=f's3://{constants.CLEARML_BUCKET}'
)
task.setup_aws_upload(
bucket=constants.CLEARML_BUCKET,
regi...

Yup I already setup my aws configs for clearML that way but I needed to have generally accessible credentials too so I used the init script option in this config menu ^^

Okay I confirm having default parameters fixes that issue, but kinda sad to have lost 3 days into that super weird behavior

Well solved, it's not as beautiful but I guess i can put them in a env file with an arbitrary name in the init script and just pass that file as exec argument...

And running with a Python 3.10 interpreter

Oh okay, my initial implementation was not far off:
` task = Task.init(project_name='VINZ', task_name=f'VINZ Retraining {datetime.now().strftime("%Y-%m-%d %H:%M:%S")}')
task.set_progress(0)

print("Training model...")
os.system(train_cmd)
print("✔️ Model trained!")

task.set_progress(75)

print("Converting model to ONNX...")
os.system(f"python export.py --weights {os.path.join(training_data_path, 'runs', 'train', 'yolov5s6_results', 'weights', 'best.pt')} --img...

@<1523701205467926528:profile|AgitatedDove14> Yup I tested to no avail, a bit sad that there is no working integration with one of the leading time series framework...

Ooooo okay I see the @PipelineDecorator.pipeline decorator you can have a function to orchestrate your components and manipulate their return data

Sorry, I meant the scalar logging doesn't collect anything like it would do during a vanilla Pytorch Lightning training, here is the repo of the lib https://github.com/unit8co/darts

It doesnt seem so if you look at the REST api documentation, might be available as an ENterprise plan feature

BoredHedgehog47 here, enlight yourself https://github.com/allegroai/clearml/releases

Oh, that's nice, if I import a model using InputModel do I still need to specify a OutputModel ?

As opposed to the Controller/Task component where the add_step() only allows to sequentially execute them

upsy sorry didnt read the entire backtrace