You need to set a specific port for your clearml server and then just set a rule in your reverse proxy (eg. nginx) to make point the specific route you want toward that port

@<1523701087100473344:profile|SuccessfulKoala55> here you go

Hey, I'm a SaaS user in PRO tier and I was wondering if it was a feature available on the auto-scaler apps so I could improve the cost-efficiency of my provisionned GCP A100 instances

alias : "Alias of the dataset. If set, the ‘alias : dataset ID’ key-value pair will be set under the hyperparameters section ‘Datasets’"

I have to concede that I found that description a bit vague at first but if you check that https://clear.ml/docs/latest/docs/clearml_data/best_practices#organize-datasets-for-easier-access you see that:

"In cases where you use a dataset in a task (e.g. consuming a dataset), you can easily track which dataset the task is using by using ` Dataset.get...

Yup, so if I understand this is strictly an Enterprise feature and is not planned to be available in the Pro version ?

I would like instead of having to:
Fetch latest dataset to get the current latest version Increment the version number Create and upload a new version of the datasetTo be able to:
Select a dataset project by name Create a new version of the dataset by choosing what increment in SEMVER standard I would like to add for this version number (major/minor/patch) and upload

CostlyOstrich36 Having the same issue running on a remote worker, even tho the line works correctly on python interpreter and the component run correctly in local debug mode (but not standard local mode):
` File "/root/.clearml/venvs-builds/3.10/code/generate_dataset.py", line 18, in generate_dataset
time_range = pd.date_range(start=start_date, end=end_date, freq='D').to_pydatetime().tolist()
File "/root/.clearml/venvs-builds/3.10/lib/python3.10/site-packages/pandas/core/indexes/date...

Thus the main difference of behavior must be coming from the _debug_execute_step_function property in the Controller class, currently skimming through it to try to identify a cause, did I provide you enough info btw CostlyOstrich36 ?

Oh, that's nice, if I import a model using InputModel do I still need to specify a OutputModel ?

There is a gap in the GPU offer on GCP and there is no modern middle-ground for a TPU with more than 16GB GRAM and less than 40GB, so sometime we need to provision a A100 to get the training speed we want but we don't use all the GRAM so I figured out if we could batch 2 training tasks on the same A100 instance we would still be on the winning side in term of CUDA cores and getting the most of the GPU-time we're paying.

The train.py is the default YOLOv5 training file, I initiated the task outside the call, should I go edit their training command-line file ?

But the task appeared with the correct name and outputs in the pipeline and the experiment manager

Okay I confirm having default parameters fixes that issue, but kinda sad to have lost 3 days into that super weird behavior

I'm looking for the github issue so I can check the PR fixing that, do you have any info on that ?

Wll idk the scalars are not reported and I get this message ClearML Monitor: Could not detect iteration reporting, falling back to iterations as seconds-from-start , i'll go open the pull request for that right away

And by extension is there a way to upsert a dataset by automatically creating an entry wich a incremented version or create it if it does not exists ? Or am I forced to do a get, check if the latest version is fainallyzed, then increment de version of that version and create my new version ?

When running with PipelineDecorator.run_locally() I get the legitimate pandas error that I fixed by specifying the freq param in the pd.date_range(.... line in the component:
Launching step [generate_dataset] ClearML results page: `
[STEP 1/4] Generating dataset from autocut logs...
Traceback (most recent call last):
File "/tmp/tmp2jgq29nl.py", line 137, in <module>
results = generate_dataset(**kwargs)
File "/tmp/tmp2jgq29nl.py", line 18, in generate_dataset
...

ClearML package version used: 1.9.1
ClearML Server: SaaS - Pro Tier

AnxiousSeal95 Okay it seems to work with a compute optimized c2-standard-4 instance

Ah no i cant since the pipeline is in its own dummy model and you cannot reattach pipelines to real projects so I must instanciate a dummy task just to attach the output model to the correct project

Oh that's great, thanks 😍 🎉

Hey SuccessfulKoala55 currently using the clearml package version 1.7.1 and my server is a PRO SaaS deployment

Seems pretty critical tho, SuccessfulKoala55 is the 1.7.2 releasing soon ? ( I see the rc2 has been built 3 days ago)

So it seems to be an issue with the component parameter called in:
` @PipelineDecorator.pipeline(
name="VINZ Auto-Retrain",
project="VINZ",
version="0.0.1",
pipeline_execution_queue="Quad_VCPU_16GB"
)
def executing_pipeline(start_date, end_date):
print("Starting VINZ Auto-Retrain pipeline...")
print(f"Start date: {start_date}")
print(f"End date: {end_date}")

window_dataset_id = generate_dataset(start_date, end_date)

if name == 'main':
PipelineDec...

Or can I just set it to Task.current_task() ?

oopsy bad chunk

Okay, thank you for the explanations!

And additionally does the When executing a Task (experiment) remotely, this method has no effect). part means that if it is executed in a remote worker inside a pipeline without the dataset downloaded the method will have no effect ?

I got some credentials issues to in some pipelines steps and I solved it using

task = Task.current_task()
task.setup_aws_upload(...)

It can allows you to explicitaly specify credentials

Hum, must be more arcane then, I guess the official support would be able to provide an answer, they usually answer within 24 hours