Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Profile picture
FierceHamster54
Moderator
29 Questions, 178 Answers
  Active since 10 January 2023
  Last activity one year ago

Reputation

0

Badges 1

131 × Eureka!
0 Votes
15 Answers
2K Views
0 Votes 15 Answers 2K Views
Hey having an issue passing parameters to a component in a pipeline, the parameters appear to be None inside the component function: @PipelineDecorator.compo...
3 years ago
0 Votes
5 Answers
2K Views
0 Votes 5 Answers 2K Views
Hey, would it be possible to add a way to edit autoscaler configs without having to clone them ? This is really frustrating especially when you reached the q...
3 years ago
0 Votes
7 Answers
2K Views
0 Votes 7 Answers 2K Views
Heya, is there any plan for ClearML to leverage the new https://developer.nvidia.com/blog/getting-the-most-out-of-the-a100-gpu-with-multi-instance-gpu/ tech ...
3 years ago
0 Votes
4 Answers
2K Views
0 Votes 4 Answers 2K Views
3 years ago
0 Votes
4 Answers
2K Views
0 Votes 4 Answers 2K Views
3 years ago
0 Votes
13 Answers
2K Views
0 Votes 13 Answers 2K Views
Hey all, hope you're having a great day, having an unexpected behavior with a training task of a YOLOv5 model on my pipeline, I specified a task in my traini...
2 years ago
0 Votes
6 Answers
2K Views
0 Votes 6 Answers 2K Views
one year ago
0 Votes
8 Answers
1K Views
0 Votes 8 Answers 1K Views
Hey everyone, I am having some difficulties passing environment variables on my piepeline components running on agents ( 1.6.1 ) without docker mode: - I set...
one year ago
0 Votes
6 Answers
2K Views
0 Votes 6 Answers 2K Views
Hey guys, i hope you all have a nice days, I had to use the Task method .setup_aws_upload(bucket=...,region=...) to overcome a incorrect region specified for...
2 years ago
0 Votes
1 Answers
2K Views
0 Votes 1 Answers 2K Views
3 years ago
0 Votes
4 Answers
2K Views
0 Votes 4 Answers 2K Views
2 years ago
0 Votes
8 Answers
2K Views
0 Votes 8 Answers 2K Views
Hey, is there a shortcut on the Dataset SDK to directly get the latest version of a dataset ?
3 years ago
0 Votes
7 Answers
2K Views
0 Votes 7 Answers 2K Views
Hey I hope everyone is having a good day, two quick questions about datasets: Does squashing two datasets deletes the two original datasets ? Is it possible ...
2 years ago
0 Votes
1 Answers
2K Views
0 Votes 1 Answers 2K Views
Heya, good day to everyone, I'm finding myself facing this random error with a very opaque backtrace when attempting to squash two distinct versions of the s...
2 years ago
0 Votes
13 Answers
2K Views
0 Votes 13 Answers 2K Views
3 years ago
0 Votes
15 Answers
2K Views
0 Votes 15 Answers 2K Views
Heya, trying to setup a GCP autoscaler for general-purpose CPU instances ( e2-standard-4 ) but I get into that error: googleapiclient.errors.HttpError: And I...
3 years ago
0 Votes
25 Answers
2K Views
0 Votes 25 Answers 2K Views
Hey, trying to figure out how to create an https://clear.ml/docs/latest/docs/clearml_sdk/model_sdk#output-models , the doc says it needs a TaskId but my trai...
2 years ago
0 Votes
9 Answers
2K Views
0 Votes 9 Answers 2K Views
Hello everyone! I setup a GCP autoscaler on my Pro SaaS deployment but I keep getting this error: clearml_agent: ERROR: Server does not support --use-owner-t...
3 years ago
0 Votes
10 Answers
2K Views
0 Votes 10 Answers 2K Views
3 years ago
0 Votes
8 Answers
2K Views
0 Votes 8 Answers 2K Views
3 years ago
0 Votes
15 Answers
2K Views
0 Votes 15 Answers 2K Views
Heya, I hope you're all well in this beautiful day, my GCP Autoscaler just died with that strange but short backtrace, wondered if it rang a bell to any of y...
3 years ago
0 Votes
7 Answers
2K Views
0 Votes 7 Answers 2K Views
Hey, is there a way to pass docker args for the execution of a pipeline controller defined through decorator @PipelineDecorator.pipeline the same way we can ...
2 years ago
0 Votes
9 Answers
2K Views
0 Votes 9 Answers 2K Views
Hey just wanting to know: what is the recommended best practice to write ClearML Pipelines between controller and decorators ?
3 years ago
0 Votes
7 Answers
1K Views
0 Votes 7 Answers 1K Views
Hey everyone, As a Pro-tier SaaS user, I'm experiencing a very high latency when finalizing a dataset, it is attached in a big dataset version hierarchy and ...
one year ago
0 Votes
1 Answers
2K Views
0 Votes 1 Answers 2K Views
Hey is ClearML MLFlow based ? Is it exposed to CVE-2023-1176 and CVE-2023-1177 ?
2 years ago
0 Votes
1 Answers
2K Views
0 Votes 1 Answers 2K Views
Hey is .get_local_copy() thread-safe ? I mean can I concurrently download several datasets without breaking the cache and StorageManager ?
3 years ago
0 Votes
20 Answers
2K Views
0 Votes 20 Answers 2K Views
Hey currently trying to run a pipeline locally to test a pipeline component with PipelineDecorator.run_locally() , first try returned a random pandas error, ...
3 years ago
0 Votes
2 Answers
2K Views
0 Votes 2 Answers 2K Views
Hey, it is said in the pipeline decorator example that requirements for executing a pipeline component is inferred from the imports inside the component func...
3 years ago
0 Votes
12 Answers
2K Views
0 Votes 12 Answers 2K Views
Hey has anyone managed to capture Darts logging with ClearML when using the temporal fusion transformers ? Even when overriding their trainer with a custom P...
2 years ago
0 Hello. In Which Yaml Files For Kubernetes And Where Should I Specify The Aws File_Server And Credentials For Clearml-Agent So That When Running Tasks Remotely In The Queue, The Artifacts Are Saved In The S3 Bucket?

I got some credentials issues to in some pipelines steps and I solved it using

task = Task.current_task()
task.setup_aws_upload(...)

It can allows you to explicitaly specify credentials

one year ago
0 Hey Currently Trying To Run A Pipeline Locally To Test A Pipeline Component With

So it seems to be an issue with the component parameter called in:
` @PipelineDecorator.pipeline(
name="VINZ Auto-Retrain",
project="VINZ",
version="0.0.1",
pipeline_execution_queue="Quad_VCPU_16GB"
)
def executing_pipeline(start_date, end_date):
print("Starting VINZ Auto-Retrain pipeline...")
print(f"Start date: {start_date}")
print(f"End date: {end_date}")

window_dataset_id = generate_dataset(start_date, end_date)

if name == 'main':
PipelineDec...

3 years ago
0 [Datasets] Is It Possible To Get An Individual File From A Dataset? Example Would Be Accessing Only A Single Feature From A Feature Store Dataset When It Could Be Costly To Download The Entire Dataset

I suppose your worker is not persistent, so I might suggest having a very cheap instance as a persistent worker where you have your dataset persistently synced using . https://clear.ml/docs/latest/docs/references/sdk/dataset/#sync_folder and then taking the subset of files that interests you and pushing it as a different dataset, marking it as a subset of your main dataset id using a tag

2 years ago
0 Hey Currently Trying To Run A Pipeline Locally To Test A Pipeline Component With

CostlyOstrich36 Should I start a new issue since I pinpointed the exact problem given than the beginning of this one was clearly confusing for both of us ?

3 years ago
3 years ago
2 years ago
0 Hey, About The Dependency Propagation Of Pipeline Components, If I Call A Vanilla Python Function From A Component Does The Dependencies Specified In The Internal Imports Propagated To This Function Call Too ? And Additionally If That Function Is In Anoth

Would have been great if the CLearML resolver would just inline the code of locally defined vanilla functions and execute that inlined code under the import scope of the component from which it is called

3 years ago
0 Hello All, Thanks For This Really Cool Software And Community! I Have A Question On

THat make sense since this function executes your component as classic pythonic functions

2 years ago
0 Hello Everyone! I Setup A Gcp Autoscaler On My Pro Saas Deployment But I Keep Getting This Error:

Okay thanks! Please keep me posted when the hotfix is out on the SaaS

3 years ago
0 Hey Currently Trying To Run A Pipeline Locally To Test A Pipeline Component With

When running with PipelineDecorator.run_locally() I get the legitimate pandas error that I fixed by specifying the freq param in the pd.date_range(.... line in the component:
Launching step [generate_dataset] ClearML results page: `
[STEP 1/4] Generating dataset from autocut logs...
Traceback (most recent call last):
File "/tmp/tmp2jgq29nl.py", line 137, in <module>
results = generate_dataset(**kwargs)
File "/tmp/tmp2jgq29nl.py", line 18, in generate_dataset
...

3 years ago
0 <no title>

Did you properly install Docker and Docker nvidia toolkit ? here's the init script i'm using on my autoscaled workers:

#!/bin/sh

sudo apt-get update -y

sudo apt-get install -y \
    ca-certificates \
    curl \
    gnupg \
    lsb-release

sudo mkdir -p /etc/apt/keyrings

curl -fsSL 
 | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg

echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] 
 \
  $(lsb_release -cs) stable" | s...
2 years ago
0 I'Ve Been Using Clearml On On-Premise Machines And Would Now Like To Deploy Everything In Gcp (Deploy Everything New From Scratch, Don'T Care About Migrating The Data). I'Ve Followed Tutorial

Yup, if you want to access it through https you're required to have a domain pointing to that IP with a certificate in place (using letsencrypt as instance) or else you'll get some SSL error

one year ago
0 Hey Has Anyone Managed To Capture Darts Logging With Clearml When Using The Temporal Fusion Transformers ? Even When Overriding Their Trainer With A Custom Pytorch Lightning Trainer It Seems That Clearml Cannot Retrieve The Iteration Log...

The expected behavior is that the task would capture the iteration scalar of the PL trainer but nothing is recorded

import clearml
from darts.models import TFTModel

model = TFTModel(
    input_chunk_length=28,
    output_chunk_length=14,
    n_epochs=300,
    batch_size=4096,
    add_relative_index=True,
    num_attention_heads=4,
    dropout=0.3,
    full_attention=True,
    save_checkpoints=True,
)

task = Task.init(
    project_name='sales-prediction',
    task_name='TFT Training 2'...
2 years ago
0 Hey Having An Issue Passing Parameters To A Component In A Pipeline, The Parameters Appear To Be

Hey SuccessfulKoala55 currently using the clearml package version 1.7.1 and my server is a PRO SaaS deployment

3 years ago
0 Hey I Hope Everyone Is Having A Good Day, Two Quick Questions About Datasets:

And additionally does the When executing a Task (experiment) remotely, this method has no effect). part means that if it is executed in a remote worker inside a pipeline without the dataset downloaded the method will have no effect ?

2 years ago
0 Heya, I Hope You'Re All Well In This Beautiful Day, My Gcp Autoscaler Just Died With That Strange But Short Backtrace, Wondered If It Rang A Bell To Any Of You ?

I was launching a pipeline run, but I don't remember having set the autoscaler to use spot instances (I believe the GCP terminology for spot instance is "preemptible" and I set it to false)

3 years ago
0 Heya, Trying To Setup A Gcp Autoscaler For General-Purpose Cpu Instances (

AnxiousSeal95 Okay it seems to work with a compute optimized c2-standard-4 instance

3 years ago
0 Hey Guys, I Hope You All Have A Nice Days, I Had To Use The

@<1523701087100473344:profile|SuccessfulKoala55> I had already bumped boto3 to its latest version and all the files I added to the dataset were pickle binary files

2 years ago
0 Heya, Is There Any Plan For Clearml To Leverage The New

There is a gap in the GPU offer on GCP and there is no modern middle-ground for a TPU with more than 16GB GRAM and less than 40GB, so sometime we need to provision a A100 to get the training speed we want but we don't use all the GRAM so I figured out if we could batch 2 training tasks on the same A100 instance we would still be on the winning side in term of CUDA cores and getting the most of the GPU-time we're paying.

3 years ago
0 Hey, Trying To Figure Out How To Create An

Oh, that's nice, if I import a model using InputModel do I still need to specify a OutputModel ?

2 years ago
0 Hey Currently Trying To Run A Pipeline Locally To Test A Pipeline Component With

Thus the main difference of behavior must be coming from the _debug_execute_step_function property in the Controller class, currently skimming through it to try to identify a cause, did I provide you enough info btw CostlyOstrich36 ?

3 years ago
0 Hey, Is There A Way To Pass Docker Args For The Execution Of A Pipeline Controller Defined Through Decorator

Takling about that decorator which shouyld also have a docker_arg param since it is executed as an "orchestration component" but the param is missing https://clear.ml/docs/latest/docs/references/sdk/automation_controller_pipelinecontroller/#pipelinedecoratorpipeline

2 years ago
Show more results compactanswers