Hi, I am trying to use the aws autoscaler to assign instance profiles to new machines. This is a better way than managing credentials. I added the configurat...

Regarding the “classic” datasets (not hyper datasets): Is there an option to do something equivalent to dvc’s “ https://dvc.org/doc/user-guide/managing-exter...

Two simple lineage related questions: Task B is a clone of Taks A. Does B store the information that it was cloned from A somewhere? Training task X loads Da...

I am trying to run a task that is completely detached from git - remotely. The script uploads fine but in the UI, the git repo appears as “origin”. When the ...

Cannot upload a dataset with a parent - seems very odd! clearml versions I tried: 1.6.1, 1.6.2 scenario: * Create parent dataset (with storage on S3) * Uploa...

Hi, I am catching up with http://clear.ml for stuff beyond exp. tracking, and have a few questions. Will ask them separately to allow threading:

autoscaler 101 questions: What is the best practice for managing credentials so that they don’t get saved in clearml webapp? When the https://clear.ml/docs/l...

what’s a good ami to use for the clearml autoscaler on AWS? the defaults offered confidently by the various auto scaler installers don’t seem to exist…| e.g....

Is there a way to tell the agent to run in docker in a way that just “trusts” the installation completely? for example, I have a local environment with pytho...

Avoiding http://Clear.ml glue code spaghetti - community best practices? Say I have training pipeline : Task 1 - data preprocessing -> create a dataset artif...

I have a pipeline with tasks A->B->C. I want to be able to trigger it manually, and skip A regardless of it’s cache status. I want to pass B value that repre...

FYI I am getting a lot of read timeouts from the community server: Retrying (Retry(total=235, connect=240, read=235, redirect=240, status=240)) after connect...

I have a situation where I’d like to “promote” the pipeline (and dataset) by creating it in a completely separate instance of clearml server which is used fo...

I have code that does torch.load(path) and deserializes a model. I am performing this in package a.b.c, and the model’s module is available in in a.b.c.model...

hi, I created a dataset with 20K files, total of 20GB, with storage pointing to S3. When I upload (or close) the dataset, during the compression phase, the c...

I have a local folder a, and a dataset B. a: a a/.DS_Store a/1.txt a/b a/b/.DS_Store a/b/1.txt a/b/c a/b/c/1.txtDataset B: b b/2.txt b/c b/c/2.txtI want to “...

hi, I am running a pipeline from decorators. the pipeline runs fine. Then I try to clone it by clicking the (successful) run and launching. The pipeline fail...

question about pipeline and long-waiting tasks: Say I want to generate a dataset. The workflow I have requires query to a DB Creating a labeling assigment in...

Is there a reference implmentation for a task in a pipeline that awaits user input?

Hello folks! I have a pipeline with three tasks: A, B, and C I want to set it up so that: A gets assigned a machine (e.g. based on the queue) B always gets a...

hi, When running a training script from pycharm, it seems that clearml logs only those packages that are explicitly imported by my .py files; it seems to not...

autoscaler from saas (pro version). I attempted to use the autoscaler “application” from clearml UI. here is what I get in the logs of the autoscaler screen ...

I have a logical task that I want to split to multiple workers. The task involves processing media files (not training). The optimal design for me would be: ...

hi folks, is there a way to force clear-ml agent with --docker to not create a virtualenv at all? And perhaps not even attempt to install requirements even? ...

Is there a case-study or ref. architecture for interacting with CI/CD i.e. exposing mature pipelines to be triggered upon code pushes (taking latest git hash...

hi, I have a task which uses hydra for configuration. I want to add this taks to a pipeline, and pass the full hydra config objects to the task. is there a w...

Did more digging, seems that you need to start the agent with CLEARML_AGENT_SKIP_PIP_VENV_INSTALL=1

Hi, I am using PipelineDecorator to create tasks. is there a way to force it to use the entire git repo it is created from on the pythonpath? vs. just the de...