Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Profile picture
BattyCrocodile47
Moderator
32 Questions, 142 Answers
  Active since 02 March 2023
  Last activity 3 months ago

Reputation

0

Badges 1

124 × Eureka!
0 Votes
6 Answers
510 Views
0 Votes 6 Answers 510 Views
Another AWS autoscaler question. The docker-compose.yml automatically adds a ClearML agent to the services queue. When I run python aws_autoscaler.py --remot...
9 months ago
0 Votes
0 Answers
185 Views
0 Votes 0 Answers 185 Views
The ClearML Session VS Code extension is live! πŸŽ‰ πŸŽ‰ If you start a ClearML session using the clearml-session CLI tool, this will display it and help you con...
3 months ago
0 Votes
2 Answers
626 Views
0 Votes 2 Answers 626 Views
11 months ago
0 Votes
20 Answers
470 Views
0 Votes 20 Answers 470 Views
How would ya'll approach backing up the elastic-search/redis/etc. data in self-hosted ClearML? Any drawbacks/risks of doing a simple process that periodicall...
7 months ago
0 Votes
3 Answers
557 Views
0 Votes 3 Answers 557 Views
11 months ago
0 Votes
2 Answers
563 Views
0 Votes 2 Answers 563 Views
Does clearML have a good story for offline/batch inference in production? I worked in the Airflow world for 2 years and these are the general features we use...
11 months ago
0 Votes
4 Answers
531 Views
0 Votes 4 Answers 531 Views
Hi,
Hi, @<1565509803839590400:profile|MoodyBear54> and I are trying to deploy the ClearML autoscaler to deploy the EC2 instances into a pre-existing VPC/subnet. ...
9 months ago
0 Votes
2 Answers
502 Views
0 Votes 2 Answers 502 Views
Well, we accidentally leaked some super powerful credentials today. Is there a way to not have the UI or console logs show our AWS access keys, git credentia...
10 months ago
0 Votes
3 Answers
605 Views
0 Votes 3 Answers 605 Views
If I want to run tensorflow (version 2.10.0 with Python 3.8) with the AWS autoscaler, which AMI and Docker base image should I choose?
10 months ago
0 Votes
7 Answers
674 Views
0 Votes 7 Answers 674 Views
Hey friends, how do you configure ClearML to use an S3 bucket? Specifically: does every data scientist have to have hard-coded AWS credentials with read/writ...
11 months ago
0 Votes
0 Answers
313 Views
0 Votes 0 Answers 313 Views
If anyone wants to join remotely, there’s a remote-first AI/ML hackathon happening tomorrow. Some of the details are out of date right now—I’ll have this pag...
3 months ago
0 Votes
11 Answers
643 Views
0 Votes 11 Answers 643 Views
More of pushing ClearML to it's data engineering limits πŸ˜… . Could you use ClearML in a event-driven system? That would be so sick! I'm wondering if we could...
11 months ago
0 Votes
10 Answers
515 Views
0 Votes 10 Answers 515 Views
I’m working on an automated deployment of ClearML with IaC. I’ve got a script to start an EC2 instance that runs the docker compose file. Separately, I’ve go...
9 months ago
0 Votes
9 Answers
586 Views
0 Votes 9 Answers 586 Views
Security question: in my journey of running ClearML the "hard way" (self-hosted), one problem I haven't solved is security. Some discussion here...
11 months ago
0 Votes
2 Answers
506 Views
0 Votes 2 Answers 506 Views
AWS autoscale question: can the autoscaler use the IAM role of the EC2 instance it’s running on rather than needing to be provided AWS keys?
9 months ago
0 Votes
13 Answers
645 Views
0 Votes 13 Answers 645 Views
one year ago
0 Votes
7 Answers
435 Views
0 Votes 7 Answers 435 Views
Hi friends, we got on a sales call with ClearML yesterday and a discussion about webhooks came up. ClearML seems to not natively implement webhooks It seems ...
8 months ago
0 Votes
1 Answers
562 Views
0 Votes 1 Answers 562 Views
You guys, thank you so much. I have super positive things to say about ClearML and I'm very excited to try it out at work (still selling it) https://www.link...
11 months ago
0 Votes
21 Answers
312 Views
0 Votes 21 Answers 312 Views
Crazy idea: what if ClearML had a VS Code extension? It could help you start and join ClearML sessions! It could use your local ~/clearml.conf file for read ...
4 months ago
0 Votes
3 Answers
587 Views
0 Votes 3 Answers 587 Views
11 months ago
0 Votes
5 Answers
620 Views
0 Votes 5 Answers 620 Views
I'm getting some weird ClearML behavior. I've deployed it to an EC2 instance. When I access :8080 , I see the full ClaerML app as expected (first screenshot)...
11 months ago
0 Votes
5 Answers
311 Views
0 Votes 5 Answers 311 Views
Whelp. Here's our hackathon demo submission for a ClearML VS Code extension @<1523701205467926528:profile|AgitatedDove14> and @<1523701087100473344:profile|S...
3 months ago
0 Votes
4 Answers
241 Views
0 Votes 4 Answers 241 Views
Hey
Hey @<1523701205467926528:profile|AgitatedDove14> ! Don't know if you're up but we're working on the VS Code extension at the hackathon rn!
3 months ago
0 Votes
19 Answers
261 Views
0 Votes 19 Answers 261 Views
Hey
Hey @<1523701205467926528:profile|AgitatedDove14> , I saw this SO answer you gave about ClearML's docker-compose.yaml . You described getting a secret key pa...
3 months ago
0 Votes
6 Answers
228 Views
0 Votes 6 Answers 228 Views
Sorry for always posting such cryptic problems. I managed to create a docker-compose file that runs ClearML and it sets up 2 agents, one on the services queu...
3 months ago
0 Votes
0 Answers
504 Views
0 Votes 0 Answers 504 Views
I gave a demo of ClearML to our data engineering team (is also the ML infra team) and it went over really well! πŸŽ‰ We have two data science teams that we are...
11 months ago
0 Votes
18 Answers
415 Views
0 Votes 18 Answers 415 Views
Can anyone recommend a good workflow for clearml-session ? I like to code in the desktop app of VS Code. Right now, I - run clearml-session ... to start a se...
7 months ago
0 Votes
1 Answers
431 Views
0 Votes 1 Answers 431 Views
Is there a command line interface that lets you query and download models from the ClearML model registry the way you can with MLFlow? Example: # search for ...
8 months ago
0 Votes
7 Answers
215 Views
0 Votes 7 Answers 215 Views
Working on the VS Code extension. Pretty stumped on this one...
3 months ago
0 Votes
7 Answers
599 Views
0 Votes 7 Answers 599 Views
Can you help me make the case for ClearML pipelines/tasks vs Metaflow? Context within...
9 months ago
Show more results questions
0 Hi All, Is There Any Kind Of Clearml Adapter For Mlflow? I'M Talking About The Dozens Of Awesome Integrations With Mlflow That Allow You To Easily Log Artifacts, Visuals, Etc. To Mlflow Without Any Any Lines Of Code. Does Clearml Have It'S Own Competing

Hey, thanks for responding!

Does there happen to be ClearML auto-logging... for MLFlow? That would make it super easy for us to migrate our existing training/batch inference jobs to ClearML πŸ˜„

11 months ago
0 Crazy Idea:

@<1594863216222015488:profile|ConvincingGrasshopper20> throwing this out there... would you want to make this with me at the Hackathon??

4 months ago
0 My Autoscaled Instance Fails When Running "Git Clone" On A Private Repo. I

So here's a snippet from my aws_autoscaler.yaml file

9 months ago
9 months ago
0 My Autoscaled Instance Fails When Running "Git Clone" On A Private Repo. I

I have the same behavior whether or not I put task.execute_remotely(...) before or after the call to run_shell_script()

9 months ago
0 My Autoscaled Instance Fails When Running "Git Clone" On A Private Repo. I

Wow, it really does not want to show the output of those print statements in stdout. Here's the output of the task from the console after cloning it. Confirmed that the setup script and all code changes are present:

9 months ago
0 My Autoscaled Instance Fails When Running "Git Clone" On A Private Repo. I

I don't see it as an argument in Task.init or Task.execute_remotely

9 months ago
0 My Autoscaled Instance Fails When Running "Git Clone" On A Private Repo. I

Actually, dumb question: how do I set the setup script for a task?

9 months ago
0 Crazy Idea:

Oh awesome @<1523701132025663488:profile|SlimyElephant79> ! If you want to take a look, I made a big list of things to add. I'm working on a docker-compose.yaml file so we can have a good local development environment.

There's a lot of room to improve this from cleaning up the code to adding features on the list.

None

3 months ago
0 Can You Help Me Make The Case For Clearml Pipelines/Tasks Vs Metaflow? Context Within...

So, we've already got a model registry: MLFlow

And we've got a serving framework we really like: BentoML and FastAPI

The debate is between ClearML and Metaflow for

  • training models during the research phase
  • re-training models on a schedule or event-based trigger in production
  • running batch inference jobs on a schedule or event-based trigger in production
9 months ago
0 My Autoscaled Instance Fails When Running "Git Clone" On A Private Repo. I
configurations:
  extra_clearml_conf: ""
  extra_trains_conf: ""
  extra_vm_bash_script: |
    aws ssm get-parameter --region us-west-2 --name /clearml/github_ssh_private_key --with-decryption --query Parameter.Value --output text > ~/.ssh/id_rsa && chmod 600 ~/.ssh/id_rsa
    source /clearml_agent_venv/bin/activate

hyper_params:
  iam_arn: arn:aws:iam::<my account id>:instance-profile/clearml-2-AutoscaledInstanceProfileAutoScaledEC2InstanceProfile56A5348F-90fmf6H5OUBx
9 months ago
0 My Autoscaled Instance Fails When Running "Git Clone" On A Private Repo. I

Actually that's wrong: really this is the current volume mount

'-v', '/tmp/clearml_agent.ssh.cbvchse1:/.ssh',

Could changing these values to /root/.ssh work? Do you know what use within the docker image ClearML is using?
image

9 months ago
0 I'M Getting Some Weird Clearml Behavior. I'Ve Deployed It To An Ec2 Instance. When I Access

So the problem came back even with this new URL. I discovered clearing your cookies fixes it.

11 months ago
9 months ago
0 Sorry For Always Posting Such Cryptic Problems. I Managed To Create A Docker-Compose File That Runs Clearml

Hmm... these people are recommending restarting docker completely. I may have tried that already, but I'll do it again when I get some time to be sure.

3 months ago
0 Crazy Idea:

I'll search around some more when I get time. I have no idea, but it feels like ClearML has already done the hard part which is creating clearml-session in the first place.

This could be a really low-hanging OSS contribution that could make a real impact πŸ˜„ .

4 months ago
0 Hi,

I may be able to prepare a PR that only allows specifying the subnet ID. Can you help me brainstorm scenarios you’d want to see tested? Also, do these need to be automated tests?

9 months ago
0 My Autoscaled Instance Fails When Running "Git Clone" On A Private Repo. I

Here's a screenshot if a session where I first try to clone as ssm-user , but it fails, then I change to root and it succeeds
image

9 months ago
0 My Autoscaled Instance Fails When Running "Git Clone" On A Private Repo. I

I'm not seeing a extra_docker_shell_script in my clearml.conf generated by clearml-agent init like in this guide

9 months ago
0 Can You Help Me Make The Case For Clearml Pipelines/Tasks Vs Metaflow? Context Within...

For these functions, Metaflow offers:

  • triggering: integration with AWS event bridge. It's really easy to use Boto3 and AWS access keys to emit events for Metaflow DAGs. It's nice not to have to worry about networking for this.
  • Scheduling: The fact that Metaflow uses stepfunctions is reassuring.
  • observability: this lovely flame graph where you can view the logs and duration of each step in the DAG, it's easy to view all the DAG runs including the ones that have failed. Ideally, we w...
9 months ago
0 My Autoscaled Instance Fails When Running "Git Clone" On A Private Repo. I

cc: @<1565509803839590400:profile|MoodyBear54>

9 months ago
0 Working On The Vs Code Extension. Pretty Stumped On This One...

The issue went away. I'm still not sure why, but what finally made it work was creating a set of credentials manually in the UI and then setting those in my ~/clearml.conf file.

Do you happen to have a link to a docker-compose.yaml file that has a hardcoded set of credentials?

I want to seed the clearml instance with a set of credentials and ~/clearml.conf to run automated tests.

3 months ago
0 Working On The Vs Code Extension. Pretty Stumped On This One...

While I'm wishing for things: it'd be awesome if it had a queue already set up. But if there's not a way to do that in the docker compose file, I could potentially write a script that uses the creds to create one using API calls

3 months ago
0 Working On The Vs Code Extension. Pretty Stumped On This One...

Interesting . It’s actually just running locally on my laptop. It seemed only to be an issue when pointing the ClearML session CLI at my local version of ClearML. Still thinking about this one.

3 months ago
Show more results compactanswers