Yeah, I believe all VS Code Extensions are in TypeScript. My main point was that this is an example of a VS Code extension that executes a Python CLI.

Playing around this weekend to learn the ins and outs of extensions

I think it will work. There's a lot of really useful code in the black extension. I'm recruiting people now to join in on Friday. I'm actually very confident about it after messing around.

Yeah. I'd need to clone this and run it locally to start to understand how it all works. Would be a cool exercise. They advertise that it's really easy to author VS Code extensions. I've seen pretty junior folks do it which makes me think it can't be too bad 😆

I did a post on Linkedin with several slides on how I plan to build it here

Oh awesome @<1523701132025663488:profile|SlimyElephant79> ! If you want to take a look, I made a big list of things to add. I'm working on a docker-compose.yaml file so we can have a good local development environment.

There's a lot of room to improve this from cleaning up the code to adding features on the list.

None

@<1594863216222015488:profile|ConvincingGrasshopper20> throwing this out there... would you want to make this with me at the Hackathon??

In a future iteration, it'd be cool if you could configure presets. Like maybe you have an on-startup.sh script you really like using to set up your instance, and VS Code extensions you want to pass to the --install-extensions ... flag

Duh! I bet VS Code's Python extensions like the VS Code Black Extension would be a really good starting place. They are small and are wrappers around a Python CLI tool. I bet there's a lot we could adapt for the ClearML CLI

I'll search around some more when I get time. I have no idea, but it feels like ClearML has already done the hard part which is creating clearml-session in the first place.

This could be a really low-hanging OSS contribution that could make a real impact 😄 .

^^^ For my own notes: this is the web request made by the frontend to create a set of credentials

That could work! Is that an option? Something that lets me spin up the ClearML and get a services worker to connect to it without manual steps.

The question I'm exploring remains: is it possible to acquire that initial set of ClearML API keys programmatically so that the manual steps of 1-4 above can be avoided for an initial deployment?

I SOLVED IT, NO NEED TO READ FURTHER 😄

I'm a chump and didn't read the docs: None

Oh, I think I got overexcited and didn't look at this closely. So this ACCESS/SECRET key pair is on the agent-services container.

I can see that agent-services is simply a container running `clearml-agent daemon --queue ser...

I did a quick local experiment and observed that credentials created from the UI indeed become invalid if you delete the ClearML volumes.

• starting docker-compose locally
• creating a set of credentials from the UI
• hardcodign those credentials into the docker-compose file
• restarting
• the agent-services container started up and successfully became a registered worker
• I killed the docker-compose and deleted the volume folders
• restarted the docker-compose (with the same hard-coded...

One idea: is it possible to store usable credentials in advance and place them in a volume that the ClearML containers can access and then use?

This is a low-key open-source project if anyone wanted to contribute. Since the project is early, there are lots of high-impact things, e.g. UI polish, that would be relatively low effort 😄

Disclaimer: I'm not familiar enouch with the ClearML codebase to vouch for the quality of this PR, although it is short which is typically good . The feature we're interested in is the ability to specify the subnet_id .

I may be able to prepare a PR that only allows specifying the subnet ID. Can you help me brainstorm scenarios you’d want to see tested? Also, do these need to be automated tests?

Hey! Sorry, I don't think I ever solved this for elasticsearch 😕

Hi! Yes, it just logs the .py file.

Oh duh, thanks. What about non standard entrypoints (as opposed to arguments) like accelerate launch train.py ?

*or Gateway

Oh there's parallelization as well. You could have step 1 gather the data, and then fan out to N parallel steps that all do different things with the data, for example hyper parameter tuning

Oh this is thought provoking. Yeah, the idea of using ClearML for R&D is super appealing (to me speaking as an MLOps engineer 😆 ). And having the power of Metaflow's scheduler (on Step Functions with Event Bridge since we'd do the AWS-native deployment) also makes sense to me.

I'll keep asking questions about how we could do event-based jobs with alerting built in on ClearML in a different thread later on.

I pasted your points (anonymously) onto the Metaflow slack to le...

If this works, we might be able to fully replace Metaflow with ClearML!

(Refering to the feature where Metaflow creates Step Functions state machines for you, and then you can use those to trigger event-driven batch jobs in the same way described here)

I don't know that you'd have to pre-build credentials into docker. If you could specify a set of credentials as environment variables to the docker run ... command or something, that would work just fine.

The goal is to be able to run docker-compose up in CI, which starts a clearml-server. And then make several API calls to the started ClearML server to prove that the VS Code extension code is working.

Examples:

• Assert that the extension can auth with ClearML
• Assert that the ext...

Oh wow. If this works, that will be insanely cool. Like, I guess what I'm going for is that if I specify "username: test" and "password: test" in that file, that I can specify "api.access_key: test" and "api.secret_key: test" in the clearml.conf used for CI. I'll give it a try tonight!

I use the Dark Reader chrome extension 😆

For now, I've written a headless selenium script to generate credentials for the fresh ClearML instance in CI.