3.3.0 😅

Any thoughts SuccessfulKoala55 ?

We have a more complicated case but I'll work around it 😄

Follow up though - can configuration objects refer to one-another internally in ClearML?

Oh and clearml-agent==1.1.2

SuccessfulKoala55 This happens pip >= 22.3 btw.

Another semi-related issue is that I now encounter these kind of error messages:
clearml_agent: ERROR: __init__() got an unexpected keyword argument 'types'

Thanks LazyFish41 , that explains it 👍

Yup, latest version of ClearML SDK, and we're deployed on AWS using K8s helm

Of course now it's not there anymore 😆 If/when it happens again I'll ping you here 🙂

Yes, using this extra_clearml_conf parameter you can add configuration

This is again exposing the environment variables on the WebUI for everyone to see.
The idea was to specify just the names of the environment variables, and that those would be exposed automatically to the EC2 instance, without specifying what values they should have (the value is taken from the agent running the scaler)

(Following this)

Is Task.create the way to go here? 🤔

I don't think there's a PR issue for that yet, at least I haven't created one.

I could have a look at this and maybe make a PR.
Not sure what would the recommended flow be like though 🤔

Thanks, that's what I thought - so I'm missing something else in the installation. I'll dig further 🙂

Another example - trying to validate dataset interactions ends with

` else:
self._created_task = True
dataset_project, parent_project = self._build_hidden_project_name(dataset_project, dataset_name)
task = Task.create(
project_name=dataset_project, task_name=dataset_name, task_type=Task.TaskTypes.data_processing)
if bool(Session.check_min_api_server_version(Dataset.__min_api_version)):
get_or_create_proje...

I'm not sure; the setup is not unique to Mac.
Each user has their own .env file which is given to the code entry point, and at some point will be loaded with dotenv.load_dotenv() .
The environment variables are not set in code anywhere, but the clearml.conf uses them directly.

No it does not show up. The instance spins up and then does nothing.

Maybe it's the missing .bashrc file actually. I'll look into it.

Anything specific we should look into TimelyPenguin76 ?

Nice, that works 🙂 Thanks!

We just redeployed to use the 1.1.4 version as Jake suggested, so the logs are gone 😞

... and any way to define the VPC is missing too 🤔

We do not CostlyFox64 , but this is useful for the future 🙂 Thanks!
TimelyPenguin76 I'll have a look, one moment.

SweetBadger76 TimelyPenguin76
We're finally tackling this (since it has kept us back at 1.3.2 even though 1.6.2 is out...), and noticed that now the bucket name is also part of the folder?

So following up from David's latest example:
StorageManager.download_folder(remote_url='s3://****-bucket/david/', local_folder='./')Actually creates a new folder ./****-bucket/david/ and puts it contents there.

EDIT: This is with us using internal MinIO, so I believe ClearML parses that end...

Sorry, I misspoke, yes of course, the agents config file, not the queues

Seems like you're missing an image definition (AMI or otherwise)

SuccessfulKoala55 help me out here 🙂
It seems all the changes I make in the AWS autoscaler apply directly to the virtual environment set for the autoscaler, but nothing from that propagates down to the launched instances.
So e.g. the autoscaler environment has poetry installed, but then the instance fails because it does not have it available?

I'm trying to build an easy SDK that would fit DS work and fit the concept of clearml pipelines.
In doing so, I'm planning to define various Step classes, that the user can then experiment with, providing Steps as input to other steps, etc.

Then I'd like for the user to be able to run any such step, either locally or remotely. Locally is trivial. Remotely is the issue. I understand I'll need to upload additional data to the remote instance, and pull a specific artifact back to the notebo...

It failed on some missing files in my remote_execution, but otherwise seems fine now

(the extra_vm_bash_script is what you're after)

Hey @<1523701070390366208:profile|CostlyOstrich36> , thanks for the reply!
I’m familiar with the above repo, we have the ClearML Server and such deployed on K8s.
What’s lacking is documentation regarding the clearml-agent helm chart. What exactly does it offer, etc.
We’re interested in e.g. using karpenter to scale our deployments per demand, effectively replacing the AWS autoscaler.