Yes. Though again, just highlighting the naming of foo-mod is arbitrary. The actual module simply has a folder structured with an implicit namespace:

foo/
  mod/
    __init__.py
    # stuff

FWIW, for the time being I’m just setting the packages to all the packages the pipeline tasks sees with:

    packages = get_installed_pkgs_detail()
    packages = [f"{name}=={version}" if version else name for name, version in packages.values()]
    packages = task.data.script.require...

Last but not least - can I cancel the offline zip creation if I'm not interested in it 🤔
EDIT: I see not, guess one has to patch ZipFile ...

packages an entire folder as zip

What if I have multiple files that are not in the same folder? (That is the current use-case)

It otherwise makes sense I think 🙂
Our workaround now for using a Dataset as we do, is to store the dataset ID as a configuration parameter, so it's always included too 😉

I guess it does not do so for all settings, but only those that come from Session()

We're still working these quirks out. But one issue after we changed the AMI is that the VPC (SubnetId?) was missing from the instance so it could not reach the ClearML API server.

I think maybe the autoscaler service is missing some additional settings...

Sorry to keep this up - what about support for minio using the environment variable? Do I set the CLEARML_FILES_HOST to the end point instead of an s3 bucket?

And agent too, I hope..?

I'd be happy to join a #releases channel just for these!

Just randomly decided to check and saw there's a server 1.4 ready 🎉

I realized it might work too, but looking for a more definitive answer 😄 Has no-one attempted this? 🤔

The deferred_init input argument to Task.init is bool by default, so checking type(deferred_init) == int makes no sense to begin with, and is altering the flow.

It does (root in a docker container); it shouldn't touch /run/systemd/generator/systemd-networkd.service anyway though

Would be great if it is 😍 We have few files that change frequently and are quite large in size, and it would be quite a storage hit to save all of them

Hm, I did not specify any specific versions previously. What was the previous default?

One more thing that may be helpful SweetBadger76 , I've gone ahead and looked into clearml.storage.helper , and found that at least if I specify the bucket name directly in the aws.s3.credentials configuration for MinIO, then:
In [4]: StorageHelper._s3_configurations.get_config_by_uri(' ` ')
Out[4]: S3BucketConfig(bucket='clearml', host='some_ip:9000', key='xxx', secret='xxx', token='', multipart=False, acl='', secure=False, region='', verify=True, use_credentials_chain=False)...

Okay so the new functionality is maintained; thanks!

Heh, good @<1523704157695905792:profile|VivaciousBadger56> 😁
I was just repeating what @<1523701070390366208:profile|CostlyOstrich36> suggested, credits to him

Yes

SuccessfulKoala55 This is due to the pip update in clearml-agent

I'm aware, but it would be much cleaner to define them in the worker's clearml.conf and let ClearML expose them locally to running tasks.

EDIT: Also the above is specifically about serving, which is not the target here 🤔 At least not yet 😄

Could also be that the use of ./ is the issue? I'm not sure what else I can provide you with, SweetBadger76

We just redeployed to use the 1.1.4 version as Jake suggested, so the logs are gone 😞

I'd like to remove the hidden system tag from a project

Uhhh, not really unfortunately :white_frowning_face: . I have ~20 tasks happening in a single file, and it's quite random if/when this happens. I just noticed this tends to happen with the shorter tasks

Seemed to work fine again in detached mode, what went wrong there :shocked_face_with_exploding_head:

In the Profile section, yes, they are well defined (bucket, secret, key, and endpoint)

21s is just ridiculous, it's scanning the entire file system starting at /

Yes exactly 👍 Good news.

Thanks, that's what I thought - so I'm missing something else in the installation. I'll dig further 🙂

minio was a tiny bit of headache to configure, but I'd be happy to help if you want CrookedWalrus33 , I just went through this process yesterday and today (see a few threads up...)

But we do use S3

The agent also uses a different clearml.conf , so it should not matter?