Hi PerplexedGoat65

it appears, in a practical sense, this means to mount the second drive, and then bind them in ClearML’s configuration

Yes, the entire data folder (reason is, if you loose it, you loose all the server storage / artifacts)

Also, thinking about Docker and slower access speed for Docker mounts and such,

If the host OS is linux, you have nothing to worry about, speed will be the same.

Hi UnsightlySeagull42
Could you test with the latest RC
pip install clearml==1.0.4rc0Also could you provide some logs?

It seems like the web server doesn’t log the call to AWS, I just see this:

This points to the browser actually sending the AWS delete command. Let me check with FE tomorrow

Hi DeterminedToad86
I just verified on a clean sagemaker instance everything should just work, see here: https://demoapp.demo.clear.ml/projects/0e919ea1cc5c499b99e1ab85004b6e97/experiments/887edef09d4549e88b829a34c87d4d5b/output/execution Yes if you have more than one file (either notebook or python script) than you must have a git repo, in order to run the task using the Agent.

RoughTiger69 I think you need the latest version (+1.3.0 with UI support)
If you are using an older version, you need to specify that you are continuing an execution (Change the "Configuration/Args/continue_pipeline" to True)
EDIT: clearml 1.3.x will work with clearml-server 1.2

IntriguedRat44 could I ask you to open a GitHub issue on it?
I really do not want it to slip through our fingers...
(BTW: meanwhile I was not able to reproduce it, what's the OS / nvidia drivers you are using )?

Great ascii tree 🙂
GrittyKangaroo27 assuming you are doing:
@PipelineDecorator.component(..., repo='.') def my_component(): ...The function my_component will be running in the repository root, so in thoery it could access the packages 1/2
(I'm assuming here directory "project" is the repository root)
Does that make sense ?
BTW: when you pass repo='.' to @PipelineDecorator.component it takes the current repository that exists on the local machine running the pipel...

Hi GrievingTurkey78
I think it is already fixed with 0.17.5, no?

You will have to build your own docker image based on that docker file, and then update the docker compose

Hi @<1526371965655322624:profile|NuttyCamel41>
so sorry I just realized I have not answered it it!

I just tried the pytorch example from the clearml-serving repo and got the error about the wrong model name

okay that is odd, are you using the exact same containers / docker-compose? what is the difference ?

I0603 09:44:02.665851 41 model_lifecycle.cc:693] successfully loaded 'test_model_pytorch' version 1

does that mean that even though there is a warning there you can curl to ...

Hi SpotlessWorm70

OMP: Error #15: Initializing libiomp5.dylib, but found libomp.dylib already initialized.
OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program.

This seems like OpenMP issue
I would assume something is off with the local environment (not really connected to clearml but to one of the frameworks, for example TF, Keras, etc.)

Hi GiddyTurkey39
Are you referring to an already executed Task or the current running one?
(Also, what is the use case here? is it because the "installed packages are in accurate?)

But from the log it seems that:
you are not running as root in the docker? Python3.8 is installed (and not python 3.6 as before)

I look forward to your response on Github.

Great, I would like to make this discussion a bit more open and accessible so GitHub is probably better

I'd like to start contributing to the project...

That will be awesome!

Thank you! 😊

Hi MammothGoat53
Basically what you are missing are the headers with the Token you have:
https://blog.logrocket.com/secure-rest-api-jwt-authentication/

👍

Hi @<1603198134261911552:profile|ColossalReindeer77>

Hello! does anyone know how to do

HPO

when your parameters are in a

Hydra

Basically hydra parameters are overridden with "Hydra/param"
(this is equivalent to the "override" option of hydra in CLI)

Sounds good.
BTW, when the clearml-agent is set to use "conda" as package manager it will automatically install the correct cudatoolkit on any new venv it creates. The cudatoolkit version is picked direcly when "developing" the code, assuming you have conda installed as development environment (basically you can transparently do end-to-end conda, and not worry about CUDA at all)

This one is used when the agent manually downloads wheels, (pytorch mostly), but as you can see it is under ~/.clearml directory, which usually is already shared on the host

URLs that it was uploaded with, as that URL could change.

How would that change, the actual files are there ?

Hi @<1635088270469632000:profile|LividReindeer58>
You mean the clearml.conf?
You can do:

from clearml.config import config_obj

you should have the entire configuration file as an object (dict interface)
fyi: under the hood it uses pyHOCON

So you could change it down the road if infra/hosting changes.

Internally this is doable and Enterprise edition supports it, at the end this is stored in DBs 🙂

Also in this case, I'm uploading the data to the public file server URL, but my k8 pod can't reach that for security reasons.

Yes, this is solvable as well (again sorry for pointing it, but only in the enterprise version), where you can specify per client or globally:
` path_substitution = [
# Replace regis...

is this a config file on your side or something I can change, if we had enterprise version?

Yes, this is one of the things you can configure

Ohh, hmm, that is odd, there should not be a limit there. Let me check ....

Hmm interesting ...
Any chance you create an Issue on GitHub with this feature suggestion,
If we have some support we could accelerate the implementation

100% of things with 

task_overrides

 would be the most convenient way

I think the issue is that you have to pass the project ID not project name (the project unique IS is the property that is actually stored on the Task)
@<1523707653782507520:profile|MelancholyElk85> can you check the following works:

pipe.add_task(, ..., task_overrides={'project': Task.get_project_id(project_name='examples')},)

Thanks GreasyPenguin14 !

Do you want to open an issue in pip?
Funny enough this works in:

pip3 install "torch >=2.1.0.*, <2.1.1.*" --extra-index-url 

is it there ?