delete logged images and texts though

logged images are also stored there?

Sorry I missed the additional "." in the _update_requirements
Let me check ....

Weird issue, I'll make sure we fix compatibility with python 3.9

JitteryCoyote63

IAM role to the web app could access

you mean the web client key/secret to access S3 data ?

but I don’t get to this line, because my task is already of type data_processing

Ohh I see now, it should have added the Tag regardless, you are correct.

Hi SquareFish25
Sure, here are a few:
HPO
https://github.com/allegroai/trains/blob/master/examples/optimization/hyper-parameter-optimization/hyper_parameter_optimizer.py
Pipeline
https://github.com/allegroai/trains/blob/master/examples/pipeline/pipeline_controller.py
Automation:
https://github.com/allegroai/trains/blob/master/examples/automation/task_piping_example.py

Thanks!

Sure, venv mode

but I am think they done it for a reason no?

Not a very good one, they just installed everything under the user and used --user for the pip.
It really does not matter inside a docker, the only reason one might want to do that is if you are mounting other drives and you want to make sure they are not accessed with "root" user, but with 1000 user id.

No I mean configure the files_server in the clearml.conf

Hi @<1523701601770934272:profile|GiganticMole91>

to use https although the scheduled task is using ssh for git?

Sure as long as it has git_user / git_pass configured in the agents clearml.conf it will automatically convert ssh to http git pull
None

Basically the idea is that you create the pipeline once (say debug), then once you see it is running, you have a Task of your pipeline in the system (with any custom logic you added). With a Task in the system you can always clone/modify and launch externally (i.e. from code/ui. Make sense ?

OutrageousSheep60

I found the task in the UI -

and in the

UNCOMMITTED CHANGES

execution section there is

No changes logged

This is the issue.

and then run the

session

via docker

clearml-session --docker nvidia/cuda:10.1-cudnn7-runtime-ubuntu18.04 \ --packages "clearml" "tensorflow>=2.2" "keras" \ --queue MY_QUEUE \ --verboseAre you running the "cleamrl-session" from your machine? (i.e. not from inside a docker) ?...

No Task.create is for creating an external Task not logging your own process,
That said you can probably override the git repo with env vars:
None

are you planning on changing to f-strings incrementally?

There is still py 2.7 & 3.5 support...
Hopefully we will be able to drop both (apparently enough users have legacy code), then we will probably switch to the nicer f' strings 🙂

Hi @<1523702307240284160:profile|TeenyBeetle18>

and url of the model refers to local file, no to the remote storage.

Do you mean that in the Model tab when you look into the model details the URL points to a local location (e.g. file:///mnt/something/model) ?
And your goal is to get a copy of that model (file) from your code, is that correct ?

Sure thing :)

WittyOwl57 what about? vm.max_map_count echo "vm.max_map_count=262144" > /tmp/99-clearml.conf
sudo mv /tmp/99-clearml.conf /etc/sysctl.d/99-clearml.conf
sudo sysctl -w vm.max_map_count=262144
sudo service docker restart `https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server_linux_mac (5)

This really makes little sense to me...

Can you send the full clearml-session --verbose console output ?

Something is not working as it should obviously, console output will be a good starting point

But I'm sure there is a cleaner way to proceed.

Maybe ?!
path = task.get_output_destination().replace('file://', '', 1)

LudicrousParrot69 you mean post execution or while you are executing the hyperparameter optimizer ?

Hopefully once things calm down at work I will find more time.

Sounds good 🙂

You can always log it manually:
from clearml import InputModel input_model = InputModel.import_model(weights_url='/tmp/keras_example/weight.6.hdf5')

Correct 🙂

I believe that happens natively thanks to pyhocon? No idea why it fails on mac

That's the only explanation ...
But the weird thing is, it did not work on my linux box?!
Sounds good let's work on it after the weekend, 🙂

@<1687643893996195840:profile|RoundCat60> can you access the web UI over https ?

Hmm can you test with the latest RC?
pip install clearml==0.17.6rc1

Yes, this is exactly how the clearml k8s glue works (notice the resource allocation, spin nodes up/down, is done by k8s which sometimes do take some time, if you only need "bare metal nodes" on the cloud, it might be more efficient to use the aws autoscaler, that essentially does the same thing

ShallowGoldfish8 how did you get this error?
self.Node(**eager_node_def) TypeError: __init__() got an unexpected keyword argument 'job_id'

in this week I have met at least two people combining ClearML with other tools (one with Kedro and the other with luigi)

I would love to hear how/what is the use case 🙂

If I run the pipeline twice, changing only parameters or code of taskB, ...

I'll start at the end, yes you can clone a pipeline in the UI (or from code) and instruct it to reuse previous runs.
Let's take our A+B example, Let's say I have a pipeline P, and it executed A and then B (which relies on A's output...