send the agent's logs to log management and monitoring service,

These are stored into ELK, it was built to store large amounts of logs, I cannot see any reason why one would want to remove it?

Maybe if there would be a way to change their format, it could also help filtering them from my side.

You mean in the UI?

Do you think this is better ? (the API documentation is coming directly from the python doc-string, so the code will always have the latest documentation)
https://github.com/allegroai/clearml/blob/c58e8a4c6a1294f8acec6ed9cba81c3b91aa2abd/clearml/datasets/dataset.py#L633

https://github.com/allegroai/trains/blob/master/docs/trains.conf#L47

The problem is of course filling in all the configuration details, so that they are viewable.
Other than that, check out:
https://allegro.ai/docs/task.html#trains.task.Task.export_task
https://allegro.ai/docs/task.html#trains.task.Task.import_task
Sounds good ?

Hi RipeGoose2
Any logs on the console ?
Could you test with a dummy example on the demoserver ?

Oh I see, these are to secure your server (basically we recommend you replace the default key/secret 🙂 )
Make sense ?

Correct 🙂

check here None

is there a way to visualize the pipeline such that this step is “stuck” in executing?

Yes there is, the pipelline plot (see plots section on the Pipeline Task, will show the current state of the pipeline.
But I have a feeling you have something else in mind?
Maybe add Tag on the pipeline Task itself (then remove it when it continues) ?
I'm assuming you need something that is quite prominent in the UI, so someone knows ?
(BTW I would think of integrating it with the slack monitor, to p...

Hmm make sense, then I would call the export_task once (kind of the easiest to get the entire Task object description pre-filled for you) with that, you can just create as many as needed by calling import_task.
Would that help?

Regrading the helm, how did you get the link, ? http://github.io ? and the subdomain allegroai?

GleamingGrasshopper63 what do you have configured in the "package manager" section?
https://github.com/allegroai/clearml-agent/blob/5446aed9cf6217f876d3b62226e38f21d88374f7/docs/clearml.conf#L64

Hi @<1523702932069945344:profile|CheerfulGorilla72>
I think more details re needed here:)

And the trains version?

Hi SquareFish25
Sure, here are a few:
HPO
https://github.com/allegroai/trains/blob/master/examples/optimization/hyper-parameter-optimization/hyper_parameter_optimizer.py
Pipeline
https://github.com/allegroai/trains/blob/master/examples/pipeline/pipeline_controller.py
Automation:
https://github.com/allegroai/trains/blob/master/examples/automation/task_piping_example.py

I thought about the fact that maybe we need to write everything in one place

It will be in the same place, under the main Task
Should work out of the box

PompousBeetle71 Check the beginning of the log, it should print the configuration, including the access key (excluding the secret) see if it makes sense...

Hi PleasantGiraffe85
Did you set git_host to only point to your host ? do you expect all the git clones to use SSH? how does the requirements.txt git link looks like ?
https://github.com/allegroai/clearml-agent/blob/bf07b7f76d3236c1118b81730c6d9718705a795a/docs/clearml.conf#L22

(We should probably better state it in the GitHub readme)

thanks @<1715900788393381888:profile|BitingSpider17> for attaching the log it really helps/
Notice from the log:

'-v', '/home/clearml/.clearml/cache:/clearml_agent_cache'

and as expected we also get:

sdk.storage.cache.default_base_dir = /clearml_agent_cache

Yet I can see the error you pointed:

FileNotFoundError: [Errno 2] No such file or directory: '/clearml_agent_cache/storage_manager/datasets'

Now, could it be that the same folder is used for both root and...

Woot woot, will do!

WickedGoat98 let me check...

Was going crazy for a short amount of time yelling to myself: I just installed clear-agent init!

oh noooooooooooooooooo
I can relate so much, happens to me too often that copy pasting into bash just uses the unicode character instead of the regular ascii one
I'll let the front-end guys know, so we do not make ppl go crazy 😉

I mean the caching will work, but it will reinstall this repository on top of the cached copy.
make sense ?

should i only do mongodb

No, you should do all 3 DBs ELK , Mongo, Redis

Thanks a lot. I meant running a bash script after cloning the repository and setting the environment

Hmm that is currently not supported 😞
The main issue in adding support is where to store this bash script...

Perhaps somewhere inside clear ml there is an order of actions for starting that can be changed?

Not that I can think of,
but let's assume you could have such a thing, what would you have put in the bash script (basically I want to see maybe there is a worka...

Make sense 🙂
Just make sure you configure the git user/pass in the docker-compose so the agent has your credentials for the repo clone.

why not let the user start with an empty comparison page and add them from "Add Experiment" button as well?

Apologies, I was not clear. Yes I'm with you, this is a great idea 🙂

Even before we had a chance to properly notice everyone 🙂
Thank you! All the details will follow in a dedicated post, for the time being, I can say that pushing a model with pre/post processing python code and full scalable inference solution has never been easier
https://github.com/allegroai/clearml-serving/tree/main/examples/sklearn

Task.current_task().get_logger().flush(wait=True). # <-- WILL HANG HERE

Okay a bit of theoretical "how it actually works" (and I might be mistaken here...)
Console logging is being reported because the underlining DDP infra (gloo) is pipeline stdout to the main process, where clearml will catch it (I think) The scalars not working on the subprocesss & the flush wait stuck I think are related, as the wait actually waits for the flush process, and it seems it cannot actually "talk" to i...