But pytorch has no specific backend, it uses TB.
No?! Can you point me to an example? What I mostly find is how to calc metrics not standard way to then store them...

PompousBeetle71 If this is argparser and the type is defined, the trains-agent will pass the equivalent in the same type, with str that amounts to '' . make sense ?

And is there an easy way to get all the metrics associated with a project?

Metrics are per Task, but you can get the min/max/last of all the tasks in a project. Is that it?

time-based, dataset creation, model publish (tag),
Anything you think is missing ?

Good news a dedicated class for exactly that will be out in a few days 🙂
Basically task scheduler and task trigger scheduler, running as a service cloning/launching tasks either based on time (cron alike) or based on a trigger).
wdyt?

Hi FlutteringWorm14

Is there some way to limit that?

What do you mean by that? are you referring to the Free tier ?

Hi PompousBeetle71
I remember it was an issue, but it was solved a while ago. Which Trains version are you using?

UnevenDolphin73 if the repo does not include a poetry file it will revert to pip

Hi GreasyPenguin14

However the cleanup service is also running in a docker container. How is it possible that the cleanup service has access and can remove these model checkpoints?

The easiest solution is to launch the cleanup script with a mount point from the storage directory, to inside the container ( -v <host_folder>:<container_folder> )
The other option, which clearml version 1.0 and above supports, is using the Task.delete, that now supports deleting the artifacts and mod...

yes 🙂

Let me check... I think you might need to docker exec
Anyhow, I would start by upgrading the server itself.
Sounds good?

Hi ElegantCoyote26
what's the clearml version you are using?

ClumsyElephant70
Could it be virtualenv package is not installed on the host machine ?
(From the log it seems you are running in venv mode, is that correct?)

Hi GrittyCormorant73
At the end everything goes through session.send, you can add a print there?
btw: why would you print all the requests? what are we debugging here?

Is this information stored anywhere or do I need to explicitly log this data somehow?

On the creating Task along side all the other reports.
Basically each model stores its creating Task (Task ID), using the Task ID you can query all the metrics reported by the task

I'm guessing this is done through code-server?

correct

I'm currently rolling a JupyterHub instance (multiuser, with codeserver inside) on the same machine as clearml-server. That’s where tasks are executed etc. so, all browser dev env.

Yeah, the idea with clearml-session each user can self serve themselves the container that works best for them. With a jupyterhub they start to step on each other's toes very quickly ...

I think you are correct, this is odd, let me check ...

Can you send the full log ?

is it planned to add a multicursor in the future?

CheerfulGorilla72 can you expand? what do you mean by multicursor ?

based on this one:
https://stackoverflow.com/questions/31436407/git-ls-remote-returns-fatal-no-remote-configured-to-list-refs-from
I think this is a specific issue of the local git repo configuration, can you verify
(btw: I tested with git 2.17.1 git ls-remote --get-url will return the remote url, without an error)

Yeah.. that should have worked ...
What's the exact error you are getting ?

Another point I see is, that in the workers & queses view the GPU usage is not been reported

It should be reported, if it is not, maybe you are running the trains-agent in cpu mode ? (try adding --gpus)

Yes! I checked it should work (it checks if you have load(...) function on the preprocess class and if you do it will use it:
None

def load(local_file)
    self._model = joblib.load(local_file_name)
    self._preprocess_model = joblib.load(Model(hard_coded_model_id).get_weights())

Hi GiganticTurtle0
ClearML will only list the directly imported packaged (not their requirements), meaning in your case it will only list "tf_funcs" (which you imported).
But I do not think there is a package named "tf_funcs" right ?

Full markdown edit on the project so you can create your own reports and share them (you can also put links to the experiments themselves inside the markdown). Notice this is not per experiment reporting (we kind of assumed maintaining a per experiment report is not realistic)

Like this one:
https://demoapp.trains.allegro.ai/projects/380af79766304cf78b40e15656013d89/experiments/c41d64546aa54414b7c848a5261e85c7/output/debugImages

Ohh then YES!
the Task will be closed by the process, and since the process is inside the Jupyter and the notebook kernel is running, it is still running

DefiantCrab67
Where will you copy it from ?

Hi FunnyTurkey96
Which pip are you using, basically pip changed the dependency resolver after 20.1
Change: https://github.com/allegroai/clearml-agent/blob/aede6f4bac71c8fc56e7cf982318a48527953a3c/docs/clearml.conf#L57
pip_version: "<20.2"See if that helps

Hi JitteryCoyote63
Show running experimentsIt doesn't?
Have the legend clickable, to hide/show experiments based on their status:+1:
Have a line connecting points that are SOTA (example in https://paperswithcode.com/sota/image-generation-on-cifar-10 )I like that, how is that selected? (I know FE are thinking of replacing this entire graph library, so maybe good timing in terms of what to look at)