Hi WackyRabbit7 ,
Regrading git credentials, see here in the trains.conf https://github.com/allegroai/trains-agent/blob/master/docs/trains.conf#L18

Trains assumes one of two (almost three) possible setups
Your code/script is in a git repository. Then when executing manually all the git references incl` uncommitted changes are stored. Then when executing with the trains-agent, it will clone the code based on these references apply the uncommitted changes and run your code. To do that the ...

Which works for my purposes. Not sure if there's a good way to automate it

Interesting, so if we bind to hydra.compose it should solve the issue (and of course verify we are running on a jupyter notebook)
wdyt?

How do you run the 

clearml-agent

 in docker mode

clearml-agent --docker
See here:
https://clear.ml/docs/latest/docs/clearml_agent#docker-mode

Let me check...

🙏

Hi OutrageousSheep60

Is there a way to instantiate a

clearml-task

while providing it a

Dockerfile

that it needs to build prior to executing the task?

Currently not really, as at the aned the agent does need to pull a container,
But you can cheive basically the same by adding the "dockerfile" script as --docker_bash_setup_script Notice of course that this is an actual bash script not Docker script, so no need for "RUN" prefix.
wdyt?

sounds good 🙂

IrritableOwl63 in the profile page, look at the bottom right corner

ThickDove42 Windows also works 😞
Any specifics on the setup?

Correct, (if this is running on k8s it is most likely be passed via env variables , CLEARML_WEB_HOST etc,)

trained model class...

You mean the pytorch model object?

URLs that it was uploaded with, as that URL could change.

How would that change, the actual files are there ?

Ohh sorry you will also need to fix the def _patched_task_function
The parameter order is important as the partial call relies on it.

IntriguedRat44 how do I reproduce it ?
Can you confirm that marking out the Task.init(..) call will fix it ?

Hi SubstantialElk6

but in terms of data provenance, its not clear how i can associate the data versions with the processes that created it.

I think DeliciousBluewhale87 ’s approach is what we are aiming for, but with code.
So using clearml-data from CLI is basically storing/versioning of files (with differentiable based storage etc, but still).
What ou are after (I think) is in your preprocessing code using the programtic Dataset class, to create the Dataset from code, this a...

Hi ScantChimpanzee51
btw: this seems like an S3 internal error
https://github.com/boto/s3transfer/issues/197

I think I was not able to fully express my point. Let me try again.
When you are running the pipeline Fully locally (both logic and components) the assumption is this is for debugging purposes.
This means that the code of each component is locally available, could that be a reason?

Hover over the border (I would suggest to use the full screen, i.e. maximize)

My main issue with this approach is that it breaks the workflow into “a-sync” set of tasks:

This is kind of the way you depicted it, meaning, there is an an initial dataset, "offline process" (i.e. external labeling) then, ingest process.

I was wondering if the “waiting” operator can actually be a part of the pipeline.
This way it will look more clear what is the workflow we are executing.

Hmm, so pipeline is "aborted", then the trigger relaunches the pipeline, and the pipeli...

You can however pass a specific Task ID and it will reuse it "reuse_last_task_id=aabb11", would that help?

Hmm I'm sorry it might be "continue_last_task", can you try:
Task.init(..., continue_last_task="aabb11")

But in credentials creation it still shows 8008. Are there any other places in docker-compose.yml where port from 8008 to 8011 should be replaced?

I think there is a way to "tell" it what to out there, not sure:
https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server_config#configuration-files

Something like the TYPE_STRING that Triton accepts.

I saw the github issue, this is so odd , look at the triton python package:
https://github.com/triton-inference-server/client/blob/4297c6f5131d540b032cb280f1e[…]1fe2a0744f8e1/src/python/library/tritonclient/utils/init.py

and since the update the docs seem to be a bit off but I think I got it

Working on a whole new site 😉

Notice you have configure the shared driver for the docker, as the volume mount doesn't work without it. https://stackoverflow.com/a/61850413

Hi BurlyRaccoon64
What do you mean by "custom_build_script" ? not sure I found it in "clearml,conf"
https://github.com/allegroai/clearml-agent/blob/master/docs/clearml.conf

Would you have an example of this in your code blogs to demonstrate this utilisation?

Yes! I definitely think this is important, and hopefully we will see something there 🙂 (or at least in the docs)

Let me check something

  is the "installed packages" part editable? good to know

Of course it is, when you clone a Task everything is Editable 🙂

Isn't it a bit risky manually changing a package version?

worst case it will crash quickly, and you reset/edit/enqueue 🙂
(Should work though)

Yes I think the writer.add_figure somehow crops the image

Does this mean that I need to create multiple ssh keys? 1 key for each user?

I think so

Use .git-credentials

This might also support multiple user/repo