Hi AbruptHedgehog21

How i can add S3 credentials to S3 bucket in example.env for clearml-serving-triton? I need to add bucket name, keys and endpoint

Basically boto (s3) environment variables would just work:
https://clear.ml/docs/latest/docs/clearml_serving/clearml_serving#advanced-setup---s3gsazure-access-optional

Oh right, I missed the fact the helper functions are also decorated, yes it makes sense we add the tags as well.
Regarding nested pipelines, I think my main question is , are they independent or are we generating everything from the same code base?

Hi DeliciousBluewhale87
My theory is that the clearml-agent is configured correctly (which means you see it in the clearml-server). The issue (I think) is that the Task itself (running inside the docker) is missing the configuration. The way the agent passes the configuration into the docker is by mapping a temporary configuration file into the docker itself. If the agent is running bare-metal, this is quite straight forward. If the agent is running on k8s (or basically inside a docker) th...

nfs version 3

That's the thing, NFS will automatically set file access and flags based on the mount options you cannot change them post mount.
How about creating a new user just for the agent, it makes sense from security / credentials perspective

HandsomeCrow5 I see, my bad.
BTW: Did you see this one?
https://github.com/allegroai/trains/blob/master/examples/optimization/hyper-parameter-optimization/hyper_parameter_optimizer.py
And the helper classes here: https://github.com/allegroai/trains/tree/master/trains/automation

AstonishingSeaturtle47 that's awesome! Could you explain the hack, it might be helpful for others (I assume :))

I'm glad to hear 🙂
If you can reproduce it, let me know

Hi BoredHedgehog47
Just make sure it is installed as part of the "installed packages" 🙂
You should end up with something like
git+You can actually add it from your code:
Task.add_requirements("git+ ") task = Task.init(...)Notice you can also add a specific commit or branch git+ https://github.com/user/repo.git@ <commit_id_here_if_needed>

Is this what you are looking for ?

EDIT:
you can also do "-e ." that should also work:
` Task.add_requirements("-e .")
task = Ta...

JitteryCoyote63 are you running the agent in docker mode ?

Update us if it solved the issue (for increased visibility)

will my datasets be stored on the same machine that hosts the clearml server?

By default yes, they will be stored to the files-server (but you can change it, this is an argument for both the CLI and the python interface)

I'm just trying to see what is the default server that is set, and is it responsive

I'm assuming you mean your own server, not the demo server, is that correct ?

and then second part is to check if it is up and alive

Yes, you can curl to the ping endpoint :
https://clear.ml/docs/latest/docs/references/api/debug#post-debugping

Is there a way to do this all elegantly?

Of yes there is, this is how TaskB code will look:

` task = Task.init(..., 'task b')
param = {'TaskA' :'TaskAs ID HERE'}
task.connect(param)
taska_model = Task.get_task(param['TaskA']).models['output''][-1]
torch.load(taska_model.get_local_copy())

train

torch.save('modelb') `I might have missed something there, but generally speaking this will let you:
Select TASKA as a parameter of TaskB training process Will register automagically Tasks'A...

Thanks PompousBaldeagle18 !

Which software you used to create the graphics?

Our designer, should I send your compliments 😉 ?

You should add which tech is being replaced by each product.

Good point! we are also missing a few products from the website, they will be there soon, hence the "soft launch"

FriendlySquid61 what do you think?

pip install "pyjwt<2.0.0"

copy paste the trains.conf from any machine, it just need the definition of the trains-server address.
Specifically if you run in offline mode, there is no need for the trains.conf and you can just copy the one on GitHub

GrievingTurkey78 whats the repository link you see in the UI? Does it start with ssh:// or https://
Did you add git_user/git_pass to the trains.conf of the trains-agent ? (if you did it should replace any ssh:// link with https:// user/pass link)

I look forward to your response on Github.

Great, I would like to make this discussion a bit more open and accessible so GitHub is probably better

I'd like to start contributing to the project...

That will be awesome!

Hi EnviousStarfish54
After the pop up do you see the plot on the web UI?

GrievingTurkey78 can you send the entire log?

PompousParrot44 unfortunately not yet 😞
But the gist is :
MongoDB stores experiment data (i.e. execution parameters, git ref etc.)
ElasticSearch stores results (i.e. metrics console logs, debug image links etc.)
Does that help?

his means that you guys internally catch the argparser object somehow right?

Correct 🙂 this is how you get the type checking casting abilities, and a few other perks

We are planning an RC later this week, I'll make sure this fix is part of it

MotionlessCoral18 I think there is a fix in the latest clearml-agent RC 1.4.0rc0 can you test and update if your are still having this issue?

Actually unless you specifically detached the matplotlib automagic, any plt.show() will be automatically reported.

GrievingTurkey78 please feel free to send me code snippets to test 🙂

Really what I need is for A and B to be separate tasks, but guarantee they will be assigned to the same machine so that the clearml dataset cache on that machine will be warm.

I think that what you are looking for is multi-machine cache (which is fully supported). Basically mount an NFS/SMB folder from a NAS to any of those machines, configure the cache folder to point to it, and not you do not need to worry about affinity ?
no?

Is there a way to group A and B into a sub-pipeline, h...

Thanks FiercePenguin76 , I can totally understand your point on running proper tests, and reluctance to break other things.
I suggest to add a comment with the temp fix that solved the problem for you, and we will make sure the team takes it from there. wdyt?

The driver script (the one initializes models and initializes a training sequence) was not at git repo and besides that one, everything is.

Yes there is an issue when you have both git repo and totally uncommitted file, since clearml can store either standalone script or a git repository, the mix of the two is not actually supported. Does that make sense ?