Let me check if we can hack something...

It was set to true earlier, I changed it to false to see if there would be any difference but doesn’t seem like it

I would actually just add:
Task.add_requirements('google.cloud')Before the Task.init call (Notice, it has to be before the the init call)

The problem is of course filling in all the configuration details, so that they are viewable.
Other than that, check out:
https://allegro.ai/docs/task.html#trains.task.Task.export_task
https://allegro.ai/docs/task.html#trains.task.Task.import_task
Sounds good ?

BoredHedgehog47 that actually depends on the container, are you running as root inside the container ?
if not I think the easiest hack is to always map /etc/hosts as a k8s secret file?

If this is GitHub/GitLab/Bitbucket what I'm thinking is just a link opening an iframe / tab with the exact entry point script / commit.
What do you think?

Does that work?

1. Yes the main diff between add task and decorator is basically creating dag and " executes " the tasks in parallel, based on the dag dependencies
2. Decorator will also take care of serializing the data in / out of the function. Imagine the pipeline logic is running as python code where the logic will wait for the function to finish only when the result of the function is being used. This means that if you need a parllel loop you can create thread pool.
   Make sense

Agreed, MotionlessCoral18 could you open a feature request on the clearml-agent repo please? (I really do not want this feature to get lost, and I'm with you on the importance, lets' make sure we have it configured from the outside)

SillyPuppy19 yes you are correct, actually I can promise you the callback will be called from a different thread (basically the monitoring thread) so it's on the user to make sure the callback can handle it .
How about we move this discussion to GitHub?

Hi @<1571308003204796416:profile|HollowPeacock58>

parameters = task.connect(config, name='config_params')

It seems that your DotDict does not support the python copy operator?
i.e.

from copy import copy
copy(DotDict())

fails ?

Okay, I'll make sure we change the default image to the runtime flavor of nvidia/cuda

WickedGoat98 are you running the agent with --gpus ?

size 0

Basic setup:
glues service per "job template" (e.g. k8s resources, for example cpu requirement, or gpu requirement).
queue per glue service, e.g. cpu_machine queue, and 1xGPU queue
wdyt?

but this would be still part of the clearml.conf right?

You can pass it per Task , also you can configure the agent to always pass it add this env.
https://github.com/allegroai/clearml-agent/blob/5a080798cb4292e198948fbe16cba70136cb6bdf/docs/clearml.conf#L137

@<1535793988726951936:profile|YummyElephant76>

Whenever I create any task the "uncommitted changes" are the contents of

ipykernel_launcher.py

, is there a way to make ClearML recognize that I'm running inside a venv?

This sounds like a bug, it should have the entire notebook there, no?

Hi SkinnyPanda43
Do you mean the cleaml-agent or the cleaml python (a.k.a the auto package detection) ?

Hi @<1631102016807768064:profile|ZanySealion18>

I'm using SSH for authentication, however, known_hosts doesn't seem to be passed to the docker so it prompts for authentification/fingerprint. Any ideas?

Hmm it is supposed to automatically mount your ~/.ssh folder into the docker to solve for that.
First try to set force_git_ssh_protocol: true
None
If that does not he...

sounds good, CheerfulGorilla72 could I ask you to open a github issue and suggest it? just so we do not forget ?

JitteryCoyote63 What did you have in mind?

HurtWoodpecker30 could it be you hit a limit of some sort ?

LOL I see a meme waiting for GrumpyPenguin23 😉

With default settings, to upload 2 datasets of 120 GB and 70 Gb it took more than 6 hours!

SmugSnake6 at the end s the an outcome of limited bandwidth or limited CPU ?

is how you would create different queues,

SarcasticSquirrel56 you can create them from the UI, when the server is already running
(if you are saying, how do I create them in the first installaiton, then yes you are correct, this is possible in the helm chart, I think 😞 )

I'm not sure this is configurable from the outside 😞

AstonishingWorm64 I found the issue.
The cleamlr-serving assume the agent is working in docker mode, as it Has to have the triton docker (where triton engine is installed).
Since you are running in venv mode, tritonserver is not installed, hence the error

named as 

venv_update

 (I believe it's still in beta). Do you think enabling this parameter significantly helps to build environments faster?

This is deprecated... it was a test to use the a package that can update pip venvs, but it was never stable, we will remove it in the next version

Yes, I guess. Since pipelines are designed to be executed remotely it may be pointless to enable an 

output_uri

 parameter in the 

PipelineDecorator.componen...

Are you saying you had that odd script entry-point created by calling Task.init? (To clarify this is the problem)
Btw after you clone the experiment you can always manually edit both entry point and working dir, which based on what you said should be "script.py" and "folder"

According to you the VPN shouldn't be a problem right?

Correct as long as all parties are on the same VPN it should work, all the connections are always http so basically trivial communication

By your description it seems to make no difference whether I added the files via sync or add, since I will have to create a new dataset either way.

Sync is design to take a local folder/s and add/remove files from a dataset based on the local changes (it does that automatically based on file existence / content)
The changes (i.e. added files) are uploaded as delta changes relative to the parent version, this means we are not always uploading all files.

Add on the other hand means you...