I though the dataset was only linked to the fileserver and not to the specific url used to upload it. (

ShinyRabbit94 yep exactly! the idea is that you can actually do the storage on any solution (S3/GS etc.) the file server is just the default one 🙂

GreasyPenguin14 thank you! that will make our life a lot easier 🙂

Then the type hints are not removed from helper and the code immediately crashes when being run

Oh yes I see your point, that does make sense (btw removing the type hints will solve the issue)
regardless let me make sure this is solved

Where did you add the Task.init call ?

ElegantCoyote26
parser = get_parser() args_ = vars(parser.parse_args()) task.connect(args_)There is no need to connect args_ Task.init will automatically catch the argparser.

JitteryCoyote63 What did you have in mind?

Hi ReassuredTiger98

However, the clearml-agent also stops working then.

you mean the clearml-agen daemon (the one that spinned the container) is crashing as well ?

MelancholyBeetle72 there is an RC with a fix, check the GitHub issue for details :)

Meaning if I create a sleep endpoint that is async

Hmm are you calling "sleep" or "async.sleep"?
Also are you running the serving service with GUNICORN or UVCORN?
see here:
None

time.sleep(time_sleep)

You should not call time.sleep in async functions, it should be asyncio.sleep,
None

See if that makes a difference

MagnificentPig49 quick update, front-end guys updated me that with the next trains-server update they will have the web client code available on the repository , ETA probably mid May or so :)

Hi @<1551376687504035840:profile|StraightSealion9>

AWS Autoscaler to create a new instance when you enqueue a task to the relevant queue.

Does that mean that you were able to enqueue a Task and have it launch on the remote EC2 machine ?

Thanks MagnificentPig49 !

SubstantialElk6 I know they have full permission control in the enterprise edition, if this is something you need I suggest you contact http://allegro.ai 🙂

Hi @<1674588542971416576:profile|SmarmyGorilla62>
You mean on your elastic / mongo local disk storage ?

Basically you create the Task and make sure the "Dataset" is attached to it:
task = Task.init(...) dataset = Dataset.create(task=task) dataset.add_files(...)This will make sure the code is attached to the Dataset

Maybe we should do that automatically ? wdyt?

Also, the IDs as an entry in the Configuration will not be clickable in the web interface, right?

No, but on the other hand, it will be editable if you clone the Task.
Which brings me to a different scenario,
In the original one, the Main Task created the Dataset, i.e. Output Dataset (and stored it both ways).
I could think of a situation the Task is using the Dataset as input (say preprocessing or traing), then we might want to enable users to clone and change the Input dataset. wdyt?

I see, let me check something 🙂

So what will you query ?

DilapidatedDucks58 You might be able to, check the links, they might be embedded into the docker, so you can map diff png file from the host 😛
BTW: what would you change the icons to?

seems like the server returned 400 error, verify that you are working with your trains-server and not the demoserver :)

The latest image seems to require drivers on the host 460+
try this one:
https://docs.nvidia.com/deeplearning/triton-inference-server/release-notes/rel_20-12.html#rel_20-12

Hi SteadyFox10 , unfortunately trains-agent currently supports only docker as a container solution (I guess they became the de-facto standard)
That said, there is the option of virtual environment, where the trains-agent installs everything inside a newly created virtual environment. That actually makes it quite easy to expand to other use cases. Essentially the docker option will spin a docker install trains-agent inside the docker and run it execute command.
Do you fee l...

add_external_files

with a very large number of urls that are

not

in the same S3 folder without running into a usage limit due to the

state.json

file being updated

a lot

?

Hi ShortElephant92
what do you mean the state.json is updated a lot?
I think that everytime you call add_external_files is updated, but add_external_files ` can get a folder to scan, that would be more efficient. How are you using it ?

which to my understanding has to be given before a call to an argparser,

SmarmySeaurchin8 You can call argparse before Task.init, no worries it will catch the arguments and trains-agent will be able to override them :)

Hi DeliciousBluewhale87 ,
Yes they do (I think it's ClearML Enterprise or Allegro ClearML). I also know it has extended capabilities in data management , permissions , and security.
More than that you should probably talk to them directly ( https://clear.ml/contact-us/ ) 🙂

WackyRabbit7
regular trains-agent modus operandi is one job at a time (i.e. until the Task is done, no other Tasks will be pulled from the queue).

When adding --services-mode, it is Not 1-1 but 1-N, meaning a single trains-agent will launch as many Tasks as it can.
The trains-agent pulls a job from the queue and spins a docker (only dockers are supported for the time being) and lets the job run in the background (the job itself will be registered as another "worker" in the system). Then the...

IrritableJellyfish76 hmm maybe we should an an extra argument partial_name_matching=False to maintain backwards compatibility?

Hmm interesting, will pass it along to FE 🙂 3. That is nice! I wonder if this is built into the graph library