When i say accessing, it means i want to use the data for training(without actually getting a local copy of it ).

How can you "access" it without downloading it ?
Do you mean train locally on a subset, then on the full dataset remotely ?

Are you using zsh by any chance?

If the right properties are set can the profile tab be added?

I guess that is doable, that said some of the graphs are not straight forward to support like this one:
https://www.tensorflow.org/guide/images/tf_profiler/trace_viewer.png

repeat it until they are all dead 🙂

Great to hear SourSwallow36 , contributions are always appreciated 🙂
Regrading (3), MongoDB was not build for large scale logging, elastic-search on the other hand was build and designed to log millions of reports and give you the possibility to search over them. For this reason we use each DB for what it was designed for, MongoDB to store the experiment documents (a.k.a env, meta-data etc.) and elastic-search to log the execution outputs.

Also, I would like to add some other plots t...

Is there any references (vlog/blog) on deploying real-time model and do the continuous training pipeline in clear-ml?

Something along the lines of this one ?
https://clear.ml/blog/creating-a-fully-automatic-retraining-loop-using-clearml-data/
Or this one?
https://www.youtube.com/watch?v=uNB6FKIi8Wg

Hey JitteryCoyote63 I think I need to better explain the config feature:
agent.package_manager.post_packages = ["PyJWT"]Basically this means that IF you have pyjwt in the installation package it will be installed after everything else is installed.
This doesn't mean it will always be installed.
Think for example "horovod" has to be installed after you have TF / PyTorch installed.
(The same goes for "pre_package" and Cython)

This would work to load the local modules, but I’m also using poetry and the

pyproject.toml

is in the subdirectory, so the agent won’t install any dependency if I don’t set the

work_dir

hmmm true, in terms of requirements, you can list them in the decorator (see packages argument)

Sure. JitteryCoyote63 so what was the problem? can we fix something?

Really stoked to start using it and introduce a more sane ML ops workflow at my workplace lol.

Totally with you 🙂

... would that be a 

Model Registry Store

 plugin?

YES please ❤
So we actually just introduced "Applications" into the clearml free tier, https://app.community.clear.ml/applications
Allowing you to take any Task in the system and make it an "application" (a python script running on one of the service agents), with the ability to configu...

it is using the wrong URL - an https:// instead of an s3:// so it can't upload the preprocess.py file

What do you see as the link in the UI for that specific model ?
What is the full model add command ?

Hi @<1523706645840924672:profile|VirtuousFish83>
Hmm so generally I think the answer is no... I mean you can download all scalars and re-report them with a different title/series, but I think you will not be able to delete a specific set, and the only way would be to reset the entire Task.
I'm curious what's the scenario here? is it like a typo you want to fix?

Btw I sometimes get a gzip error when I am accessing artefacts via the '.get()' part.

Hmm this is odd, is this a download issue? if this is reproducible maybe we should investigate further...

it is a pickle issue
‘package model doesn’t exist’

Sounds like it, why do you think clearml has anything there ?
BTW:

import_bind

.

__patched_import3

this is just so when packages that clearml autoconnects with are patched if imported After Task.init was called.

Hi UnsightlySeagull42
How can I reproduce this behavior ?
Are you getting all the console logs ?
Is it only the Tensorboard that is missing ?

but can it NOT use /tmp for this i’m merging about 100GB

You mean to configure your Temp folder for when squashing ?
you can do hack the following:
` import tempfile
tempfile.tempdir = "/my/new/temp"

Dataset squash

tempfile.tempdir = None `But regradless I think this is worth a GitHub issue with feature request, to set the temp folder///

task=Task.current_task()
Will get me the task object. (right?)

PanickyMoth78 yes, always, from anywhere, this is a singleton object 🙂

Hi @<1545216070686609408:profile|EnthusiasticCow4>
Oh dear, I think this argument is not exposed 😞

• You can open a GH
• If you want to add a PR this is very simple:None

include_archived=False,
):
   if not include_archived:
        system_tags = ["__$all", cls.__tag, "__$not", "archived"]
    else:
        system_tags = [cls.__tag]

...
   system_tag...

What's the pip verison?

or
pip install -U trains

This seems to only work for a single file (weights_path implies a single file, not multiple ones). Is that the case?See update_weights_package actually packages an entire folder as zip and will do the extraction when you get it back (check the function docstring, I think you can also specify wildcard etc if needed)

Why do you see this as preferred to the dataset method we have now?

So it answers a few requirements that you raised
It is fully visible as part of the project and se...

It talks about referencing an issue.

Yes please, just better visibility 🙏

Why does my task execution freeze after pip installation (running agent in foreground mode)?

Hi AdventurousButterfly15
Are you running in agent docker mode or venv mode ?
What do you mean freeze? do you see anything on the Taks console log in the UI? what's the host OS ?

Hi @<1697056701116583936:profile|JealousArcticwolf24>
Can you run your pipeline on an agent (i.e. remotely) but launching it from the UI (not the taskscheduler)?

You can see in the log it tries to download an artifact from a specific IP:URL is that link a valid one?
(this seems like the main cause of the error, first line in the screenshot)

If a Task is in the 'Completed' I think the only option is to 'Reset' it (see image).

In the UI yes, in code you can do task.mark_aborted(force=True)

You do clear the previous run execution but I think for a repetitive task this is fine.

I would avoid that, no?

RobustGoldfish9 do you see the trains-agent listed as a machine in the UI (under workers)

YummyWhale40 no idea what the pytorch-lighting guys did there. let me check a the actual code.

btw: any specific reason to call current_task after you closed the main Task ?

Sure thing, let me know ... 🙂