Hmm, let me check something

As I'm a Full-stack developer at Core. I'd be looking to extend the TRAINS Frontend and Backend APIs to suit my need of On-Prem data storage integration and lots of other customization for Job Scheduler(CRON)/Dataset Augmentation/Custom Annot. tool etc.

That is awesome! Feel free to post a specific question here, and I'll try to direct to the right place 🙂

Can you guide me to one such tutorial that's teaching how to customize the backend/front end with an example?

You mean l...

Hi CleanWhale17 let me see if I can address them all

Email Alert for finished Job(I'm not sure if it's already there).

Slack integration will be public by the end of the weekend 🙂
It is fully customization / extendable, I'll be happy to help .

DVC

Full dataset tracking is supported using the artifacts and the ability to integrate to any central storage (shared folders/ S3 / GS / Azure etc.)
From my experience, it is easier to work with artifacts from Data-Processing Tasks...

Yes this is Triton failing to load the actual model file

BTW: @<1673501397007470592:profile|RelievedDuck3> we just released 1.3.1 with better debugging, it prints full exception stack on failure to the clearml Serving Session Task.
I suggest you pull the latest image re run the docker compose and check what you have on the serving session Task in the UI

Oh I see, yes the "metrics" include both scalars / plots & console outputs,
I also think they are updated only once a day (or maybe twice a day?) so even if you delete them it will take to update
(archive is not delete, you then need to go to the archived view and delete it from there)

If you one each "main" process as a single experiment, just don't call Task.init in the scheduler

Out of curiosity, what ended up being the issue?

Hi @<1610083503607648256:profile|DiminutiveToad80>

<h1>Request Entity Too Large</h1>

What's the size of the file? how are you running your clearml-server?

Oh I think that I understand what's going on, @<1523701260895653888:profile|QuaintJellyfish58> let me check how to update the cron scheduler while it is running (I really like this idea, so if this is not already supported I'l like us to add this capability 🙂 )

@<1560074028276781056:profile|HealthyDove84> if you want you can PR a fix, it should be very simple basically:
None

        elif np_dtype == str:
            return "STRING"
        elif np_dtype == np.object_ or np_dtype.type == np.bytes_:
            return "BYTES"
        return None

One issue that I see is that the Dockerfile inside the agent container

Not sure I follow, these are settings for the default container to be used when the agent spins a Task for you.
How are you running the agent itself ?

Nice! So out of curiosity why didn't it work this time and you had to do it manually?

can we use a currently setup virtualenv by any chance?

You mean, if the cleamrl-agent needs to setup a new venv each time? are you running in docker mode ?
(by default it is caching the venv so the second time it is using a precached full venv, installing nothing)

notice that even inside docker the venv is cached on the host machine 🙂

Any reason not to do so in the conf file ?

Notice this is per frame (single) not per 8

It seems to try to p[ull with SSH credentials, add your user/pass(or better APIkey) to the clearml.conf
(look for git_user /git_pass)
Should solve the issue

but I belive it should have work with 0.14.1 as well

Correct

So what you’re saying is to first kick off a new run and then rename the underlying Pipeline Task, which will cause that particular run to become a new pipeline name?

Correct, basically you are not changing the "pipeline" per-se but the execution name of the pipeline, if that makes sense

What would be most ideal would be to be able to right-click on a pipeline run and have a “clone” option, like you can with a task, where you can start a new run with a new name in a single step.

...

Hi @<1691620877822595072:profile|FlutteringMouse14>

In the latest project I created, Hydra conf is not logged automatically.

Any chance the Task.init call is not on the main script (where the Hydra is) ?

hi @<1727497172041076736:profile|TightSheep99>
yes, but also set the dataset chunk filesize to something smaller than the default 500Gb (--chunk-size)
None

I think we added it somewhere in 0.14, anyhow I just checked the Logger doc, it is there now 🙂

Hi @<1523701304709353472:profile|OddShrimp85>

there anywhere I could get a charr that can work with lower version of k8s? Or any other methods?

I think the solution is to install it manually from the helm chart (basically take it out and build a Job YAML, wdyt?

Please let me know what you find 🤞

Hi @<1668427971179843584:profile|GrumpySeahorse51>
Could you provide the full stack log?
this erros seems to originate from psutil (which is used) but it lacks the clearml-session context

MagnificentSeaurchin79 making sure the basics work.
Can you see the 3D plots under the Plot section ?
Regrading the Tensors, could you provide a toy example for us to test ?

LOl my pleasure - I guess we should have a link in the doc string of add_requirements to set_packages , I will tell the guys

Yeah we should definitely have get_requirements 🙂

without the ClearML Server in-between.

You mean the upload/download is slow? What is the reasoning behind removing the ClearML server ?

ClearML Agent per step

You can use the ClearML agent to build a socker per Task, so all you need is just to run the docker. will that help ?