Upgrade back?

PleasantGiraffe85

it took the repo from the cache. When I delete the cache, it can't get the repo any longer.

what error are you getting ? (are we talking about the internal repo)

FrustratingWalrus87 If you need active one, I think there is currently no alternative to TB tSNE 🙂 it is truly great 🙂
That said you can use plotly for the graph:
https://plotly.com/python/t-sne-and-umap-projections/#project-data-into-3d-with-tsne-and-pxscatter3d
and report it to ClearML with Logger report_plotly :
https://github.com/allegroai/clearml/blob/e9f8fc949db7f82b6a6f1c1ca64f94347196f4c0/examples/reporting/plotly_reporting.py#L20

Should I use 

update_weights_package

Yes
BTW, config.pbtxt you should pass when "registering" the endpoint with the CLI

one can containerise the whole pipeline and run it pretty much anywhere.

Does that mean the entire pipeline will be running on the instance spinning the container ?
From here: this is what I understand:
https://kedro.readthedocs.io/en/stable/10_deployment/06_kubeflow.html

My thinking was I can use one command and run all steps locally while still registering all "nodes/functions/inputs/outputs etc" with clearml such that I could also then later go into the interface and clone an...

Hmm can you test with the latest RC? or even better from the GitHub (that said the Github will break the interface, as we upgraded the pipeline 🙂 )

RipeGoose2 https://github.com/allegroai/trains/blob/f27aed767cb3aa3ea83d8f273e48460dd79a90df/trains/task.py#L2072

great!

BTW: 0.14.3 solved the issue you are referring to, so you can import trains before / parsing the args without an issue. Regrading passing project/name as parameters. A few thoughts: (1) you can always rename / move projects from the UI (2) If you are running it with trains-agent there is no meaning to these arguments, as by definition the Task was already created... Maybe we should give an option to exclude a few arguments from argparser, I think this topic came up a few times... What d...

ElegantKangaroo44 I think TrainsCheckpoint would probably be the easiest solution. I mean it will not be a must, but another option to deepen the integration, and allow us more flexibility.

What was the url you ended up with ?

Now in case I needed to do it, can I add new parameters to cloned experiment or will these get deleted?

Adding new parameters is supported 🙂

It appears that "they sell that" as Triton Management Service, part of

. It is possible to do through their API, but would need to be explicit.

We support that, but this is Not dynamically loaded, this is just removing and adding models, this does not unload them from the GRAM.
That's the main issue. when we unload the model, it is unloaded, to do dynamic, they need to be able to save it in RAM and unload it from GRAM, that's the feature that is missing on all Triton deployme...

So you are saying 156 chunks, with each chunk about ~6500 files ?

but it still not is able to run any task after I abort and rerun another task

When you "run" a task you are pushing it to a queue, so how come a queue is empty? what happens after you push your newly cloned task to the queue ?

Hi @<1715175986749771776:profile|FuzzySeaanemone21>

and then run "clearml-agent daemon --gpus 0 --queue gcp-l4" to start the worker.

I'm assuming the docker service cannot spin a container with GPU access, usually this means you are missing the nvidia docker runtime component

with 

PipelineController

, is there any way to avoid creating a new development environment for each step of the pipeline?

You are in luck, we are expanding the PipelineController to support functions. basically allowing you to run the step on the node running the entire pipeline, but I'm not sure this covers all angles of the problem.
My main question here is, who/how the initial setup is created by cleaml-agent ?

I would like to be more efficient and re-use that ...

I think that just backing up /opt/clearml and moving it should be just fine 🤞

It all depends how we store the meta-data on the performance. You could actually retrieve it from the say val metric and deduce the epoch based on that

ReassuredTiger98 regrading the agent error, can you see the package some_packge in the "Installed Packages" in the UI? Was it installed ? are you using pip or conda as package manager in the agent (check the clearml.conf) is the agent running in docker mode ?

${PWD} works!

This will be resolved every call to Task.init (so I would recommend against it), how about "$HOME/" ?

TrickySheep9 Yes, let's do that!
How do you PR a change ?

It will store everything locally, later you can import it back to the server, if you want.

Also, can the image not be pulled from dockerhub but used from the local build instead?

If you have your docker configured to pull from local artifactory, then the agent will do the same 🙂 (it is calling the docker command just like you do)

agent.default_docker.arguments: "--mount type=bind,source=$DATA_DIR,target=/data"

Notice that you are use default docker arguments in the example
If you want the mount to always be there use extra_docker_arguments :
https://github.com/...

Creating a dataset sounds like a good idea, but that does not seem to be the issue.
Can you verify you can manually clone using the same link (notice the log should specify the exact clone it is using, with the password replaced with *)

JitteryCoyote63 if this is simulating an agent, the assumption is that the Task was already created, hence the task ID.

If i am working with Task.set_offline(True)

How would the two combine ? I mean off-line is be definition not executed by an agent, what am I missing ?

My only point is, if we have no force_git_ssh_port or force_git_ssh_user we should not touch the SSH link (i.e. less chance of us messing with the original URL if no one asked us to)

I think that is good enough

I wonder, does it launch all "step two" instances in parallel ?
In theory it should , but in practice since these are the same "template" I'm not sure what would happen.
One last note, you can call PipelineDecorator.debug_pipeline() to debug the pipeline locally, it will have the exact same behavior only it will run the steps as subprocesses.

👍