so moving b in to a won’t work if some subfolders are already there

I though that if they are already there you would merge / overwrite, isn't that what you need ?
a/b/c/2.txt seems like the result of moving b from dataset B into folder b in Dataset A, what am I missing?
(My assumption is that you have both datasets locally on the same machine and that you can just copy the files from b of Datasset B into b folder of Dataset A)

You cannot call exit(0) and kill the kernel from the SageMake notebook

Hi WhimsicalLion91
You can always explicitly send a value:
from trains import Logger Logger.current_logger().report_scalar("title", "series", iteration=0, value=1337)A full example can be found here:
https://github.com/allegroai/trains/blob/master/examples/reporting/scalar_reporting.py

Are you inheriting from their docker file ?

Hmm I wonder, can you try with this line before?
Task._report_subprocess_enabled = False frameworks = { 'tensorboard': True, 'pytorch': False } Task.init(...)

making me realize that this may have been optional

I think it is optional, and this is why it was not entered in the first place.
Could you double check and just remove it from your manual pbtxt ?

Hi EnviousStarfish54
Color coding on the entire UI is stored per user (I think that on your local cookies, but I might be wrong). Anyhow any title/series combination will have the select color regardless of the project.
This way you can configure once that loss is red and accuracy is green, etc.

(Just a thought, maybe we just need to combine Kedro-Viz ?)

Hi LethalDolphin75
I think you are right there isn't one (although I remember a discussion about it...)
Anyhow it will be very easy to implement, just inherit from:
https://github.com/allegroai/clearml/blob/400c6ec103d9f2193694c54d7491bb1a74bbe8e8/clearml/automation/parameters.py#L111
And return the power of the parent value here:
https://github.com/allegroai/clearml/blob/400c6ec103d9f2193694c54d7491bb1a74bbe8e8/clearml/automation/parameters.py#L146
And
https://github.com/allegroai/...

SmugOx94 Yes, we just introduced it 🙂 with 0.16.3
Discussion was here (I'll make sure to update the issue that the version is out)
https://github.com/allegroai/trains/issues/222
In your trains.conf add the following line:
sdk.development.store_code_diff_from_remote = trueIt will store the diff from the remote HEAD instead of the local one.

PompousBeetle71 so basically exclude parameters that are considered "local" only, so that other people will not accidentally use them?

Could it be in a python at_exit event ?

Yes JitteryCoyote63 I think you are correct, this currently the easiest to do. PompousParrot44 notice that you should have a "services" queue with a trains-agent "services mode" running to enqueue those type pf mostly sleeping services 🙂
I was thinking we can quickly create a service that does that, maybe leverage one of these ?
https://github.com/mehrdadmhd/scheduler-py
https://github.com/dbader/schedule
WDYT?

from clearml.backend_api.session.client import APIClient client = APIClient() result = client.queues.get_next_task(queue='queue_ID_here')Seems to work for me (latest RC 1.1.5rc2)

Hi SmallDeer34
ClearML automagical logging will work on the current python process. But in your example yyour Bash is running another python script (that has nothing to do with the original notebook), hence clearml automagic is not aware of it (i.e. it cannot "patch" the tensorboard calls).
In order to make it work.
you should do something like:
from joeynmt import train train.main(...)Or something similar 🙂
Make sense ?

None

Like what would be the exact query given an endpoint, for requests per sec.

You mean in Grafana ?

you can also get it flattened with:
task.get_parameters()Type in both cases is string

Okay I think I found the confusion here (and it is confusing, but also very cool)
This line:
metrics_names = {"metrics": ["name", "bias", "r2"]} task.connect(metrics_names)When running in "manual mode" (i.e. not by an agent), will take the dict metrics_names and put it on the Tasks HyperParameters section.
But, when executed by the Agent, it will do the opposite! it will take the data stored on the Task's hyperparameters section and put it back into the metrics_names ` variable...

Hi MinuteCamel2

I can I disable it from automatically uploading model checkpoints to ClearML servers?

Maybe this one can help :)
https://www.youtube.com/watch?v=etGjxOKG9lo

deleted all of the models from my ClearML project but I still receive this message. Do you know why?

It might take it a few hours to update... 😞

Hi @<1581454875005292544:profile|SuccessfulOtter28>

Why would you archive an experiment?

Because you do not want to see it any longer (i.e. not very important) but you do not want to loose the ability to later do some forensics and look into it (meaning you do not want to completely delete it)
does that make sense ?

This doesn't seem to be running inside a container...
What's the clearml-agent launch command you are using ? (i.e. do you have --docker flag)

how specifically I map my clearml.conf to the containers?

You add:

    volumes:
      - $HOME/clearml.conf:/root/clearml.conf

to here:
None
and here:
None

Is it not possible to say just look at my requirements.txt file and the imports in the script?

I think there is a GitHub Issue for this feature
(basically the issue is, requirements.txt are very often not updated, and have no real version lock, so replicating a working env is always safer)

Hi, Is there a way to stop a clearml-agent from within an experiment?

It is possible but only in the paid tier (it needs backend support for that) 😞

My use case it: in a spot instance marked for termination after 2 mins by aws

Basically what you are saying is you want the instance to spin down after the job is completed, correct?

Hi GiganticTurtle0
Sure, OutputModel can be manually connected:
model = OutputModel(task=Task.current_task()) model.update_weights(weights_filename='localfile.pkl')

it's in the docker image, doesn't the git clone command run in the container

Then this should have worked.
Did you pass in the configuration: force_git_ssh_protocol: true
https://github.com/allegroai/clearml-agent/blob/e93384b99bdfd72a54cf2b68b3991b145b504b79/docs/clearml.conf#L25

Hi EnviousStarfish54
I remember this feature request, let me check where it stands..

Hi WittyOwl57
Are you starting a new server from scratch or is it running on previously stored data?

Hi PungentLouse55
it depends on the trains-server version you are running.
If the trains-server >= 0.16 then you have to add "Args/" prefix. If you are running an older version, then you should not add any prefix.