Creating a dataset sounds like a good idea, but that does not seem to be the issue.
Can you verify you can manually clone using the same link (notice the log should specify the exact clone it is using, with the password replaced with *)
JitteryCoyote63 if this is simulating an agent, the assumption is that the Task was already created, hence the task ID.
If i am working with Task.set_offline(True)
How would the two combine ? I mean off-line is be definition not executed by an agent, what am I missing ?
My only point is, if we have no force_git_ssh_port or force_git_ssh_user we should not touch the SSH link (i.e. less chance of us messing with the original URL if no one asked us to)
I wonder, does it launch all "step two" instances in parallel ?
In theory it should , but in practice since these are the same "template" I'm not sure what would happen.
One last note, you can call PipelineDecorator.debug_pipeline() to debug the pipeline locally, it will have the exact same behavior only it will run the steps as subprocesses.
I aborted the task because of a bug on my side
🙂
Following this one, is treating abort as failed a must feature for the pipeline (in your case) or is it sort of a bug in your opinion ?
@<1639799308809146368:profile|TritePigeon86> +1
LOL, Okay I'm not sure we can do something that one.
You should probably increase the storage on your instance 🙂
Hi @<1585078763312386048:profile|ArrogantButterfly10>
Now i want to clone the pipeline and change the hyperparameters of train task, is it possible? If so, how??
the pipeline arguments are for the pipeline DAG/logic, you need to pass one of the arguments as an argument for the training step/task. Make sense ?
I want to optimizer hyperparameters with trains.automation but: ...
Yes you are correct, in case of the example code, it should be "General/..." if you have ArgParser, it should be "Args/..." Yes it looks like the metric is wrong, it should be "epoch_accuracy" & "epoch_accuracy"
directly from the UI from the services queue?
Spin the agent with --service-mode it will keep pulling jobs from the queue and spinning them (BTW, it will only start the next job after the first one finished the env setup, and you must be running with --docker mode 🙂
GreasyPenguin14
In the process MyProcess other processes are created via a ProcessPoolExecutor.
Hmm that is interesting, the sub-process has an additional ProcessPoolExecutor inside it ?
GrittyKangaroo27 if you can help with reproducible code that will be great (or any insight on reproducing the issue)
and you have clearml v0.17.2 installed on the "system" packages level, and 0.17.5rc6 installed inside the pyenv venv ?
The configuration tab -> configuration objects -> pipeline is empty
That's the reason it is doing nothing 😞
How come it is empty if you Cloned the local one?
Ohh yes, if you deleted the token then you have to recreate the cleaml.conf
BTW: no need to generate a token, it will last 🙂
generally speaking the agent will convert the repo url to the auth scheme it is configured with, ssh->hhtp if using user/pass, and http->ssh if using ssh
Oh this is so internally, the background thread can signal it is not deferred, are you saying there is bug or the code is odd?
Hi PanickyMoth78 , an RC is out with a fix.
pip install clearml==1.6.3rc0
Thank you for noticing the graph issue.
Btw do notice that since data is being changed inside the controller loop the parents are still kind of odd, because it is not clear to the logic the source of the data so it assumes it depends on the current state (i.e. all the leaves)
Is there still an issue? Could it be the browser cannot access the file server directly?
This looks like 'feast' error, could it be a configuration missing?
Sorry my bad:config_obj['sdk']['stuff']['here'] = value
No worries, just found it. Thanks!
I'll make sure to followup on the GitHub issue for better visibility 🙂
I mean if I enter my host machine ssh password it works. But we will disable password auth in future, so it’s not an option
To clarify, it should not allow users to ssh into the host machine (if you can do that this means you own it), it only allows users to SSH into the container the host machine spins, make sense ?
Hi GrittyHawk31
but it could not connect to the grafana dashboard through port 3000, is there any particular reason for that? I may have missed something.
Did you run the full docker-compose.yml ?
Are you able to curl to the endpoints ?
@<1546303293918023680:profile|MiniatureRobin9>
, not the pipeline itself. And that's the last part I'm looking for.
Good point, any chance you want to PR this code snippet ?
def add_tags(self, tags):
# type: (Union[Sequence[str], str]) -> None
"""
Add Tags to this pipeline. Old tags are not deleted.
When executing a Pipeline remotely (i.e. launching the pipeline from the UI/enqueuing it), this method has no effect.
:param tags: A li...Is there a quicker way to abort all running experiments in a project? I have over a thousand running anonymous data tasks in a specific project and I want to abort them before debugging them.
We are adding "select" all in the next UI version to do that as quickly as possible 🙂
This is exactly what I did here, and it is working 😞
https://demoapp.demo.clear.ml/projects/0e919ea1cc5c499b99e1ab85004b6e97/experiments/887edef09d4549e88b829a34c87d4d5b/output/execution