BeefyCow3 see this https://allegroai-trains.slack.com/archives/CTK20V944/p1593077204051100 :)

MelancholyElk85 notice there is the pipeline controller queue (i.e. which agent will run the logic of the pipeline), and the default queue for the pipeline steps (i.e. the actual steps of the pipeline).
The default queue for the pipeline logic itself is services . you can change it ( pipeline.start(..., queue='another_q') )
Make sense ?

Yeah.. that should have worked ...
What's the exact error you are getting ?

1633204289496 clearml-services DEBUG docker: invalid reference format.

This is the strange message, like the execution command is not valid...

LazyFox65 seems like a docker issue.
Can you manually run the docker ?

the question remains though: why docker containers won't launch on 

services

Maybe something with the way it launched on the docker-compose?
(I'm assuming it will fail on any docker container regardless, right?!)

what's the docker version?

Hi LazyFox65
So the idea is that you add two lines of code to your codebase :
from clearml import Task task = Task.init(project_name='examples', task_name='change me')And you run it once, then it will create the experiment, environment arguments etc.
Now that you have it in the UI you can clone / change all the fields and send for execution.
That said you can also create an experiment from CLI (basically pointing to a repo and entry point)
You can read here:
https://github.com/allegroa...

Hmm I'm assuming something wrong here:
https://github.com/allegroai/clearml-server/blob/a64c4d264d00eadd2d11818b37151d3cc6266d99/docker/docker-compose.yml#L119
What's the host machine OS ?

Hi MelancholyElk85

However, when I clone the pipeline from web UI and launch it once again, it works. Is there a way to bypass this?

In both cases, are you seeing a different behavior on the same machine running the agent (i.e. clonening from the UI vs code) ?

Maybe ?
https://stackoverflow.com/questions/45682010/docker-invalid-reference-format/51379726

Hmm can you run:
docker run -it allegroai/clearml-agent-services:latest

So “wait” is a better metaphore for me

So I would do something like (I might have a few typos but that's the gist):

def post_execute_callback_example(a_pipeline, a_node):
    # type (PipelineController, PipelineController.Node) -> None
    print('Completed Task id={}'.format(a_node.executed))
    # wait until model is tagged, then pass it as argument
    while True:
        found = Moodel.query_models(...) # model filter here, inlucing tag and project
        if found:
         ...

is there a way to visualize the pipeline such that this step is “stuck” in executing?

Yes there is, the pipelline plot (see plots section on the Pipeline Task, will show the current state of the pipeline.
But I have a feeling you have something else in mind?
Maybe add Tag on the pipeline Task itself (then remove it when it continues) ?
I'm assuming you need something that is quite prominent in the UI, so someone knows ?
(BTW I would think of integrating it with the slack monitor, to p...

Hi @<1523704198338711552:profile|RoughTiger69>
From this scenario can we assume the "selection" will be tagging the model manually?
Also, how would an human operator decide on the best model, that is what is the input to base the decision on?

Awesome! let me know how/if it works 🙂

That was the idea behind the feature (and BTW any feedback on usability and debugging will be appreciated here, pipelines are notorious to debug 🙂 )

the ability to exexute without an agent i was just talking about thia functionality the other day in the community channel

What would be the use case ? (actually the infrastructure now supports it)

Hi RoughTiger69

How about using the pipeline decorator as a way to run this logic?
https://github.com/allegroai/clearml/blob/master/examples/pipeline/pipeline_from_decorator.py

I think I'm missing the context of where the code is executed....

btw: you can now set the configuration_objects directly when calling add_step 🙂
https://clearml.slack.com/archives/CTK20V944/p1633355990256600?thread_ts=1633344527.224300&cid=CTK20V944

https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/elb-create-https-ssl-load-balancer.html
🙂

An example for something like spacy would be useful for the community.

That awesome, any chance you can PR something? (no need for it to be perfect, we can take it from there)

Interesting, do you think you could PR a "fixed" version ?
https://github.com/allegroai/clearml-web/blob/2b6aa6043c3f36e3349c6fe7235b77a3fddd[…]app/webapp-common/shared/single-graph/single-graph.component.ts

Hi @<1536518770577641472:profile|HighElk97>

Is there a way to change the smoothing algorithm?

Just like with TB, this is front-end, not really something you can control ...
That said you can report a smoothed value (i.e. via python) as additional series, wdyt ?

As I'm a Full-stack developer at Core. I'd be looking to extend the TRAINS Frontend and Backend APIs to suit my need of On-Prem data storage integration and lots of other customization for Job Scheduler(CRON)/Dataset Augmentation/Custom Annot. tool etc.

That is awesome! Feel free to post a specific question here, and I'll try to direct to the right place 🙂

Can you guide me to one such tutorial that's teaching how to customize the backend/front end with an example?

You mean l...

check here None

Hi DeliciousBluewhale87
So basically no webhooks, the idea is that you have full API to query everything in the system and launch task based on any logic. You can check the slack monitoring example, it is basically doing that. Wdyt?

DepressedChimpanzee34
I might have an idea , based on the log you are getting LazyCompletionHelp in stead of str
Could it be you installed hyrda bash completion ?
https://github.com/facebookresearch/hydra/blob/3f74e8fced2ae62f2098b701e7fdabc1eed3cbb6/hydra/_internal/utils.py#L483

Hi DepressedChimpanzee34
Why do you need to have the configuration added manually ? isn't the cleaml.conf easier ? If not I think OS environments are easier no? I run run above code, everything worked with no exception/warning... What is the try/except solves exactly ?

Hmm let me rerun (offline mode right ?)

well at this point I'm not sure it is still essential, we have 3 run-modes offline, local-server, cloud-sever and this option made it work for all of them.. can be that it is not required anymore and its just legacy..

LOL, sure if you have so many setups, that makes sense 🙂

this is strange.. you ran it with the dataclass config I added?

Yes but I had to remove the:
from config_files import cfgand instead used:
` @hydra.main(config_path="config_files", config_name="confi...

Any chance you can zip the entire folder? I can't figure out what's missing, specifically "from config_files" , i.e. I have no packages nor file named config_files