this task - https://github.com/allegroai/clearml/blob/5a9155b2039413280f13dfded1121470c4c4323d/examples/pipeline/step2_data_processing.py

What’s the point of saying General?

AgitatedDove14 - mean this - says name=None but text says default is General.

Sorry, don’t get it

In params:

parameter_override={'General/dataset_urlWhat’s the General for?

Ok the doc needs fix

Thanks you! Does this go as a root logging {} element in the main conf? outside SDK right?

I only see published getting preference, not a way to filter only to published

Initially thought use_current_task in Dataset.create does it, but that seems to set the DataprocessingTask itself to the current task

Yeah that would be good

or atleast a way to register these

Great. Is there a good view of the roadmap?

Thanks AlertBlackbird30

From code perspective what changes?

@<1523701205467926528:profile|AgitatedDove14> - any thoughts on this. Would like to use profile / iam roles as well.

Ok it’s not implemented right, that’s what I was asking

I guess this is a advantage with docker mode. Will try that out as well sometime.

Hey SuccessfulKoala55 Like I mentioned, I have a spacy ner model that I need to serve for inference.

I guess the question is - I want to use services queue for running services, and I want to do it on k8s

I don’t want to though. Will run it as part of a pipeline

Yeah got it

Thanks AgitatedDove14 - is the datasets feature pretty new?

AgitatedDove14 - thoughts on this? I remember that it was Draft before, but maybe because it was in a notebook vs now I am running a script?

I am doing something like this with a yaml based pipelines DSL

Got it thanks TimelyPenguin76 .

That's cool AgitatedDove14 , will try it out and pester you a bit more. 🙂

AgitatedDove14 - these instructions are out of date? https://allegro.ai/clearml/docs/docs/deploying_clearml/clearml_server_kubernetes_helm.html

Ok got it, thanks

The first line of it is getting a dataset and it’s failing with no project name

don’t know what’s happening there

` pipe = PipelineController(
default_execution_queue="minerva-default",
add_pipeline_tags=True,
target_project=pipelines_project,
)

    for step in self.config["steps"]:
        name = self._experiment_name(step)
        pipe.add_step(
            name=name,
            base_task_project=pipelines_project,
            base_task_name=name,
            parents=self._get_parents(step),
            task_overrides...