Another question: can I mix steps with Task and Function?

Sorry, my test environment crashed. I have to fix it first to reproduce the problem. 😓

Sorry, I can not understand what does nut from mean.

I am sure that I delete this user in auth.user and backend.user before call auth.create_user

Thanks for your reply 😀 ！

ok, I check it now

README introduces that no real scheduling when using Kubernetes. But I think no matter launch a Pod or Job that the Pod will be scheduled through k8s scheduler exception specifying the nodeName in Pod specification. This confuses me a lot. What is the meaning of no real scheduling I don’t understand no verification of docker image

which collection?

Do you use ClearML Open Source?

Got it!

Hi, SuccessfulKoala55 Maybe this is not a good question. In other words, docs introduce that ClearML Open Source supports orchestration, how can I found the relating codes? And what the role of clearml-agent in orchestration, a combination of kube-scheduler and kubelet?

Thanks TimelyPenguin76 , I think I got it, but I have new questions when reading README of ClearML Agent. And I gave my questions in community just now, could you have a look at them? https://clearml.slack.com/archives/CTK20V944/p1626251661217200

Hi TimelyPenguin76 , thanks for you reply. I still confuse about ClearML’s orchestration. Could you describe it in detail?

For example, I have a lot of tasks in a queue, but there is a default agent(services, mode = daemon) after using helm to deploy the ClearML in Kubernetes. As far as I know, agent will pull and execute the task in specified queue one by one, which results in other tasks are blocked. However, the physical resource is enough in cluster. So how can ClearML to improve the efficiency of execution? I can solve it by implementing a custom scheduler which is used to watch the queue, pull the task and...

Hi, TimelyPenguin76 , could you help me?

Since k8s has no real priority/order the scheduling order is not guaranteed form this point.

K8s can schedule pod with different priorities. So maybe no real scheduling means there is no ClearML scheduling after applying pod to k8s.

only if it is sure there are enough resources to actually spin the job now

In 1.0.2 version, I do not find any logic about checking k8s resource in k8s.py. Does it will implement in the future?

According to pipeline_from_functions.py , it is easy to understand that step1 returns data_frame and I can use it as input of step2. But I have no idea what string reference could be used when steps come from Task?

code

pipe.add_function_step(
    name='step_one',
    function=step_one,
    function_kwargs=dict(pickle_data_url='${pipeline.url}'),
    function_return=['data_frame'],
    cache_executed_step=True,
)
pipe.add_function_step(
    name='step_two',
    # parents=['st...

I have a similar question. I want to extend the authorization of clearml, but I have no idea about it according to docs. By the way, is it possible to implement the custom authorization based on the basic authz api in clearml? Or does clearml support it?

I want to know the total fields(string reference mentioned above) which can be used as input of step2. Are these fields of ClearML Task?

Thanks

hi , @<1687645223846416384:profile|GracefulToad94> .
About Upgrade.

This version is deployed in the productive environment, so it is not easy to upgrade it.

About special user.

an ordinary user with the following information:
url:

http://{webServerHost}/api/v2.13/auth/create_user

createRequest:

type AuthCreateUserReq struct {
   Avatar     string 
   // brain, this value is also configured for apiserver's env CLEARML__APISERVER__DEFAULT_COMPANY(brain)
  ...

Are you sending these requests manually?
COK. In any case, you should be looking at the

I send them with curl.

no, I just suspend this issue.

I check it with the following result:

action:

auth.create_user

mongo:

see picture

apiserver logs:
`[2022-05-17 10:02:50,024] [8] [WARNING] [clearml.service_repo] Returned 400 for auth.create_user in 8ms, msg=Value combination already exists: email=

`

But you’re not using the webserver, you’re sending the cURL requests manually, aren’t you?

yes, I send them manully.