Thank you, I will look into it :)

Just if I leave it empty, it will not work.

Is it possible to set extra-index-url on a per-task basis? Just asking because of the way you wrote it with the two dashes 🙂

It is explained here: https://github.com/allegroai/clearml/issues/91#issuecomment-578544866

The one I posted on top 22.03-py3 😄

Yes, that works fine. Just the http vs https was the problem. The UI will automatically change s3://<minio-address>:<port> to http://<minio-address>:<port> in http://myclearmlserver.org/settings/webapp-configuration . However what is needed for me is https://<minio-address>:<port>

To answer my own question: In the WebUI where one inputs the credentials, use https for the host instead of the auto-added http

I think I still don't get how clearml is supposed to work/be used. Why wouldn't the following work currently?
Example:
` task = Task.init(...)
if not running_remotely:
task_dict = task.export_task()
requirements = task_dict["script"]["requirements"]["pip"].splitlines()
requirement_torch = [r for r in requirements if r.startswith("torch==")]
requirements.remove(requirement_torch[0])
requirements.append("torch >= 1.8.1")
task_dict["script"]["requirements"]["pip"] = "\n"....

481.2130692792125 seconds
Done

I just wanna avoid that ClearML leaves files lingering around. Btw: a better default behavior in my opinion would be to delete tasks only after files have been deleted. And only with the force option to delete the task anyways!

I guess this is the current way to do it: https://github.com/tensorflow/tensorboard/issues/39#issuecomment-568917607 so I would say: Yes, it supports gif.

Maybe if you have time you can take a look at the log I posted in the beginning. I think I have the same extra_index_url and the nightly flag activated 😕

Sure, let me try.

It is only a single agent that is sending a single artifact. server-->agent is fast, but agent-->server is slow.

What exactly do you mean by that?

Unfortunately, not. Quick question: Is there caching happening somewhere besides .clearml ? Does the boto3 driver create cache?

Or there should be an early error for trying to run conda based tasks on pip agents

Afaik, clearml-agent will use existing installed packages if they fit the requirements.txt. E.g. pytorch >= 1.7 will only install PyTorch if the environment does not already provide some version of PyTorch greater or equal to 1.7.

Thanks! I am fascinated by what you guys offer with clearml 🙂

Not for the logger, I just tested it.

I installed as told on pytorch.org : pip3 install --pre torch torchvision torchaudio --index-url None

Latter

Thank you, gonna try that!

But you can manually add them with Task.add_requirements, no?

In my opinion an ugly solution. I would have to keep track of which requirements are missing. Then I would rather just add all requirements manually.

I will create a minimal example.

Can you maybe also tell me which docker image you used? For me this is all not working unfortunately

Thank you. Seems like someone implemented a type check Error: Dataset id=8d7355655830427f9243671c8cf0a6b0 is not of type Dataset :)

I am also wondering how I integrate my (preexisting) main task in the pipeline. I start my main task like this: python my_script.py --myarg "myargs" . How are the arguments captured? I am very confused, how one integrates this correctly...

Or I am using this method wrong.

What I get for args when I print it locally is not the same as what ClearML extracts from args .