clearml-init
ClearML SDK setup process

Please create new clearml credentials through the settings page in your clearml-server web app (e.g. None )
Or create a free account at None

In settings page, press "Create new credentials", then press "Copy to clipboard".

Paste copied configuration here:
api {
web_server: http://:8080
api_server: http://:8008
file...

As I’ve used it, rsync is akin to scp, but provides more control.

There are two things I want to accomplish with this:

• I want a reliable location to store my tasks without needing to worry about running out of space.
• I want to be able to access these stored tasks in the clearml ui when they are in this storage
  While rsync would let me backup the file server contents, like cp’ing them into a rclone of onedrive. I don’t think it would let me view tasks that I have put on onedrive.

Thank you!

I wasn’t getting my hopes up on storing tasks elsewhere :)

For the models & artifacts is there a parameter to change the default directory of saving and loading to something else?

Firewall rules:

Nevermind, 😆 as you thought, seems I was just needing a few more f5s on my dashboard. Thank you very much for this

No error, just failure to upload it seems

Yes that is where they are

Found the problem. Some port rules between my server and client was blocking it. Some autossh forwarding solved my problem

If a external storage is not specified it will use internal correct?

So if I give up on having specific folder paths for each project within onedrive I could just treat it as a alternative path for internal artifact storage.

And I’m still having my other issues I mentioned. I’m just ignoring the unreachable machines for the moment.

This answered my question, I ended up setting up minio running off of a external hard drive which backups to onedrive.

Though I am still having some problems getting the client to connect to minio (see recent thread)

I’m going to give QEMU a try and see if it performs well enough for me

I’ll put in the actual copy paste later tonight thank you for the help

From running it with the credentials I got with the non-self-hosted clearml instance

clearml-init
ClearML SDK setup process

Please create new clearml credentials through the settings page in your clearml-server web app (e.g. None )
Or create a free account at None

In settings page, press "Create new credentials", then press "Copy to clipboard".

Paste copied configuratio...

The exact error I am getting is:

 line 1095, in output_uri
    raise ValueError("Could not get access credentials for '{}' "

This error is thrown by a failed .get() function call on the StorageHandler object I looked at the ._ _ dict _ _.keys() parameter list of the StorageHandler, and I don't see anyway to access the dictionary directly.

None of the default ports are changed and in my firewalld I have ports 8080, 8081, and 8008 open to tcp

This is post script completion on the client end

with the filepath leading to /clearml/task.py

And from the self-hosted ^

@<1523701070390366208:profile|CostlyOstrich36> , anything I missed?

And to show it off

And again thank you for the help with this.

>>> print(json.dumps(config_obj.get("sdk"), indent=2))
{
  "storage": {
    "cache": {
      "default_base_dir": "~/.clearml/cache"
    },
    "direct_access": [
      {
        "url": "file://*"
      }
    ]
  },
  "metrics": {
    "file_history_size": 100,
    "matplotlib_untitled_history_size": 100,
    "images": {
      "format": "JPEG",
      "quality": 87,
      "subsampling": 0
    },
    "tensorboard_single_series_per_graph": false
  },
  "network": {
    "file_upload_retries":...

Yes I will give it a try and get back to you.

and from minio

I am able to capture clearml experiments on the clearml server running on the same machine as the minio.

It gave no import error, and I'm still having problems. I returned to my original script and it shows some file transfer print statements, but I don't see the files appearing in minio

I discovered part of the problem. I did not have boto3 installed on this conda env.

With only those docker containers running I’m having this issue. In a few hours I’m going to test on a additional machine to confirm