Oh, I was assuming you are passing the entire DB backups to the cloud.

Yes, that is what I want to do.
So I need to migrate both the MongoDB database and elastic search database from my local docker instance to the equivalent in the cloud ?

but when I spin up a new server in the cloud, that server will have it's own mongodb and that will be empty no ?

following this thread as it happen every now and then that clearml miss some package for some reason ...

you should be able to test your credential first using something like rclone or azure-cli

just saw that repo: who are coder ? That not the vscode developer team is it ?

then dont use clearml to look at images

I don't think ClearML is design to vizualize millions of image per task. At least not the Debug samples section. That was design so that you can see for a given set of image, how does the model perform epoch after epoch.

For vizu millions of image, you have tool like Fiftyone.

To "attach" that zip to the model, do you just use the update_weight and point to that zip file?

so what was the solution/hack then ?

there is a tricky thing: clearml-agent should not be running from a venv itself ... don't remember where I read that doc

I don't see where you instanciate ClaerML Task in your given code. Which means that Task.current_task() will return None , thus the error you get.

We don't have a file server. The clearml conf have :
sdk.development.default_output_uri=" None "

We use task.export_task() and a hacked version to get console log:

def save_console_log(task: clearml.Task, fs, remote_path, number_of_reports=10000):
    from clearml.backend_api.services import events
    from clearml.backend_api import Session

    # Stollen from Task.get_reported_console_output()
    if Session.check_min_api_version('2.9'):
        request = events.GetTaskLogRequest(
            task=task.id,
            order='asc',
            navigate_earlier=True,
         ...

my code looks like this :

    parser = argparse.ArgumentParser()
    parser.add_argument('-c', '--config-file', type=str, default='train_config.yaml',
                        help='train config file')
    parser.add_argument('-t', '--train-times', type=int, default=1,
                        help='train the same model several times')
    parser.add_argument('--dataset_dir', help='path to folder containing the preped dataset.', required=True)
    parser.add_argument('--backup', action='s...

When i set output uri in the client, artefact are sent to blob storage
When file_server is set to azure:// then model/checkpoint are sent to blob storage
But the are still plot and metrics folder that are stored in the server local disk. Is it correct?

--gpus 0,1 : I believe this basically say that your code launched by the agent has access to both GPUs and that is it. Now it is up to your code to choose which GPU to use and what not and how ...

something like this: None ?

do you have a video showing the use case for clearml-session ? I struggle a bit about how is it used for ?

nevermind, all the database files are in data folder

• Modify the "uncommited changes"
• Enqueue the taskIs it possible to edit "uncommited changes" from a task via Python ??

you are forcing ssh with force_git_ssh_protocol: true
Have you setup ssh keys ?
If you are using ssh keys, why enable_git_ask_pass: true ?

in my case using self-hosted and agent inside a docker container:
47:45 : taks foo pulled
[ git clone, pip install, check that all requirements satisfied, and nothing is downloaded]
48:16 : start training

one specify the venv python, the other tell it to not do anything

(I don't really use HPO feature so I out sorry)

i need to do a git clone

You need to do it to test if it works. Clearml-agent will run it itself when it take in a task

@<1523701087100473344:profile|SuccessfulKoala55> Thanks. Manage to get it working now with

export REQUESTS_CA_BUNDLE=/etc/ssl/certs/zscaler.crt

(Ubuntu system)

like for dataset_dir I would expect a single path, not an array of 2 paths duplicated

I am more curious about how to migrate all the information stored in the local clearml server to the clearml server in the cloud

So I now just need to find a way to not populate Installed Package in the first place

may be I will play around a bit and ask more specific questions .... It's just I cannot find much docs around how the pipeline caching work (which is the main point of pipeline ?)

I will try it. But it's a bit random when this happen so ... We will see