@<1523701070390366208:profile|CostlyOstrich36> I am facing the same issue:

{"meta":{"id":"90841d05dfb1431a8d9dfc6bfdb39f9e","trx":"90841d05dfb1431a8d9dfc6bfdb39f9e","endpoint":{"name":"events.debug_images","requested_version":"2.23","actual_version":"2.7"},"result_code":200,"result_subcode":0,"result_msg":"OK","error_stack":"","error_data":{}},"data":{"metrics":[]}}

The thing is, even on the community server, not all the datasets have automatic previews. So for the same code/dataset, some of the runs have previews and some of them don't.

So I added the snippet above to the code,
and now the preview for the first 10 rows shows up. However, the automatic preview is still not working.

As you can see, it eventually manages to reach the apiserver however, it still says that access key was not provided and that the service will not be started. I get the same behaviour whether I set the flag to 0 or 1.

Which would make sense because of the name SHUTDOWN_IF_NO_ACCESS_KEY . The thing is, when I tried setting it to 0, it still shutdown.

Let me rerun it, so that I can capture it. I am currently running it on AWS Fargate, so I have the logs for that.

Thank you for your reply SuccessfulKoala55 . 😄

It is currently set to 1, so I am assuming setting it to 0 would mute the errors from logging?

The current behaviour is, if I keep it set to 1 the services agent would automatically shutdown if the access key is not configured. Assuming I set it to 0, then the agent services should not shutdown anymore, right?

so basically the flag also prevents it from shutting down

I was able to resolve the issue. I am currently using clearml on wsl2 and my machine is connected to a vpn that allows me to connect on to the clearml instance hosted on AWS. You were right it was a network issue, I was able to resolve it by modifying my /etc/resolv.conf file.

That's what I was thinking. But I am still having issues on the self hosted version. I think it may be an unrelated issue though. I will do some debugging and report back.

On a separate note, does clearml have a set of acceptance tests that you usually go through before a release?

I am having the same problem on both the self hosted and free community clearml.

The community server is working again.

Thanks for the reply anyways 😄

Wow!

Hi again @<1523701435869433856:profile|SmugDolphin23> ,

I was able to run the pipeline remotely on an agent, but I am still facing the same problem with the code breaking on the exact same step that requires the docker container. Is there a way to debug what is happening? Currently there is no indication from the logs that it is running the code in the docker container. Here are the docker related logs:

agent.docker_pip_cache = /home/amerii/.clearml/pip-cache
agent.docker_apt_cache =...

Hi AgitatedDove14 ,

I am planning to use terraform to retrieve the secrets from AWS, after I retrieve the user list from the secrets manager, I am going to pass them as Environment variables.

The reason I am passing them as environment variables is that, I couldn't find a way to automatically upload files to AWS EFS from Terraform. Since the config file needs to be mounted as an EFS volume to the ECS task definition.

I was able to make the web authentication work while passing the followi...

I'm actually trying that as we speak 😛

SuccessfulKoala55 That seemed to do the trick, thanks for your help! 😄

Just waiting for the changes to be completed

I added the following to the clearml.conf file

agent {
    package_manager: {
        # supported options: pip, conda, poetry
        type: pip,
        extra_index_url: ["my_url"],
    },
}

For some reason the changes were not reflected, here are the logs from the agent:

agent.package_manager.type = pip
agent.package_manager.pip_version.0 = <20.2 ; python_version < '3.10'
agent.package_manager.pip_version.1 = <22.3 ; python_version >\= '3.10'
agent.package_manager.sys...

I set my local laptop as an agent for testing purposes. I run the code on my laptop, it gets sent to the server which sends it back to my laptop. So the conf file is technically on the worker right?

Thanks @<1523701205467926528:profile|AgitatedDove14> restarting the agents did the trick!

The extra_index_url is not even showing..

Ohh, thanks! Will give it a shot now!

@<1523701205467926528:profile|AgitatedDove14> So I was able to get it to pull the package by defining packages= None

The second problem that I am running into now, is that one of the dependencies in the package is actually hosted in a private repo.

I tried getting around it by defining the environment PIP_INDEX_URL and passing it using log_os_environments in the clearml.conf and I am now getting this message:

md-ap-feature-engineering/.venv/lib/p...

How do you handle private repos in clearml for packages?

Thanks @<1523701205467926528:profile|AgitatedDove14>

I am using the latest version clearml server and I am using version 1.9.1 for the sdk.

Here is the code that I am currently using:

if __name__ == "__main__":
    # create clearml data processing task
    dataset = Dataset.create(
        dataset_name="palmer_penguins",
        dataset_project="palmer penguins",
        dataset_tags=["raw"]
    )
    dataset_path = "data/raw/penguins.csv"
    # add the downloaded files to the current dataset
    dataset.add_files(path=dataset_pa...

I have been using them for a while, they've always had autopreviews.