So what's the point of the alias? It's not very clear.. Even after specifying an alias I am still getting the following message: Dataset.get() did not specify alias. Dataset information will not be automatically logged in ClearML Server

Thanks for the reply. I was trying out this feature on a dummy example. I used the following command
dataset = Dataset.get( dataset_project="palmer penguins", dataset_name="raw palmer penguins", alias="my_test_alias_name", overridable=True)That was the only time I called the get() command. I still got the message that I should specify the alias. I can try and do a bit of debugging to see why it gets called multiple times.

Just waiting for the changes to be completed

Wow!

I set my local laptop as an agent for testing purposes. I run the code on my laptop, it gets sent to the server which sends it back to my laptop. So the conf file is technically on the worker right?

I would like to see it used in a clear example as it was intended to be used before giving my opinion on it, if that makes sense

Ohh, thanks! Will give it a shot now!

Thank you so much for your reply, will give that a shot!

22.04

How do you handle private repos in clearml for packages?

My server is hosted on AWS Fargate

Nevermind, I figured out the problem. I needed to specify the --docker flag when running the clearml-agent

I am currently running the scripts on WSL ubuntu

The above output is on the clearml community server

I am using the latest version clearml server and I am using version 1.9.1 for the sdk.

Here is the code that I am currently using:

if __name__ == "__main__":
    # create clearml data processing task
    dataset = Dataset.create(
        dataset_name="palmer_penguins",
        dataset_project="palmer penguins",
        dataset_tags=["raw"]
    )
    dataset_path = "data/raw/penguins.csv"
    # add the downloaded files to the current dataset
    dataset.add_files(path=dataset_pa...

SuccessfulKoala55 That seemed to do the trick, thanks for your help! 😄

The thing is, even on the community server, not all the datasets have automatic previews. So for the same code/dataset, some of the runs have previews and some of them don't.

Right so I figured out why it was calling it multiple times. Everytime a dataset is serialiazed, it calls the _serialize() function inside of clearml/datasets/dataset.py file, the _serialize() method calls self.get(parent_dataset_id) which is the same get() method. This means that the user will always be prompted with the log, even if they are not "getting" a dataset. So anytime a user creates, uploads, finalizes a dataset, they will be prompted with the message...

I have been trying to contribute as well...

I have created some PRs, in an attempt to improve the current situation. I'm just surprised that currently there is no CI process, and that it's been 2 months since the last release.

Again, I'm more than happy to help and contribute to the overall CI process.

so basically the flag also prevents it from shutting down

Thanks @<1523701205467926528:profile|AgitatedDove14> restarting the agents did the trick!

Yes, I am using a virtualenv that has pandas and clearml installed.

Hi AgitatedDove14 ,

I am planning to use terraform to retrieve the secrets from AWS, after I retrieve the user list from the secrets manager, I am going to pass them as Environment variables.

The reason I am passing them as environment variables is that, I couldn't find a way to automatically upload files to AWS EFS from Terraform. Since the config file needs to be mounted as an EFS volume to the ECS task definition.

I was able to make the web authentication work while passing the followi...

The community server is working again.

Thanks @<1523701205467926528:profile|AgitatedDove14>

I was able to resolve the issue. I am currently using clearml on wsl2 and my machine is connected to a vpn that allows me to connect on to the clearml instance hosted on AWS. You were right it was a network issue, I was able to resolve it by modifying my /etc/resolv.conf file.

That's what I was thinking. But I am still having issues on the self hosted version. I think it may be an unrelated issue though. I will do some debugging and report back.

On a separate note, does clearml have a set of acceptance tests that you usually go through before a release?

Let me rerun it, so that I can capture it. I am currently running it on AWS Fargate, so I have the logs for that.

Not exactly, the dataset gets called in the script using Dataset.get() and the second dataset is an output dataset using Dataset.create().. Which means that dataset_1 is a parent dataset of dataset_2.