That's up and running and is perfectly fine.

Odd; switching to virtual environment results in
fatal: could not read Username for ' ': terminal prompts disabledeven though it does earlier show that:
agent.git_user = xxx

I think ClearML boots up only afterwards, so those environment variables may not be available yet.
You should set them manually in the bootstrap code unfortuantely.

I see! The Hyper Datasets don't really fit our use case - it seems really focused on CNNs and image-based data, but lacking support for database-oriented tabular data.

So for now we mainly work with parquet and CSV files, and I was hoping there'd be an easy way to view those... I'll make a workaround with a "Datasets" project I suppose!

SuccessfulKoala55 CostlyOstrich36 actually it is the import statement, just finally got around to the traceback:

` File "/home/.../ccmlp/configs/mlops.py", line 4, in <module>
from clearml import Task
File "/home/.../.venv/lib/python3.8/site-packages/clearml/init.py", line 4, in <module>
from .task import Task
File "/home/.../.venv/lib/python3.8/site-packages/clearml/task.py", line 31, in <module>
from .backend_interface.metrics import Metrics
File "/home/......

So the config loading is not deferred until execution 😞

Generally, really. I've struggled recently (and in the past), because the documentation seems:
Very complete wrt available SDK (though the formatting is sometimes off) Very lacking wrt to how things interact with one anotherA lot of what I need I actually find from pluging into the source code.
I think ClearML would benefit itself a lot if it adopted a documentation structure similar to numpy ecosystem (numpy, pandas, scipy, scikit-image, scikit-bio, scikit-learn, etc)

We're using 1.1.5 at the moment -- I'll make sure everyone updates to 1.1.6 on Monday.

That solution does not work for us unfortunately -- the .env is an argument from argparse, and because we cannot attach non-git files to a remote task (again issue #395), we have to first download CLI arguments for remote execution and ensure they exist on the remote agent.

So the pipeline runs successfully, I can find all the different tasks, but I cannot see them in the Pipelines tab…

When I use the APIClient to fetch the tags for the project, I get an empty collection of system tags:

<projects.GetProjectTagsResponse: {
    "tags": [],
    "system_tags": []
}>

Ah I see, if the pipeline controller begins in a Task it does not add the tags to it…

AFAIU, something like this happens (oversimplified):

` from clearml import Task # <--- Crash already happens here
import argparse
import dotenv

if name == "main":
# set up argparse with optional flag for a dotenv file
dotenv.load_dotenv(args.env_file)
# more stuff `

Let me know if you do; would be nice to have control over that 😁

I didn't mention code in #340 nor did I mention data here 😄 The idea was to package non git-specific files for remote execution

Nope, no other config files

Is it CLEARML_CONFIG_FILE ? (I had to dig this from the GH code 😅 )

That will come at a later stage

Any updates @<1523701087100473344:profile|SuccessfulKoala55> ? 🫣

AgitatedDove14

hmmm... they are important, but only when starting the process. any specific suggestion ?
(and they are deleted after the Task is done, so they are temp)

Ah, then no, sounds temporary. If they're only relevant when starting the process though, I would suggest deleting them immediately when they're no longer needed, and not wait for the end of the task (if possible, of course)

ClearML 1.1.4, Matplotlib 3.3.0 (it's not the latest as we have some backward compatibility issues)

clearml.backend_api.session.defs.ENV_HOST.get() did not work unfortunately 🤔

Let me try that real quick

But it does work on linux 🤔 I'm using it right now and the environment variables are not defined in the terminal, only in the .env 🤔

As in here SuccessfulKoala55 - https://github.com/allegroai/clearml/blob/master/clearml/automation/cloud_driver.py#L169

SuccessfulKoala55 The changelog wrongly cites https://github.com/allegroai/clearml/issues/400 btw. It is not implemented and is not related to being able to save CSVs 😅

Woot! What about clearml-agent 1.2.0? 🙂

I've also followed https://clearml.slack.com/archives/CTK20V944/p1628333126247800 but it did not help

Ah it already exists https://github.com/allegroai/clearml-server/issues/134 , so I commented on it

https://github.com/allegroai/clearml-agent/pull/98 AgitatedDove14 😁

Hm, just a small update - I just verified and it does indeed work on linux:

` import clearml
import dotenv

if name == "main":
dotenv.load_dotenv()
config = clearml.backend_api.Config.load() # Success, parsed with environment variables `