From the log you shared, the task is picked up by the 

worker_d1bd92a3b039400cbafc60a7a5b1e52b_4e831c4cbaf64e02925b918e9a3a1cf6_<hostname>:gpu0,1

 worker

I can try and target the default one if it helps..?

It's pulled from the remote repository, my best guess is that the uncommitted changes apply only after the environment is set up?

Follow up on this btw, from the WebUI/Server POV, I see there's an "Admin" role, etc. Do those have additional views available, such as users etc?

I'll try it out, but I would not like to rewrite that code myself maintain it, that's my point 😅

Or are you suggesting I Task.import_offline_session ?

But we do use S3

Now, the original pyhocon does support include statements as you mentioned - https://github.com/chimpler/pyhocon

The last kind when launching the agents.
The former on task execution.

Sure SuccessfulKoala55 , and thanks for looking into it.

As an alternative (for now, or in general), we could consider reverting back to pip. The issue we encounter is that we have a monorepo, so frozen requirements should specify relative paths, but pip freeze does not seem to do that, so ClearML also fails in pip mode

Thanks @<1537605940121964544:profile|EnthusiasticShrimp49> ! That’s definitely the route I was hoping to go, but the create_function_task is still a bit of a mystery, as I’d like to use an entire class with relevant logic and proper serialization for inputs, and potentially I’ll need to add more “helper functions” (as in the case of DataTransformationStep , for example). Any thoughts on that? 🤔

Just a side note - the 1.1.1 notice keeps popping up even though the server is at 1.1.1 (and I've cleared browser cache etc)

Yes; I tried running it both outside venv and inside a venv. No idea why it uses 2.7?

I also tried switching to dockerized mode now, getting the same issue 🤔

Hmmm maybe 🤔 I thought that was expected behavior from poetry side actually

Don't even need to specify json=... 😉 Thanks!

It could be related to ClearML agent or server then. We temporarily upload a given .env file to internal S3 bucket (cache), then switch to remote execution. When the remote execution starts, it first looks for this .env file, downloads it using StorageManager, uses dotenv, and then continues the execution normally

Thanks! I'll wait for the release note/docs update 😁

But since this has come up a lot recently, any updates on #340? 😍

Opened a matching feature request issue for this -> https://github.com/allegroai/clearml/issues/418

It's not exactly "debugging", but rather a description of the generated model/framework (generated with pygraphviz).

Right so it uses whatever version is available on the agent.
Yeah it would be nice to have either a poetry_version (a-la https://github.com/allegroai/clearml-agent/blob/5afb604e3d53d3f09dd6de81fe0a494dacb2e94d/docs/clearml.conf#L62 ), rename the latter to manager_version , or just install from the captured environment, etc? 🤔

Created None

Oh nono, more like:

• Create a pipeline
• Add N steps to it
• Run the pipeline
• It fails/succeeds, the user does something with the output
• The user would like to add/modify some steps based on the results now (after closer inspection).I wonder if at (5), do I have to recreate the pipeline every time? 🤔

We have an internal mono-repo and some of the packages are required - they’re all available correctly for the controller, only some are required for the individual tasks, but the “magic” doesn’t happen 😞
That is, the controller does not identify them as a requirement, so they’re not installed in the tasks environment.

We're using self hosted account

I’ve tracked it down further, it seems the pigar utility does not apply any smart logic there.
The case we have is the following -

• We have a monorepo, but all modules/libs share a common namespace foo ; so e.g. working on module mod , we use from foo.mod import …
• This then looks for a module called foo , even though it’s just a namespace
• In the dist-info requirement, it seems any hyphen, dot, etc are swapped for an underscore, so our site-packages represents this as `foo_m...

It’s just that for the packages argument, ClearML says:

If not provided, packages are automatically added based on the imports used inside the wrapped function.

So… 🤔

We can change the project name’s of course, if there’s a suggestion/guide that will make them see past the namespace…

Still; anyone? 🥹 @<1523701070390366208:profile|CostlyOstrich36> @<1523701205467926528:profile|AgitatedDove14>

There's code that strips the type hints from the component function, just think it should be applied to the helper functions too :)

That is, we have something like:
` task = Task.init(...)
ds = Dataset.create(dataset_name=task.name, dataset_project=task.get_project_name(), use_current_task=True)

upload files

dataset.upload(show_progress=True)
dataset.finalize()

do stuff with task and dataset

task.close() `But because the dataset is linked to the task, the task is then moved and effectively becomes invisible 😕

Any thoughts AgitatedDove14 SuccessfulKoala55 ?