None of these submodules are required for the tasks, they are there for a different part of the project dealing with data generation.

So even having them fetched (even when cached) is quite the delay on the actual task.

Hi @<1523701087100473344:profile|SuccessfulKoala55>

I am using 1.8.0 for the clearml-agent.

Attached is the logfile.

I cleared the vcs cache manually already, it results in the same behaviour illustrated above
(allthough the logs show that it used the cache, I had another run without cache - but don't have the logs from that)

Any help would be greatly appreciated

Here is the latest version with all issues ironed out.

Alright, good to know.

This is the full log of the task.

I am trying to run HPO.

If there's some or any mechanism that would allow me to constrain what the task sees, it would really help me alot.

For anyone else interested in this, I wrote a little script which pulls all the data from a given project, seems to work well enough

On another attempt with a cleaned repository (no dirty commits) I get the same result, even though it states that it got a new commit id, so I'm at a loss at what is actually going wrong here:

`Using cached repository in "/root/.clearml/vcs-cache/lvgl-ui-detector.git.7c8ae2688810ceed26c1ebcc1e911cf2/lvgl-ui-detector.git"
remote: Enumerating objects: 11, done.
remote: Counting objects: 100% (11/11), done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 8 (delta 4), reused 7 ...

According to None I am supposed to install

libgl1

I changed my clearml.conf to include that installation for the task container started by the agent.

Will see if it helps in a minute

Yea, but even though it's cached, it takes quite a long time, because my project has really alot of submodules, due to the submodules having their own submodules as well.

I don't really understand why fetching the submodules is the default.

You mean a seperate branch to work in without the submodules linked?

Not really sure how I'd go about doing that.

I'd be more happy with an option to say 'pull_submodules=False'

When developing I use the poetry environment, but in the queues I let clearML handle the installation via pip.

There is no need to use poetry if the task is a one-time thing

just to give an idea about the scale of the problem on my side.

These are the submodules...

`Fetching submodule lvgl_ui_generator
2024-04-27 20:45:34
Fetching submodule lvgl_ui_generator/lv_drivers
Fetching submodule lvgl_ui_generator/lvgl
Fetching submodule lvgl_ui_generator_v2
Fetching submodule lvgl_ui_generator_v2/lv_micropython
Fetching submodule lvgl_ui_generator_v2/lv_micropython/lib/asf4
2024-04-27 20:45:40
Fetching submodule lvgl_ui_generator_v2/lv_micropython/lib/axtls
Fetch...

The installed packages of the task say this:

# Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0]

PyYAML == 6.0.1
clearml == 1.15.1
google google_api_core
google_cloud_storage == 2.16.0
ultralytics == 8.2.2

I do not know where the google_api_core comes from and I'd like to remove it.

Figured it out, I installed clearml[gs] but since I don't need that I removed it. it's gone now.

None

Here is an updated and improved version.

if anyone can tell me on how to improve the cookie situation, I'd be grateful

If there's source URLs in the plots of the task, how can I authenticate against ClearML to properly download them?

Or is there some SDK way to download them?

The code that is run in regards to clearml is really small.

If there's no mechanism on side of clearml, I might consider just putting that codebase into it's own submodule, making it a different repo without knowledge of the others.

I noticed poetry can be a problem in my run.
Not specifically due to the cache, but due to the installation of much more packages than the runtime might need.

When using regular pip, it will use the requirements list determined by ClearML to install necessary packages, which usually already excludes all dev-tools and similar.

I am not sure if poetry uses the cache properly, but I can't verify either atm.

None

It happens on all of my pipeline run attempts and there's nothing more that gives insight.

As an example:

python src/train.py 
ClearML Task: created new task id=102a4f25c5ac4972abd41f1d0b6b9708
ClearML results page: 

<unknown>:1: SyntaxWarning:

invalid decimal literal

<unknown>:1: SyntaxWarning:

invalid decimal literal

<unknown>:1: SyntaxWarning:

invalid decimal literal

<unknown>:1: SyntaxWarning:

invalid decimal literal

<unknown>:1: SyntaxWarning:

invalid decimal...

Here are the codefiles for my pipelines.

They do not work yet, I am struggling with the pipeline stuff quite a bit.

But both pipelines always give these warnings.

I am getting the same when starting regular tasks.
I think it has something to do with my paramaters, which contain an environment variable which contains a list of datasets

It comes from the PipelineDecorator.pipeline I assume or from PipelineDecorator.component

I noticed that it's actually independent of the pipelines

1.15.1

Yup.

I really don't know what it's about.

It doesn't affect the process. Everything seems to run fine.

If the warnings would provide a bit more info I could maybe pinpoint it better, but it's really all I got