looks like this already been solved, it's just not merged yet -- https://github.com/allegroai/clearml/issues/457

AgitatedDove14 that looks good, i'd like to request an addition to control the other max_connections i see on that file, as i also noticed that uploads are sometimes slow, and i see here max_connections=2

AgitatedDove14 hm, that fails to install for me

` Installing collected packages: clearml
Attempting uninstall: clearml
Found existing installation: clearml 1.0.5
Uninstalling clearml-1.0.5:
Successfully uninstalled clearml-1.0.5
Rolling back uninstall of clearml
Moving to /opt/anaconda/bin/clearml-data
from /tmp/pip-uninstall-tcd44rxv/clearml-data
Moving to /opt/anaconda/bin/clearml-init
from /tmp/pip-uninstall-tcd44rxv/clearml-init
Moving to /opt/anaconda/bin/...

1.0.5 vs 1.0.6+

hey CostlyOstrich36 , it's a bit different than that

I have a git repository that containers multiple git submodules
I'm starting the task at the root repository, however, if I have changes within the submodules, clearml doesn't see them

I was looking into implementing my own diff system, as I see the Task.create http API supports that, but the Task.create python API doesn't seem to have support for it

additionally, I found is that clearml==1.0.5 package is able to find these partial changes, newer versions find nothing at all, maybe it's because it's always comparing against remote

weird, the other versions work just fine

AgitatedDove14

I'm creating it like this
` task = clearml.Task.create(
task_name=task_name,
project_name=settings.project,
repo=settings.tps_repo,
script=ep_fn,
docker=settings.image,
docker_args="-e TRAIN_CONFIG={} -e VALIDATION_CONFIG={} -e EXP={} -e EXP_CWD={} -e EXP_GIT={} {}".format(
settings.train, settings.validation,
settings.exp, settings.cwd,
settings.cwd_git,
settings.docker_args
...

nm, i restarted the instance, now install works, let me see the diffs

` # pip install clearml==1.0.6rc2
Collecting clearml==1.0.6rc2
Downloading clearml-1.0.6rc2-py2.py3-none-any.whl (1.0 MB)
|████████████████████████████████| 1.0 MB 12.1 MB/s
Requirement already satisfied: pyjwt<3.0.0,>=1.6.4 in /opt/anaconda/lib/python3.8/site-packages (from clearml==1.0.6rc2) (2.1.0)
Requirement already satisfied: PyYAML>=3.12 in /opt/anaconda/lib/python3.8/site-packages (from clearml==1.0.6rc2) (5.3.1)
Requirement already satisfied: numpy>=1.10 in /opt/anaconda/lib/...

no diff still

one thing i noticed is that it's not able to find the branch name on >=1.0.6x , while on 1.0.5 it can

BTW: submodule diff should always get stored, can you provide some error logs on fail cases?

It's hard to pinpoint, I have a repository with several submodules, some changes are found, others not. I've added untracked files, but sometimes it simply doesn't pick them. Running git diff --submodule=diff (which is the same command in the GitDetector class) always return the correct results

Before manually modifying the diff:
If you have local commits (i.e. un-pushed) this m...

so,
1.0.5 = yes diffs
1.0.6rc1 = no diffs
1.0.6rc2 = no diffs

well that's much faster, from 1mb/s to 60mb/s 🙂

hm, maybe, i will try to override this to see what happens, thanks!

one final addition, this flag: store_code_diff_from_remote don't seem to change anything in regards of this issue

i'll be using the update_task, that worked just fine, thanks 🙂

AgitatedDove14 I want to schedule bulk tasks to run via agents, so I'm running create

That makes, sense, will investigate, thanks