no, I’m providing the id of task which generated the model as a “hyperparam”

“supply the local requirements.txt” this means I have to create a separate requirements.txt for each of my 10+ modules with different clearml tasks

and the problem is with system_tags=[“dataset”]

here: https://github.com/allegroai/clearml/blob/master/clearml/datasets/dataset.py#L75

it certainly does not use tensorboard python lib

as I understand, it uses tensorboard from C++ code

I guess this is the one https://catboost.ai/docs/concepts/python-reference_catboostipythonwidget.html

https://github.com/catboost/catboost/search?q=tensorboard

looking into the output folder of catboost, I see 3 types of metrics outputs:
tfevents (can be read by tensorboard) catboost_training.json (custom (?) format). Is read here to be shown as an ipython widget: https://github.com/catboost/catboost/blob/c2a6ed0cb85869a73a13d08bf8df8d17320f8215/catboost/python-package/catboost/widget/ipythonwidget.py#L93 learn_error.tsv, test_error.tsv, time_left.tsv which have the same data as json. Apparently they are to be used with this stale metrics viewer pr...

nope, catboost docs offer to manually run tensorboard against the output folder https://catboost.ai/docs/features/visualization_tensorboard.html

Wanted to check if MLFlow supports catboost. Apparently, it does. Pull request was merged 16 hours ago. Nice timing 😃

Although it is only for model tracking, autologging is yet to be implemented there

https://github.com/mlflow/mlflow/pull/2417

docker: nvidia/cuda:11.2.1-cudnn8-runtime-ubuntu20.04
jupyterlab 3.0.11
clearml lib 0.17.5
no warnings
2021-03-24 17:55:44,672 - clearml.Task - INFO - No repository found, storing script code instead

yeah, I missed the fact that I’m running it not by opening remote jupyter in browser, but by connecting to remote jupyter with local VS Code

also, I tried running the notebook directly in remote jupyter - I see correct uncommitted changes

found the code: https://github.com/ipython/ipykernel/blob/master/ipykernel_launcher.py

so I assume it’s somehow related to remote connection form VS Code

Also, installed packages are also incorrect (not including ones that I install fmor within the notebook using !pip install package_name_here )

` # Python 3.8.5 (default, Jan 27 2021, 15:41:15) [GCC 9.3.0]
azure_storage_blob == 12.8.0
boto3 == 1.17.30
clearml == 0.17.5
google_cloud_storage == 1.36.2
ipykernel == 5.5.0

Detailed import analysis

**************************

IMPORT PACKAGE azure_storage_blob

clearml.storage: 0

IMPORT PACKAGE boto3

clearml.storage: 0

IMPORT PACKA...

yes, but note that I’m not talking about VS Code instance set up be clearml-session, but about a local one. I’ll do another test to determine whether VS Code from clearml-session suffers from the same problem

“VSCode running locally connected to the remote machine over the SSH” - exactly

for the tasks that are not deleted, log is different:
[2021-09-09 12:19:07,718] [8] [WARNING] [clearml.service_repo] Returned 400 for tasks.dequeue in 4ms, msg=Invalid task id: status=stopped, expected=queued

log:
[2021-09-09 11:22:09,339] [8] [WARNING] [clearml.service_repo] Returned 400 for tasks.dequeue in 2ms, msg=Invalid task id: id=28d2cf5233fe41399c255950aa8b 8c9d,company=d1bd92a3b039400cbafc60a7a5b1e52b

this does not prevent from enqueuing and running new tasks, rather an eyesore

I think they appeared when I had a lot of HPO tasks enqueued and not started yet, and then I decided to either Abort or Archive them - I don’t remember already

self-hosted. Just upgraded to latest version today (1.1.1). The problem appeared when we were still using 1.0.2

no new unremovable entries have appeared (although I haven’t tried)

but the old ones are there, and I can’t do anything about them

not sure - ideally I would like to see these tables (e.g. with series_name, series_dtype, number_of_non_na_values as columns) back to back in the GUI to track the transformations. I think it isn’t possible with Dataset . Anyway, this whole scenario is not a must have, but a nice to have.