I use ngrok to connect to it

sorry, not very clear.

also - line 77 which sets (non-system) tags is not invoked for me, thus if I define different tags for both task and dataset - then latter is being lost

Adding venv into cache: /root/.clearml/venvs-builds/3.8 Running task id [aa2aca203f6b46b0843699d1da373b25]: [.]$ /root/.clearml/venvs-builds/3.8/bin/python -u '/root/.clearml/venvs-builds/3.8/code/-m filprofiler run catboost_train.py'

So I thought, maybe I can tell clearml-session to use hostname from ngrok

yes, I’ll try it out

DisgustedDove53 ?

But here you can see why it didn’t succeed

I tried this, but didn’t help:
input_models = current_task.models["input"] if len(input_models) == 1: input_model_as_input = {"name": input_models[0].name, "type": ModelTypeEnum.input} response = current_task.send(DeleteModelsRequest( task=current_task.task_id, models=[input_model_as_input] ))

not a full log yet (will have to inspect it to not have any non-public info), but something potentially interesting

I guess that’s because ngrok is not like a Dynamic DNS

https://github.com/catboost/catboost/search?q=tensorboard

I think we need logging here: https://github.com/allegroai/clearml-session/blob/bf1851cd3831c19cc0eadd9b2ffc0613f97f16e1/clearml_session/main.py#L564

the task is running, but no log output from fil-profiler (when ran totally locally, then it does some logging at the very beginning)

Wanted to check if MLFlow supports catboost. Apparently, it does. Pull request was merged 16 hours ago. Nice timing 😃

task_trash_trash is probably irrelevant, as the latest entry there is from Dec 2021

I see that scheduler task UI has the capabilities to edit user properties. But I don’t see how I can read and/or write them through code

I got it working!

yeah, I missed the fact that I’m running it not by opening remote jupyter in browser, but by connecting to remote jupyter with local VS Code

I do see the “Data Processing” type task in UI together with all other dataset-related features, like lineage plot

I create the dataset like this:
` project_name = "Sandbox"
task_name = "get_raw_data"
task = Task.init(project_name=project_name,
task_name=task_name,
task_type=Task.TaskTypes.data_processing,

             )

dataset = Dataset.create(use_current_task=True)

adding some files here

dataset.upload(verbose=True)

dataset.finalize(verbose=True) `

ideally, I want to hardcode, e.g. use_staging = True, enqueue it; and then via clone-edit_user_properties-enqueue in UI start the second instance

yes, same

also, I tried running the notebook directly in remote jupyter - I see correct uncommitted changes

gotcha, thanks!

but the old ones are there, and I can’t do anything about them

but I don’t get to this line, because my task is already of type data_processing

I guess so, lemme double-check

system tags cannot be seen from UI. right?