I guess so

the Task object has a method called Task.execute_remotely

Look it up here:
https://allegro.ai/docs/task.html#trains.task.Task.execute_remotely

I restarted the server and it works now

its like ps + grep together 😄

but I can't seem to run docker-compose down

2021-10-11 10:07:19 ClearML results page: `
2021-10-11 10:07:20
Traceback (most recent call last):
File "tasks/hpo_n_best_evaluation.py", line 256, in <module>
main(args, task)
File "tasks/hpo_n_best_evaluation.py", line 164, in main
trained_models = get_models_from_task(task=hpo_task)
File "tasks/hpo_n_best_evaluation.py", line 72, in get_models_from_task
with open(pickle_path, 'rb') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/home/elior/.clearml/c...

Version 1.1.1

Snippet of which part exactly?

Yeah, logs saying "file not found", here is an example

Another thing I noticed now it happens on my personal computer, when I execute the same pipeline from the exact same commit with exact same data on another host it works without these problems

this is another execution:

No absolutely not. Yes I do have a GOOGLE_APPLICATION_CREDENTIALS environment variable set, but nowhere do we save anything to GCS. The only usage is in the code which reads from BigQuery

I double checked that the task object is the correct one

💪

I will

Good, so if I'm templating something using clearml-task (without queue, so the task is in draft mode) it will use this task? Even though it never exeucted?

Did not solve this

Saving part from task A:

pipeline = trials.trials[index]['result']['pipeline'] output_prefix = 'best_iter_' if i == 0 else 'iter_' task.upload_artifact(name=output_prefix + str(index), artifact_object=pipeline)

I jsut think that if I use "report_table" I might as well be able to download it as CSV or something

I get this
` [ec2-user@ip-10-0-0-95 ~]$ docker-compose down
WARNING: The TRAINS_HOST_IP variable is not set. Defaulting to a blank string.
WARNING: The TRAINS_AGENT_GIT_USER variable is not set. Defaulting to a blank string.
WARNING: The TRAINS_AGENT_GIT_PASS variable is not set. Defaulting to a blank string.
ERROR: Couldn't connect to Docker daemon at http+docker://localhost - is it running?

If it's at a non-standard location, specify the URL with the DOCKER_HOST environment variable. `

so in my code, I'll use this environment variable to read from disk

I guess what I want is a way to define environment variables in agents

SuccessfulKoala55 AppetizingMouse58

[ec2-user@ip-10-0-0-95 ~]$ df -h Filesystem Size Used Avail Use% Mounted on devtmpfs 3.9G 0 3.9G 0% /dev tmpfs 3.9G 0 3.9G 0% /dev/shm tmpfs 3.9G 880K 3.9G 1% /run tmpfs 3.9G 0 3.9G 0% /sys/fs/cgroup /dev/nvme0n1p1 8.0G 6.5G 1.5G 82% / tmpfs 790M 0 790M 0% /run/user/1000

From the UI

Can you lend a few a words about how the not-pip freeze mechanism of detecting packages work?

Okay Jake, so that basically means I don't have to touch any server configuration regarding the file-server on the trains server. It will simply get ignored and all I/O initiated by clients with the right configuration will cover for that?

Exactly

I want to collect the dataframes from teh red tasks, and display them in the pipeline task

I just tried setting the conf in the section Martin said, it works perfectly

To be clearer - how to I refrain from using the built in file-server altogether - and use MINIO for any storage need?

Could be, my message is that in general, the ability to attach a named scalar (without iteration/series dimension) to an experiment is valuable and basic when looking to track a metric over different experiments