Nvm, that does not seem to be a problem. I added a part to the logs in the post above. It shows that some packages are found from conda.

TRAINS_LOG_ENVIRONMENT=MYENVVAR works. Thank you!

Is there a way to capture uncommited changes with Task.create just like Task.init does? Actually, I would like to populate the repo, branch and packages automatically...

` args = parser.parse_args()
print(args) # args PRINTED HERE ON LOCAL

command = args.command
enqueue = args.enqueue
track_remote = args.track_remote
preset_name = args.preset
type_name = args.type
environment_name = args.environment
nvidia_docker = args.nvidia_docker

# Initialize ClearML Task
task = (
    Task.init(
        project_name="reinforcement-learning/" + type_name,
        task_name=args.name or preset_name,
        tags=...

That seems to be the case. After parsing the args I run task = Task.init(...) and then task.execute_remotely(queue_name=args.enqueue, clone=False, exit_process=True) .

Python 3.8.8, clearml 1.0.2

Good, at least now I know it is not a user-error 😄

clearml does everything for me 😄

Thank you, good to know!

(btw: the simulator is called carla, not clara :))

I use this snippet:

Logger.current_logger().set_default_upload_destination(
    "

" # or 

)

Now it works.

Thank you very much!

So if understand correctly, something like this should work?

task = Task.init(...) task.connect_configuration( {"agent.package_manager.system_site_packages": False} ) task.execute_remotely(queue_name, clone=False, exit_process=True)

Exactly. I don't want people to circumvent the queue 🙂

But I do not have anything linked correctly since I rely in conda installing cuda/cudnn for me

You can add and remove clearml-agents to/from the clearml-server anytime.

Same error.

I am not sure what happened, but my experiments are gone. However, the data directory is still filled.

There is no way to create an artifact/model/dataset without a task, right? Just always inherit from the parent task. And if cloned change the user to the user who did the clone.

(just for my own interest: how much does the enterprise version divert from the open source version? It it just extended or are there core changes to the enterprise version)

Oh, the hacked one.

Okay.

Let me check again.

Maybe something like this is how it is intended to be used?

` # run_with_clearml.py

def get_main_task():
task = Task.create(project="my_project", name="my_experiment", script="main_script.py")
return task

def run_standalone(task_factory):
Task.enqueue(task_factory())

def run_in_pipeline(task_factory):
pipe = Pipelinecontroller()
pipe.add_step(preprocess, ...)
pipe.add_step(base_task_factory=task_factory, ...)
pipe.add_step(postprocess, ...)
pipe.start()

if...

So with pipeline decorators can I implement this logic?

It seems like the services-docker is always started with Ubuntu 18.04, even when I use
task.set_base_docker( "continuumio/miniconda:latest -v /opt/clearml/data/fileserver/:{}".format( file_server_mount ) )

It seems like clearml removes the dev... from torch == 1.14.0.dev20221205+cu117 in the /tmp/ cached requirements.txt

No problem! I profit so much from clearml 🙂

[root@dc01deffca35 elasticsearch]# curl `
{
"cluster_name" : "clearml",
"status" : "yellow",
"timed_out" : false,
"number_of_nodes" : 1,
"number_of_data_nodes" : 1,
"active_primary_shards" : 10,
"active_shards" : 10,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 10,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_nu...

Can you ping me when it is updated in None so I can update my installation?

Could be clean log after restart. Unfortunately, I restarted the server right away 😞 I gonna post if it happens again with the appropriate logs.