ClearML maintains a github action that sets up a dummy clearml-server,

You have one, it's the http://app.clear.ml (not a dummy one, but for this purpose it will work)
thoughts ?

oh, if this is the case, why not use the "main" server?

and just run the same code I run production

even if I move the Github workers internally where they could have access to the prod server, I am not sure I would like that, because it would pile up test data in the prod server that is not necessary

I think the best case scenario would be that ClearML maintains a github action that sets up a dummy clearml-server, so that anyone can use it as a basis to run their tests, so that they just have to change to URL of the server to the local one executed in the github action and they can test seamlessly all their code, wdyt?

so that any error that could arise from communication with the server could be tested

Let me check something

I want in my CI tests to reproduce a run in an agent because the env changes and some things break in agents and not locally

That said , if you could open a github issue and explain the idea behind it, I think a lot of people will be happy to have such process , i.e. CI process verifying code. And I think we should have a "CI" flag doing exactly what we have in the "hack" wdyt?

you mean to run it on the CI machine ?

yes

That should not happen, no? Maybe there is a bug that needs fixing on clearml-agent ?

It just to test that the logic being executed in if not Task.running_locally() is correct

Since I'm assuming there is no actual task to run, and you do not need to setup the environment (is that correct?)
you can do:
$ CLEARML_OFFLINE_MODE=1 python3 my_main.pywdyt?

Because it lives behind a VPN and github workers don’t have access to it

makes sense
If this is the case, I have to admit that combining offline-mode and remote execution makes sense, no?

I want in my CI tests to reproduce a run in an agent

you mean to run it on the CI machine ?

because the env changes and some things break in agents and not locally

That should not happen, no? Maybe there is a bug that needs fixing on clearml-agent ?

So there is a hack for it:
CLEARML_OFFLINE_MODE=1 python3 my_main.pyWhich is the same as calling Task.set_offline
Then inside the code After the Task.init call:
` task = Task.init(...)

not sure what the if here is?!

Task.debug_simulate_remote_task(task_id="offline-1") `This will make things act as if this is running remotely , i.e. your logic Task.running_remotely() will be called.
Do notice that in remote mode, all the arguments / data is read from the clearml-server into the code, and since this is an offline mode, I'm not sure what you'll be getting. (it seems to be workign as expected, i.e. you get what you started with, but I would make sure again)
Make sense ?

Actually I think I am approaching the problem from the wrong angle

I’d like to move to a setup where I don’t need these tricks

JitteryCoyote63 of course there is 🙂
Task.debug_simulate_remote_task(task_id="<task_id_here>")

JitteryCoyote63 if this is simulating an agent, the assumption is that the Task was already created, hence the task ID.

If i am working with Task.set_offline(True)

How would the two combine ? I mean off-line is be definition not executed by an agent, what am I missing ?

It does, tested 🙂 but you should as well

Because it lives behind a VPN and github workers don’t have access to it

wow if this works that’s amazing

Ho wow! is it possible to not specify a remote task? (If i am working with Task.set_offline(True))

In my github action, I should just have a dummy clearml server and run the task there, connecting to this dummy clearml server

mmmh good point actually, I didn’t think about it

is it different from Task.set_offline(True)?

yes exactly

It's the same but done from outside, you want the same and "offline" as well right?