I see..
Generally speaking If that is the case, I would think it might be better to use the docker mode, it offers way more stable environment, regardless on the host machine runinng the agent. Notice there is no need to use custom containers, as the agent will basically run the venv process, only inside a container, allowing you to reuse offf the shelf containers.

If you were to add this, where would you put it? I can use a modified version of 

clearml-agent

Yep, that would be the place, basically you can add it here,
https://github.com/allegroai/clearml-agent/blob/bd411a19843fbb1e063b131e830a4515233bdf04/clearml_agent/commands/worker.py#L2095
You can get the actual script from the Task object current_task .
Please feel free to PR once you get it working, I'll be more than happy to help