Hmm the agent's venv caching is the next thing on the to do list for the agent (post clearml release).
Currently the easiest thing is to build a new docker image with the entire "Installed packages" section and use that as the base docker image.
(The installed packages format is "requirement" compatible, so you can just use it as is when building the dockerfile)
The second option is to wait for the next clearml-agent release (probably in a couple of weeks)