AgitatedDove14 I have a training job, very similar to this one: https://clearml.slack.com/archives/CTK20V944/p1654606983176539?thread_ts=1654604976.568279&cid=CTK20V944
AgitatedDove14 AFAIK, ClearML sends the git repo, branch and commit id IF the a git repo is present at the working directory, without needing me to specify it. When it does send those information, clearml agent tries to pull the repo with the specified branch and commit id, and the project goes on after that. This is what I meant by mentioning "git integration". If a git repo is not present at the working directory, clearml agent just bypasses the "pulling the repo" part, as there is none specified. Instead, it uploads my script only.
Oddly, clearml agent is unable to run the script if a git repo is present and marks the experiment as complete instantly. If a git repo is not present, it runs the script perfectly as intended.
Ohh yes, if the execution script is not on git and git exists, it will not add it (it will add it if it is in a tracked file via the uncommitted changes section)
ZanyPig66 in order to expand the support to your case. Can you explain exactly which files are on git and which are not?
Okay, after lots of trials and failures, I found that the execution script should be on git too. The changes are being sent by clearml automatically, but the files that do not exist within the repo are apparently are not being sent. This is somehow counter-intuitive.
AgitatedDove14 To elaborate, the code below does not work with git integration activated.
` from clearml import Task
task = Task.create(
Task.enqueue(task, "default") `
However, the very same code does work WITHOUT git integration activated.
The driver script (the one initializes models and initializes a training sequence) was not at git repo and besides that one, everything is.
Yes there is an issue when you have both git repo and totally uncommitted file, since clearml can store either standalone script or a git repository, the mix of the two is not actually supported. Does that make sense ?