AgitatedDove14 I tried your code snippet, something like this:
task = Task.init(...) if task.running_locally(): # wait for the repo detection and requirements update task._wait_for_repo_detection() # reset requirements task._update_requirements(None) task.execute_remotely(queue="default") ... rest of task here ...It’s able to create a draft successfully with no items in the installed packages section, but it only installed Cython and began the execution immediately. None of the requirements from requirements.txt were installed. Any workaround for this?

task = Task.init(...) if task.running_locally(): # wait for the repo detection and requirements update task._wait_for_repo_detection() # reset requirements task._update_requirements(None)Regarding this, does this work if the task is not running locally and is being executed by the trains agent?

just reran and got the same results with the flag for force_analyze_entire_repo set to false 😞 still has the following import error:
ImportError: cannot import name 'bigquery' from 'google.cloud' (unknown location)

Since the package requirements vary from task to task, setting it up in trains.conf might be a little tedious to go back and change. Most of the time, the package analysis has worked very well, but in this particular use case it failed to detect. I think task.init flag would be great!

GiddyTurkey39 my bad 😞 try this one
task._update_requirements({})

Regarding this, does this work if the task is not running locally and is being executed by the trains agent?

This line: "if task.running_locally():" makes sure that when the code is executed by the agent it will not reset it's own requirements (the agent updates the requirements/installed_packages after it installs them from the requiremenst.txt, so that later you know exactly which packages/versions were used)

If you have a few lines for resetting, I would love to have that as an option as well!

Hi GiddyTurkey39
Are you referring to an already executed Task or the current running one?
(Also, what is the use case here? is it because the "installed packages are in accurate?)

It was set to true earlier, I changed it to false to see if there would be any difference but doesn’t seem like it

GiddyTurkey39

A flag would be really cool, just in case if theres any problem with the package analysis.

Trying to think if this is a system wide flag (i.e. trains.conf) or a flag in task.init.
What do you think?

force_analyze_entire_repo to to True 🙂
(false is the default)

AgitatedDove14 worked like a charm, thank you!

I was thinking more along the lines of a draft task that has yet to be enqueued(so it can still be edited). The installed packages grabs most packages except for a few, which is why I am resorting to clearing the packages so that requirements.txt is used. I’ve been using the UI so far to do it, but would like to move the process to a script

Let me experiment with force_analyze_entire_repo and I’ll get back to you! It was missing google-cloud-bigquery , an import from a separate file in the repository. Hopefully that should resolve the problem. A flag would be really cool, just in case if theres any problem with the package analysis.

Regrading resetting it via code, if you need I can write a few lines for you to do that , although that might be a bit hacky.
Maybe we should just add a flag saying, use requirements.txt ?
What do you think?

I think task.init flag would be great!

👍

This line: “if task.running_locally():” makes sure that when the code is executed by the agent it will not reset it’s own requirements (the agent updates the requirements/installed_packages after it installs them from the requiremenst.txt, so that later you know exactly which packages/versions were used)

got it! Thanks for explaining

task = Task.init(...) if task.running_locally(): # wait for the repo detection and requirements update task._wait_for_repo_detection() # reset requirements task._update_requirements(None)🙂

I also tried detection with pip freeze and that didn’t work

GiddyTurkey39
BTW: you can always add the missing package via code:
Task.add_requirements('torch', optional_version)

Regrading the missing packages, you might want to test with:
force_analyze_entire_repo: falsehttps://github.com/allegroai/trains/blob/c3fd3ed7c681e92e2fb2c3f6fd3493854803d781/docs/trains.conf#L162

Or if you have a full venv you like to store instead:
https://github.com/allegroai/trains/blob/c3fd3ed7c681e92e2fb2c3f6fd3493854803d781/docs/trains.conf#L169

BTW:
What is the missed package?

It was set to true earlier, I changed it to false to see if there would be any difference but doesn’t seem like it

I would actually just add:
Task.add_requirements('google.cloud')Before the Task.init call (Notice, it has to be before the the init call)