It worked finally 🙂

No

and are these the correct values?

https://www.allegro.ai/docs/deploying_trains/trains_agent_install_configure/

Anyways from what I see in the logs it shows agent.default_python = 3.7, cuda = 100, cudnn=75

the agent uses the same configuration file

It would be great if these issues are well elaborated in the documentation. Though the documentation is pretty good.

I only see trains conf file

And how do I change agent configuration file?

Yes. You can see the agent's configuration in the experiment's log - all values are printed there

In the conf file?

Did you make sure the agent's default python version and cuda / cudnn are configured correctly?

Yes very much

Are you running the agent on the same machine?

However all packages are cached so it won't download again

Wherever running an experiment, it will install all required packages in a virtual environment to make sure the experiment is executed exactly as expected

But then how the normal one I.e without cloning worked well

Yes 😂

However will it always install all the packages again and again? Is there any workaround for that?

Good 🙂 I see you still have issues with your CUDA installation

I ran it outside the examples folder and it works

I can see you failed experiments in the demo server, but I can't see any completed experiment from which they were cloned...

Is there a difference in the uncommitted changes section before your changes and after?

It fails only when I make any changes

I did that with the agent without making changes it works

You can first try to run your experiment again (not by cloning and running in the agent but by executing it again locally). If you like, you can copy the example and run it from another folder which is not located inside a git repo

I need to showcase this to my senior tomorrow

So now what do I do?

I know, they represent the changes you made to the example script

I just checked there are some uncommitted changes that I see in execution