There maybe some issue with assumptions the new Trains code makes as to how the experiment was created
Hi GracefulDeer63 ,
A few questions:
When editing the hyperparameters (in v0.16.0 this is located under CONFIGURATION\HYPER PARAMETERS\General
) - do you see your changed correctly in the UI? After running the modified experiment using the Trains Agent, do you still see your edited changes in the hyper-parameters in the UI or are the values in the UI back to the default values?
There was a change in parameters nesting, perhaps it's causing it to drop your old values somehow... can you print out task._get_task_property('hyperparams')
?
Try locating the hyperparams
property in the task.data
dict... If it's not there, try seeing if you have an execution.parameters
property in the same dict
I misunderstood you then, anyway thanks a lot for your help))
it prints an empty dict
I’m doing Task.init() in the script, maybe it somehow resets connected parameters… but it used to work before, weird
DilapidatedDucks58 this does not seem to be the same issue you're having, am I correct?
On my local machine was previous version of trains package
Hey! I tried all of you said, started a new experiment from scratch, updated trains python package version to the latest version in the Installed packages section and that's all didn't work until i updated local trains python package version to 0.16.0 and then repeated the steps above. Now it seems to work fine
same here, changing arguments in the Args section of Hyperparameters doesn’t work, training script starts with the default values.
trains 0.16.0
trains-agent 0.16.0
trains-server 0.16.0
Oh, GracefulDeer63 so previously your trains
package was not 0.16.0?
I change the arguments in Web UI, but it looks like they are not parsed by trains
ValueError: Task has no hyperparams section defined
DilapidatedDucks58 what do you get when callingfrom trains.backend_api import Session print(Session.api_version)
After Task.init()
nope, same problem even after creating a new experiment from scratch
Hi GracefulDeer63 ,
What trains
and trains-server
versions are you using?
Try doing that from scratch - i.e. creating the experiment again using the new version (by running the development code again locally and letting it create a new experiment)
What exactly do you connect in your code? dict? argparser?
Can you perhaps print out the result of task.get_parameters()
and see if the argparser-related keys have an Args
prefix? when running remotely, of course
I don’t connect anything explicitly, I’m using argparse, it used to work before the update
I updated the version in the Installed packages section before starting the experiment
DilapidatedDucks58 same question - Was your experiment started using a previous Trains python package version? 🙂
OK, and in the experiment log you see the default values being used?
trains-server - 0.16.0 Yes, I see Yes, after running the modified experiment, I do see my edited values in the hyper-parameters in the UI
Oh, so that's exactly the scenario I suspected when asking if the experiment was created using a previous Trains version and ran remotely using Trains 0.16.0 🙂
A few more questions:
What is your Trains python package version? Was your experiment started using a previous Trains python package version? What hyper-parameter section are the values stored in?