I think I figured it out. I need to use argparse to set the hyperparameters. Then it logs it automatically

Thanks .. I tried both the approaches.

Hi FriendlyKoala70 , sorry for replying only now...
You are correct - you can use argparse to define hyper parameters that will be automatically logged and visible in the WebApp. You can check out the https://github.com/allegroai/trains/blob/master/examples/hyper_parameters_example.py to see this and other ways you can report and connect hyper-parameters to your experiment.

Or you can do:
param={'key': 123}
task.connect(param)

I only see these tensorflow flags and not any of the actual hyperparameters