logger.report_scalar("loss", "train", iteration=0, value=100)
logger.report_scalar("loss", "test", iteration=0, value=200)
so, if I call Task.init() before that line there is no need of calling Task.init() on line number 92
so , it will create a task when i will run it first time
So you want these two on two different graphs ?
so I want loss should be my main title and I want two different graphs of train and test loss under that loss
my scheduler will be running every 60 seconds and calling main function
Are you using tensorboard or do you want to log directly to trains ?
logger.report_scalar("loss-train", "train", iteration=0, value=100)
logger.report_scalar("loss=test", "test", iteration=0, value=200)
notice that the title of the graph is its uniue id, so if you send scalars to with the same "title" they will show on the same graph
No. since you are using Pool. there is no need to call task init again. Just call it once before you create the Pool, then when you want to use it, just do task = Task.current_task()
Create one experiment (I guess in the scheduler)
task = Task.init('test', 'one big experiment')
Then make sure the the scheduler creates the "main" process as subprocess, basically the default behavior)
Then the sub process can call Task.init and it will get the scheduler Task (i.e. it will not create a new task). Just make sure they all call Task init with the same task name and the same project name.
def combined(path,exp_name,project_name):
temp = Task.create(task_name="exp_name")
logger = temp.current_logger()
logger.report_scalar()
def main():
task=Task.init(project_name="test")
[pool.apply_async(combined, args = (row['Path'], row['exp_name'], row['project_name'])) for index,row in temp_df.iterrows()]
scheduler = BlockingScheduler()
scheduler.add_job(main, 'interval', seconds=60, max_instances=3)
scheduler.start()
yes But i want two graphs with title as train loss and test loss and they should be under main category "loss"
each subprocess logs one experiment as task
@<1523720500038078464:profile|MotionlessSeagull22> you cannot have two graphs with the same title, the left side panel presents graph titles. That means that you cannot have a title=loss series=train & title=loss series=test on two diff graphs, they will always be displayed on the same graph.
That said, when comparing experiments, all graph pairs (i.e. title+series) will be displayed as a single graph, where the diff series are the experiments.
In the side bar you get the title of the graphs, then when you click on them you can see the diff series on the graphs themselves
@<1523701205467926528:profile|AgitatedDove14> I want to log directly to trains using logger.report_scalar
so what I have done is rather than reading sequentially I am reading those experiments through multiprocessing and for each experiment I am creating new task with specified project_name and task_name
like in the sidebar there should be a title called "loss" and under that two different plots should be there named as "train_loss" and "test_loss"
It will not create another 100 tasks, they will all use the main Task. Think of it as they "inherit" it from the main process. If the main process never created a task (i.e. no call to Tasl.init) then they will create their own tasks (i.e. each one will create its own task and you will end up with 100 tasks)
and that function creates Task and log them
Just so I understand,
scheduler executes main every 60sec
main spins X sub-processes
Each subprocess needs to report scalars ?
main will initialize parent task and then my multiprocessing occurs which call combined function with parameters as project_name and exp_name
Can my request be made as new feature so that we can tag same type of graphs under one main tag
Sure, open a Git Issue :)
no i want all of them in the same experiment
so, like if validation loss appears then there will be three sub-tags under one main tag loss