If you one each "main" process as a single experiment, just don't call Task.init in the scheduler
No. since you are using Pool. there is no need to call task init again. Just call it once before you create the Pool, then when you want to use it, just do task = Task.current_task()
no i want all of them in the same experiment
but this gives the results in the same graph
i mean all 100 experiments in one project
so, if I call Task.init() before that line there is no need of calling Task.init() on line number 92
This code will give you one graph titled "loss" with two series: (1) trains (2) loss
In the side bar you get the title of the graphs, then when you click on them you can see the diff series on the graphs themselves
I have to create a main task for example named as main
So you want these two on two different graphs ?
You can do:task = Task.get_task(task_id='uuid_of_experiment')
task.get_logger().report_scalar(...)
Now the only question is who will create the initial Task, so that the others can report to it. Do you have like a "master" process ?
so, like if validation loss appears then there will be three sub-tags under one main tag loss
And you want all of them to log into the same experiment ? or do you want an experiment per 60sec (i.e. like the scheduler)
I have 100 experiments and I have to log them and update those experiments every 5 minutes
then if there are 10 experiments then I have to call Task.create() for those 10 experiments
Just so I understand,
scheduler executes main every 60sec
main spins X sub-processes
Each subprocess needs to report scalars ?
so I want loss should be my main title and I want two different graphs of train and test loss under that loss
like in the sidebar there should be a title called "loss" and under that two different plots should be there named as "train_loss" and "test_loss"
It will not create another 100 tasks, they will all use the main Task. Think of it as they "inherit" it from the main process. If the main process never created a task (i.e. no call to Tasl.init) then they will create their own tasks (i.e. each one will create its own task and you will end up with 100 tasks)
logger.report_scalar(title="loss", series="train", iteration=0, value=100)
logger.report_scalar(title="loss", series="test", iteration=0, value=200)
then if there are 100 experiments how it will create 100 tasks?
and that function creates Task and log them
logger.report_scalar("loss-train", "train", iteration=0, value=100)
logger.report_scalar("loss=test", "test", iteration=0, value=200)
notice that the title of the graph is its uniue id, so if you send scalars to with the same "title" they will show on the same graph