Not sure on the cause but if you do:

mp.set_start_method('fork', force=True)

There is no semaphore leakage

works most of time, this occurs only few times

Hmm let me check something

check on the iteration on the right side,
i tried to start the experiment few times, and sometimes, 1 or 2 of the experiment seems just won’t start

No sure what O'm seeing here

seems the not logging problem is back

Let me check, see what can be learned ...

yes, that’s the error i got

we need to evaluate the result across many random seeds, so each task needs to log the result independently.

Ohh that kind of makes sense to me 🙂
Yes I'm also getting:

/usr/local/lib/python3.6/multiprocessing/semaphore_tracker.py:143: UserWarning: semaphore_tracker: There appear to be 74 leaked semaphores to clean up at shutdown
  len(cache))

Not sure about that ...

and one experiment takes 40 hours to run, so i let them run in parallel

in my case, we need to evaluate the result across many random seeds, so each task needs to log the result independently.

Wouldn't it make sense to use a single one ?

Why are you initializing 3 diff Tasks ?

Let me check that for a sec

No sure, here is the code

there is a semaphore warning, not sure if it’s related

Can you resend it?
Is the Task marked as closed when the process ends ?

when i run the code i send you the other day, logging is perfect, but at the end of the code there is a semaphore warning, not sure if it’s related

No need, it should auto close it if you started it with Task.init (or the agent executed it)

one quick question is that do i need to do some task.close() at the end of each process?

Releasing an RC

YEY

Nice job

working very well actually

🤞

Let me know if it solved it, if it did I'll make sure we push the RC

Thanks!

yeah sure

Can I send you a wheel to test ?

Let me know if you need some tests