Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Multiprocessing.Pool.Remotetraceback: """ Traceback (Most Recent Call Last): File "/Usr/Lib/Python3.6/Multiprocessing/Pool.Py", Line 119, In Worker Result = (True, Func(*Args, **Kwds)) File "/Usr/Lib/Python3.6/Multiprocessing/Pool.Py", Line 44, I

multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/usr/lib/python3.6/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/usr/lib/python3.6/multiprocessing/pool.py", line 44, in mapstar
return list(map(*args))
File "/home/derk/rndeep/rndeep/main_scripts/train_example.py", line 45, in main
task.connect_configuration(parameters)
File "/home/derk/rndeep/venv/lib/python3.6/site-packages/clearml/task.py", line 1133, in connect_configuration
name=name, description=description, config_type='dictionary', config_dict=configuration)
File "/home/derk/rndeep/venv/lib/python3.6/site-packages/clearml/backend_interface/task/task.py", line 1829, in _set_configuration
self._edit(configuration=configuration)
File "/home/derk/rndeep/venv/lib/python3.6/site-packages/clearml/backend_interface/task/task.py", line 1771, in _edit
raise ValueError('Task object can only be updated if created or in_progress')
ValueError: Task object can only be updated if created or in_progress
"""

I am getting the following error. I try to run multiple tasks with one script (cross-validation). All the tasks are explicityly closed via task.close(). Any idea how to solve this error? The train() function is called in a for loop, for the first iteration everything goes fine. But for the second iteration there is this error

  
  
Posted 3 years ago
Votes Newest

Answers 4


GreasyPenguin14 whats the clearml version you are using, OS & Python ?
Notice this happens on the "connect_configuration" that seems to be called after the Task was closed, could that be the case ?

  
  
Posted 3 years ago

This was with using one task in a multiprocessing.pool and the next one in the main process. I switched to have all tasks in a separate process via ProcessPoolExecutor and now it runs fine 👍 (version 0.17.5)

  
  
Posted 3 years ago

Hi GreasyPenguin14 , can you share an approximate snippet of your code?

  
  
Posted 3 years ago

yes that makes send, I think what happened is one of the processes completed the Task (i.e. closed it) before the others did, and so they threw exception.

I switched to have all tasks in a separate process

I think that's probably the best (performance wise as well), nice!

  
  
Posted 3 years ago
709 Views
4 Answers
3 years ago
one year ago
Tags