Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
We Use Environment Variables In Our

We use environment variables in our clearml.conf , but those might only be available later down the line (after dotenv.load_dotenv is called).
When those environment variables are unavailable, the SDK crashes on import clearml with:
Failed loading /home/.../clearml.conf: Cannot resolve variable ${CLEARML_ENDPOINT} (line: 2, col: 25)So several relevant questions:
Can we use pseudo environment variables that will be available immediately (i.e. os.environ['CLEARML_ENDPOINT'] = None )? Will the SDK access the updated environment variables later on? Can we defer reading the configuration to a later point?

  
  
Posted 2 years ago
Votes Newest

Answers 13


You don't load the configuration during from clearml import Task config is loaded during Task.init() So you can make all your configuration additions up to the point Task.init() is run

  
  
Posted 2 years ago

From looking at the code it should be fine, I think (i.e. won't load the configuration)

  
  
Posted 2 years ago

So the config loading is not deferred until execution 😞

  
  
Posted 2 years ago

Would it also be loaded on e.g. Task.running_locally() ?

  
  
Posted 2 years ago

Yeah 🙂

  
  
Posted 2 years ago

SuccessfulKoala55 CostlyOstrich36 actually it is the import statement, just finally got around to the traceback:

File "/home/.../ccmlp/configs/mlops.py", line 4, in <module> from clearml import Task File "/home/.../.venv/lib/python3.8/site-packages/clearml/__init__.py", line 4, in <module> from .task import Task File "/home/.../.venv/lib/python3.8/site-packages/clearml/task.py", line 31, in <module> from .backend_interface.metrics import Metrics File "/home/.../.venv/lib/python3.8/site-packages/clearml/backend_interface/__init__.py", line 2, in <module> from .task import Task File "/home/.../.venv/lib/python3.8/site-packages/clearml/backend_interface/task/__init__.py", line 1, in <module> from .task import Task File "/home/.../.venv/lib/python3.8/site-packages/clearml/backend_interface/task/task.py", line 31, in <module> from ...binding.artifacts import Artifacts File "/home/.../.venv/lib/python3.8/site-packages/clearml/binding/artifacts.py", line 23, in <module> from ..backend_interface.metrics.events import UploadEvent File "/home/.../.venv/lib/python3.8/site-packages/clearml/backend_interface/metrics/__init__.py", line 2, in <module> from .interface import Metrics File "/home/.../.venv/lib/python3.8/site-packages/clearml/backend_interface/metrics/interface.py", line 11, in <module> from ..base import InterfaceBase File "/home/.../.venv/lib/python3.8/site-packages/clearml/backend_interface/base.py", line 12, in <module> from ..config import config_obj File "/home/.../.venv/lib/python3.8/site-packages/clearml/config/__init__.py", line 13, in <module> config_obj = load_config(Path(__file__).parent) # noqa: F405 File "/home/.../.venv/lib/python3.8/site-packages/clearml/backend_api/config/__init__.py", line 15, in load config.load_relative_to(this_module_path, *additional_module_paths) File "/home/.../.venv/lib/python3.8/site-packages/clearml/backend_config/config.py", line 131, in load_relative_to self.reload() File "/home/.../.venv/lib/python3.8/site-packages/clearml/backend_config/config.py", line 192, in reload self.replace(self._reload()) File "/home/.../.venv/lib/python3.8/site-packages/clearml/backend_config/config.py", line 175, in _reload config = functools.reduce( File "/home/.../.venv/lib/python3.8/site-packages/clearml/backend_config/config.py", line 178, in <lambda> self._read_single_file(file_path, verbose=self._verbose), File "/home/.../.venv/lib/python3.8/site-packages/clearml/backend_config/config.py", line 312, in _read_single_file return ConfigFactory.parse_file(file_path) File "/home/.../.venv/lib/python3.8/site-packages/clearml/utilities/pyhocon/config_parser.py", line 100, in parse_file return cls.parse_string(content, os.path.dirname(filename), resolve, unresolved_value) File "/home/.../.venv/lib/python3.8/site-packages/clearml/utilities/pyhocon/config_parser.py", line 153, in parse_string return ConfigParser().parse(content, basedir, resolve, unresolved_value) File "/home/.../.venv/lib/python3.8/site-packages/clearml/utilities/pyhocon/config_parser.py", line 443, in parse has_unresolved = cls.resolve_substitutions(config, allow_unresolved) File "/home/.../.venv/lib/python3.8/site-packages/clearml/utilities/pyhocon/config_parser.py", line 623, in resolve_substitutions is_optional_resolved, resolved_value = cls._resolve_variable(config, substitution) File "/home/.../.venv/lib/python3.8/site-packages/clearml/utilities/pyhocon/config_parser.py", line 470, in _resolve_variable raise ConfigSubstitutionException( clearml.utilities.pyhocon.exceptions.ConfigSubstitutionException: Cannot resolve variable ${CLEARML_ENDPOINT} (line: 2, col: 25)

  
  
Posted 2 years ago

Hey SuccessfulKoala55 ! Is the configuration file needed for Task.running_locally() ? This is tightly related with issue #395, where we need additional files for remote execution but have no way to attach them to the task other then using the StorageManager as a temporary cache.

  
  
Posted 2 years ago

Hmmm Regarding your issue you can use the following env vars to define your endpoint
https://clear.ml/docs/latest/docs/configs/env_vars/#server-connection

What is your usecase? Do you want to change the endpoint mid run?

  
  
Posted 2 years ago

That's what I thought too, it should only look for the CLEARML_TASK_ID environment variable?

  
  
Posted 2 years ago

I see, okay that already clarifies some stuff, I'll dig a bit more into this then! Thanks!

  
  
Posted 2 years ago

Hi UnevenDolphin73 , basically the configuration loading is deferred, but once it's loaded it's not re-loaded. As long as you won't call any ClearML SDK method before setting the env vars, I think you should be fine

  
  
Posted 2 years ago

We load the endpoint (and S3 credentials) from a .env file, so they're not immediately available at the time of from clearml import Task .
It's a convenience thing, rather than exporting many environment variables that are tied together.

  
  
Posted 2 years ago

Setting the endpoint will not be the only thing missing though, so unfortunately that's insufficient 😞

  
  
Posted 2 years ago