Reputation
Badges 1
662 × Eureka!What's new in 1.1.6rc0?
That doesn't make sense? 🤔
Maybe I was not clear, but it's a simple part of the config file.
I guess it's mixed. If #340 is resolved, then this initializer task will be a no-op: detach, and init-close new tasks as needed.
Yes, thanks AgitatedDove14 ! It's just that the configuration
object passed onwards was a bit confusing.
Is there a planned documentation overhaul? 🤔
I mean, if I search for "model", will it automatically search for tasks containing "model" in their name?
If I set the following:"extra_clearml_conf": "sdk.aws.s3.credentials = [\n{\nhost: 'ip:9000'\nkey: 'xxx'\nsecret: 'xxx'\nmultipart: false\nsecure: false\n},\n{\nhost: 'ip2:9000'\nkey: 'xxx'\nsecret: 'xxx'\nmultipart: false\nsecure: false\n}\n]"
I run into a weird furl
error:ValueError: Invalid port '9000''.
Coming back to this; ClearML prints a lot of error messages in local tests, supposedly because the output streams are not directly available:
` --- Logging error ---
Traceback (most recent call last):
File "/usr/lib/python3.10/logging/init.py", line 1103, in emit
stream.write(msg + self.terminator)
ValueError: I/O operation on closed file.
Call stack:
File "/home/idan/CC/git/ds-platform/.venv/lib/python3.10/site-packages/clearml/task.py", line 3504, in _at_exit
self.__shutdown...
Or is just integrated in the ClearML slack space and for some reason it's showing the clearml address then?
Not sure I understand your comment - why not let the user start with an empty comparison page and add them from "Add Experiment" button as well?
I'm aware, but it would be much cleaner to define them in the worker's clearml.conf
and let ClearML expose them locally to running tasks.
EDIT: Also the above is specifically about serving, which is not the target here 🤔 At least not yet 😄
Any simple ways around this for now? @<1523701070390366208:profile|CostlyOstrich36>
So a missing bit of information that I see I forgot to mention, is that we named our packages as foo-mod
in pyproject.toml
. That hyphen then get’s rewritten as foo_mod.x.y.z-distinfo
.
foo-mod @ git+
You can use logger.report_scalar
and pass a single value.
And task = Task.init(project_name=conf.get("project_name"), ...)
is basically a no-op in remote execution so it does not matter if conf
is empty, right?
Yes that's what I thought, thanks for confirming.
BTW AgitatedDove14 following this discussion I ended up doing the regex way myself to sync these, so our code has something like the following. We abuse the object description here to store the desired file path.
` config_path = task.connect_configuration(configuration=config_path, name=config_fname)
included_files = find_included_files_in_source(config_path)
while included_files:
file_to_include = included_files.pop()
sub_config = task.connect_configuration(
configurat...
Now, the original pyhocon does support include statements as you mentioned - https://github.com/chimpler/pyhocon
It misses the repository information of course, but the 'configuration/Args' were logged. So something weird in identifying the repository
Because setting env vars and ensuring they exist on the remote machine during execution etc is more complicated 😁
There are always ways around, I was just wondering what is the expected flow 🙂
` # test_clearml.py
import pytest
import shutil
import clearml
@pytest.fixture
def clearml_task():
clearml.Task.set_offline_mode(True)
task = clearml.Task.init(project_name="test", task_name="test")
yield task
shutil.rmtree(task.get_offline_mode_folder())
clearml.Task.set_offline_mode(False)
class ClearMLTests:
def test_something(self, clearml_task):
assert True run with
pytest test_clearml.py `
UPDATE: Apparently the quotation type matters for furl
? I switched the '
to \"
and it seems to work now
Since the additional credentials are available to the autoscaler when it boots up (via the config file), I thought it could use those natively?
Does that make sense CostlyOstrich36 ? Any thoughts on how to treat this? For the time being I'm also perfectly happy to include something specific to extra_clearml_conf
, but I'm not sure how to set the sdk.aws.s3.credentials
to be a list of dictionaries as needed
Would be great if it is 😍 We have few files that change frequently and are quite large in size, and it would be quite a storage hit to save all of them
I also tried setting agent.python_binary: "/usr/bin/python3.8"
but it still uses Python 2.7?
TimelyPenguin76 CostlyOstrich36 It seems a lot of manual configurations is required to get the EC2 instances up and running.
Would it not make sense to update the autoscaler (and example script) so that the config.yaml
that's used for the autoscaler service is implicitly copied to the EC2 services, and then any extra_clearml_conf
are used/overwritten?
I'm not sure I follow, how would that solution look like?