Reputation
Badges 1
662 × Eureka!We just inherit from logging.Handler
and use that in our logging.config.dictConfig
; weird thing is that it still logs most of the tasks, just not the last one?
Still failing with 1.2.0rc3 π AgitatedDove14 any thoughts on your end?
Or well, because it's not geared for tests, I'm just encountering weird shit. Just calling task.close()
takes a long time
Ah! Makes sense. Thanks!
They are set with a .env
file - it's a common practice. The .env
file is, at the moment, uploaded to a temporary cache (if you remember the discussion regarding the StorageManager
), so it's also available remotely (related to issue #395)
Just a side note - the 1.1.1 notice keeps popping up even though the server is at 1.1.1 (and I've cleared browser cache etc)
Sorry, been away for a while!
I have no additional information, since it was a bug in my model that I have since eliminated...
Maybe it was just a matplotlib error and can be dropped for now. I'll let you know if it pops up again!
As the meme goes, well yes but actually no, since the input path is provided via argparse? I'm also not sure how this would help debug from the WebUI - you can't really see the contents of a zipped file/the configuration tab is too messy for such a nested configuration as the one we have. It's best suited as an artifact.
EDIT: Or am I missing something? Point being, when the remote execution begins, the entry point tries to run e.g. python train.py --config_file path/to/local/file.yaml
...
Uhhh, but pyproject.toml does not necessarily entail poetry... It's a new Python standard
Why not give ClearML read-only access credentials to the repository?
When is the next release expected? π
The odd thing is that it was already defined, and then when I clicked an S3 link, it asked me to fill it in again, adding a duplicate credentials row
I am indeed
Hm, I'm not sure I follow π€ How does the API server config relate to the file server?
Sorry to keep this up - what about support for minio using the environment variable? Do I set the CLEARML_FILES_HOST
to the end point instead of an s3 bucket?
I'm running tests with pytest
, it consumes/owns the stream
Hmmm maybe π€ I thought that was expected behavior from poetry side actually
Of course now it's not there anymore π If/when it happens again I'll ping you here π
It's also sufficient to see StorageManager.list("/data/clear")
takes a really long time to return no results
Ah it already exists https://github.com/allegroai/clearml-server/issues/134 , so I commented on it
A follow up question (instead of opening a new thread), is there a way I could signal some files/directories to be copied to the execute_remotely
task?
Ah, you meant βfree python codeβ in that sense. Sure, I see that. The repo arguments also exist for functions though.
Sorry for hijacking your thread @<1523704157695905792:profile|VivaciousBadger56>
Does it make sense to you to run several such glue instances, to manage multiple resource requirements?
Latest (1.5.1 I believe?), full log incoming, but it's like I've posted elsewhere already π€
It just sets up the environment and immediately crashes when trying to run the code.
The setup itself is done correctly.
Perfect now π (also nice cleanup of default_new_data_root
duplicate code :D)