Reputation
Badges 1
53 × Eureka!Not sure how to check that tbh. Does this help:root@aea5d96a8ed3:/usr/agent# clearml-agent --version CLEARML-AGENT version 1.0.0Would be nice to display this info maybe somewhere inhere:
Interesting, I don't get newlines in any of my consoles:ClearML Task: overwriting (reusing) task id=38cc10401fcc43cfa432b7ceed7df0cc 2021-10-08 14:57:53,704 - clearml.Task - INFO - No repository found, storing script code instead ClearML results page: `
...
✦2 ❯ git remote showgithub
radu on vm-aimbrain-01 in experiments/runners/all via :snake: v3.8.5 via C volt ❯ grep flush ~/clearml.conf # Carriage return (\r) support. If zero (0) \r treated as \n and flushed to backend # Carriage return flush support in seconds, flush consecutive line feeds (\r) every X (default: 10) seconds console_cr_flush_period: 600
True, the Hyperparameters tab does show the environment (sorry, my bad). The repo information, or the uncommitted changes don't show up for me 😞 . What version did you use?
OK I can confirm that those display ok for me too. My problem is, they don't for my experiments, which is I care about..
you think that simply adding origin will fix this - I really don't mind doing that
I'm been overly optimistic it seems (based on the hints from this issue: https://stackoverflow.com/questions/59455268/how-to-disable-progress-bar-in-pytorch-lightning ). It really looked like it worked. But I was mislead by the fact that the issue has a peculiar pattern to it: it doesn't appear at the beginning of each epoch. There seem to be a couple of extra cr / flushes at the beginning and then a (puzzling) extra "Validation" log line seems to trigger the continuous scrolling. Anyway, my...
Hi again. After looking into the matter a little bit, I realise I'd have liked having the option of using a StoreManager ABC which I would implement myself using whatever storage provider I happen to use and whatever package versions happened to support it. To put it differently, instead of you implementing managers for gcs, azure, aws, etc, it would be a much nicer alternative (for me, and I suspect eventually for you too) for clearml's store manager to wrap whatever object the user pr...
Hi Martin, it is a tqdm parameter (the default ProgressBar in pytorch lightning is unfortunately relying on tqdm). This is from the tqdm docs:dynamic_ncols : bool, optional If set, constantly altersncolsandnrows` to the
environment (allowing for window resizes) [default: False].
nrows : int, optional
The screen height. If specified, hides nested bars outside this
bound. If unspecified, attempts to use environment...
I didn't add that to the script since the effect is persistent (i.e. it only needs to be done once, right?) In any case, I checked that multiple times and it was as expected.
Hi SuccessfulKoala55 ,yes, I am running as sudo. It's not my first time setting this up either. Just that on this vm (which is a particularly large instance) I get this issue which has me stumped.
I forgot to say I've set up a local server - we are still testing phase. I've created credentials for them because they couldn't generate them for themselves (they did clearml-init, and have eacha clearml.conf file but the ADD CRENDENTIALS part didn't show up for them).
Unfortunately it still happens 😞 :
` Epoch 51: 100%|███████████████████████████████████████████████████████████| 361/361 [02:52<00:00, 2.10it/s, loss=0.169, v_num=9-29]
2021-09-17 09:58:22,253 - clearml.Task - INFO - Waiting for repository detection and full package requirement analysis
2021-09-17 10:03:22,254 - clearml.Task - INFO - Repository and package analysis timed out (300.0 sec), giving up
2021-09-17 10:03:22,313 - clearml.Task - WARNING - Failed auto-det...
Sorry, I meant the "origin" part. The warning is no more.
We can't really know (possibly ever 🙂 ), but if the bug happens again I'll be sure to report it here.
OK I won't edit the db 😄 . Thanks for the suggestion, we'll use that!
AgitatedDove14 Yes! That would be exactly what I want (i.e. get_configuration_as_dict .) Alas, no such thing exists in 1.4.1. Is that supposed to come in a next version?
CostlyOstrich36
` {"meta":{"id":"3cceedbbc65d480096ebb02b5aba5902","trx":"3cceedbbc65d480096ebb02b5aba5902","endpoint":{"name":"tasks.get_configurations","requested_version":"2.17","actual_version":"1.0"},"result_code":200,"result_subcode":0,"result_msg":"OK","error_stack":"","error_data":{}},"data":{"configurations"...
Hi Martin, to expand on my previous comments: the template for _Driver already exists; I'm suggesting to make it public. Consequently, StorageHelper should accept a driver parameter to __init__ , defaulting to None . Only when its value is not provided by the user should the library go out of its way to do the right thing and check all the known storage providers, fetch credentials, what not - stuff that will not work for most users, most of the time (even if you ...
Hi AgitatedDove14
this is how our calls look like:
` from pytorch_lightning.loggers import TensorBoardLogger
logger = TensorBoardLogger(save_dir=".", name="debug plotting", 1)
logger.experiment.add_histogram(f"A", data[data.by == 0])
logger.experiment.add_histogram(f"B", data[data.by == 1]) `the result of which is shown in my post above.
This is some test data, and how we'd like things to look:
` def make_data(size: int=10000, n: int=5) -> pd.DataFrame:
x = np.abs(np.random.normal(siz...
Try updating to 1.1.0?
