Hi @<1524560082761682944:profile|MammothParrot39> , how do you usually fetch metadata from a dataset?
Hi 🙂
Is it self deployed?
Hi @<1523703961872240640:profile|CrookedWalrus33> , I think by "mutable" it means that the object itself is mutable when connecting.
I'm curious, what is your use case that you want to change the values in the code itself? The intended usage is to connect the config object and then control it via the webUI / API
Hi CrookedMonkey33 ,
Can you please open developer tools (F12) and see what is returned when you navigate to the 'projects' page (When you see 41 experiments)
Also go into 'Settings' -> 'Configuration' and verify that you have 'Show Hidden Projects' enabled
Do you have a code snippet that reproduces this?
I might be wrong. Did you try 1.9.1?
Hi @<1526371965655322624:profile|NuttyCamel41> , can you add the full log?
Hi @<1523722618576834560:profile|ShaggyElk85> , please see here: None
I think these are the ones you're looking for:
CLEARML_AGENT_SKIP_PYTHON_ENV_INSTALL
CLEARML_AGENT_SKIP_PIP_VENV_INSTALL
Regarding your second question, I think this is possible only when talking about submodules
@<1664079296102141952:profile|DangerousStarfish38> , are you running different python versions on the different machines? Remote vs local
ClearML logs the paths as they are when torch.save()
is run. Your problem currently is getting torch.save()
to work, no?
Hi @<1734020162731905024:profile|RattyBluewhale45> , what version of pytorch are you specifying?
@<1523701977094033408:profile|FriendlyElk26> , try upgrading to the latest version, I think it should be fixed on the latest version
Hi @<1547028031053238272:profile|MassiveGoldfish6> , what version of clearml
& pytorch-lightning
? Does this happen to you with the example as well? Are you on a self deployed or the community server?
Also with delete_external_files=False
I think the serving engine ip depends on how you set it up
StaleButterfly40 , Hi 🙂
What does that option in tensorboard do?
I would look for an AMI that has already cuda and all drivers installed. Same goes for the docker image
Hi @<1671689442621919232:profile|ItchyDuck87> , did you manage to register directly via the SDK?
Not sure what you mean, it looks like your experiment ran for 6 minutes.. Can you add the full log?
I'd suggest using the agent in --docker mode
Hi @<1554638160548335616:profile|AverageSealion33> , so if you remove the Task.init()
everything goes back to working fine?
Hi CluelessFlamingo93 , I see the following traceback in your log:
`
1672308338689 bfm:0 DEBUG ━━━━━━━━━━━━━━╸ 0.7/1.8 GB 5.9 MB/s eta 0:03:16
ERROR: Exception:
Traceback (most recent call last):
File "/home/bat/.clearml/venvs-builds/3.8/lib/python3.8/site-packages/pip/_vendor/urllib3/response.py", line 437, in _error_catcher
yield
File "/home/bat/.clearml/venvs-builds/3.8/lib/python3.8/site-packages/pip/_vendor/urllib3/response.py", line 560, in read
...
Hi @<1523704461418041344:profile|EnormousCormorant39> , is there any chance this could be indeed network related if it does manage to work sometimes?
Can you add a larger portion of the log with errors?
Also what type of machines are these? Linux to linux?
Are you getting some error?
What version of ClearML are you using?
Hi JitteryCoyote63 , you can get around it using the auto_connect_frameworks
parameter in Task.init()
Results -> Scalars 🙂
Hi @<1686547380465307648:profile|StrongSeaturtle89> , as Alon mentioned, you need to connect a worker to the relevant queues (Both for the controller and the steps)