Hi @<1639799308809146368:profile|TritePigeon86> , I think the 1.16 refers to the version of the SDK. I'd suggest upgrading your server regardless 🙂
Hi @<1654294828365647872:profile|GorgeousShrimp11> , it appears the issue is due to running with different python versions. It looks like the python you're running the agent on, doesn't have virtualenv
Are you using the community server or did you deploy yourself?
The one sitting in the repository
Hi @<1702492411105644544:profile|YummyGrasshopper29> , console logs are saved in Elastic. I would check on the status of your container
Hi @<1727859576625172480:profile|DeliciousArcticwolf54> , I'd suggest debugging using developer tools in the webUI. Also, are you seeing any errors in the API server or webserver containers? I'd suggest first testing with elasticsearch to make sure that the deployment went through OK and this is not related to something else.
Out of curiosity, why do you want to use opensearch instead of elasticsearch?
containing the correct on-premises s3 settings
Do you mean like an example for minio?
If it's deployed by you, then try running clearml-init
from the same machine the server is on. Doesn't matter if it's a cloud machine really
Please implement in python the following command curl <HOST_ADDRESS>/v2.14/debug/ping
Hi @<1577468638728818688:profile|DelightfulArcticwolf22> , I think this is what you're looking for - CLEARML_AGENT_SKIP_PIP_VENV_INSTALL
CLEARML_AGENT_SKIP_PYTHON_ENV_INSTALL
CLEARML_AGENT_FORCE_SYSTEM_SITE_PACKAGES
None
Look over the different env variables to see what fits your specific needs
Hi @<1585078763312386048:profile|ArrogantButterfly10> , does the controller stay indefinitely in the running state?
If you remove any reference of ClearML from the code on that machine, does it still hang?
What version of clearml
, clearml-agent
& server are you using?
It broke the shift holding to select multiple experiments btw
Oh! Can you please open an issue in github for tracking please?
Hi @<1597762318140182528:profile|EnchantingPenguin77> , you can set this in the docker extra arguments section of the task
Hi GorgeousMole24 , I think for this your best option would be using the API to extract this information.
` from clearml.backend_api.session.client import APIClient
client = APIClient() `is the pythonic usage
JitteryCoyote63 , are you on a self hosted server? It seems that the issue was solved for 3.8 release and I think should be released to the next self hosted release
Hi FancyTurkey50 , how did you run the agent command?
@<1526734383564722176:profile|BoredBat47> , that could indeed be an issue. If the server is still running things could be written in the databases, creating conflicts
Can you hit F12 on the browser and see what happens in the network area when you're trying to delete?
Hi MagnificentWorm7 , what version of ClearML server are you running?
DeliciousSeal67 , you need to update the docker image in the container section - like here:
Hi RattyLouse61 , what do you see in the log of the run?
I think that by default debug samples are usually saved on the fileserver. The following configuration should force the debug samples to upload to s3.
In clearml.conf
change:api.files_server:
s3://your_bucket
I think then this is the section you're looking for:
https://github.com/allegroai/clearml/blob/master/docs/clearml-task.md#tutorial
ScaryBluewhale66 ,
If you want to re-run - you need the agent It's still a Task
object so you can just use Task.close()
I'm not sure if something exists at the moment but you could write it faily easily in code
ScaryBluewhale66 , Hi 🙂
Regarding your questions
I think you can just reset the task and enqueue it You can stop it either in the UI or programmatically I'm guessing the scheduler would show errors in it's log if for some reason it failed at something for some reason
https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server_mongo44_migration
Looks like what you might need 🙂