Hello, I am first timer in ClearML and try to deploy locally a Clear ML server (successfully) and then agent in my Kubernetes cluster. I follow the helm chart from "helm repo add clearml None " and in the helm chart values for agent I changed the below parameters:
agentk8sglueKey: <API KEY>
agentk8sglueSecret: <ACCESS KEY>
-- Reference to Api server url
apiServerUrlReference: " None "
-- Reference to File server url
fileServerUrlReference: " None "
-- Reference to Web server url
webServerUrlReference: " None "
the rest all stay with default values
The pod is running and then goes into restart mode and CrashLoopBack mode
ubuntu@vm4v9lm3:~$ kubectl get pods
NAME READY STATUS RESTARTS AGE
clearml-agent-7c6d58c497-xk8hn 0/1 CrashLoopBackOff 9 (3m11s ago) 25m
clearml-apiserver-57d4f9776d-pgn6q 1/1 Running 0 7h58m
clearml-apiserver-asyncdelete-59484594b9-zdm4p 1/1 Running 0 7h58m
clearml-elastic-master-0 1/1 Running 0 7h58m
clearml-fileserver-769d646d7-tzpg6 1/1 Running 0 7h58m
clearml-mongodb-5f995fbb5-mgwbt 1/1 Running 0 7h58m
clearml-redis-master-0 1/1 Running 0 7h58m
clearml-webserver-7df664dcbf-856f9 1/1 Running 0 7h58m
jupyter-notebook-84c6f6fcf9-4lrrv 1/1 Running 0 38m
The logs are below. Any idea what is wrong?
Any other value to update in helm chart for agent?
/root/entrypoint.sh: line 29: /root/clearml.conf: Read-only file system
- echo 'api.api_server: None '
/root/entrypoint.sh: line 30: /root/clearml.conf: Read-only file system - echo 'api.web_server: None '
/root/entrypoint.sh: line 31: /root/clearml.conf: Read-only file system - echo 'api.files_server: None '
/root/entrypoint.sh: line 32: /root/clearml.conf: Read-only file system - ./provider_entrypoint.sh
- source /root/.bashrc
++ '[' -z '' ']'
++ return - export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/root/bin:/root/bin
- PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/root/bin:/root/bin
- [[ -z '' ]]
- python3 k8s_glue_example.py --queue default --namespace default --template-yaml /root/template/template.yaml
/usr/local/lib/python3.6/dist-packages/clearml_agent/_vendor/jwt/utils.py:7: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release.
from cryptography.hazmat.primitives.asymmetric.ec import EllipticCurve
Traceback (most recent call last):
File "k8s_glue_example.py", line 8, in <module>
from clearml_agent.glue.k8s import K8sIntegration
File "/usr/local/lib/python3.6/dist-packages/clearml_agent/glue/k8s.py", line 19, in <module>
from clearml_agent.commands.events import Events
File "/usr/local/lib/python3.6/dist-packages/clearml_agent/commands/init.py", line 3, in <module>
from .worker import Worker
File "/usr/local/lib/python3.6/dist-packages/clearml_agent/commands/worker.py", line 47, in <module>
from clearml_agent.commands.base import resolve_names, ServiceCommandSection
File "/usr/local/lib/python3.6/dist-packages/clearml_agent/commands/base.py", line 20, in <module>
from clearml_agent.interface.base import ObjectID
File "/usr/local/lib/python3.6/dist-packages/clearml_agent/interface/init.py", line 7, in <module>
from .base import Parser, base_arguments, add_service, OnlyPluralChoicesHelpFormatter
File "/usr/local/lib/python3.6/dist-packages/clearml_agent/interface/base.py", line 12, in <module>
from clearml_agent.session import Session
File "/usr/local/lib/python3.6/dist-packages/clearml_agent/session.py", line 23, in <module>
from clearml_agent.helper.docker_args import DockerArgsSanitizer, sanitize_urls
File "/usr/local/lib/python3.6/dist-packages/clearml_agent/helper/docker_args.py", line 279, in <module>
class CustomTemplate(Template):
File "/usr/lib/python3.6/string.py", line 74, in init
cls.pattern = _re.compile(pattern, cls.flags | _re.VERBOSE)
File "/usr/lib/python3.6/re.py", line 233, in compile
return _compile(pattern, flags)
File "/usr/lib/python3.6/re.py", line 301, in _compile
p = sre_compile.compile(pattern, flags)
File "/usr/lib/python3.6/sre_compile.py", line 562, in compile
p = sre_parse.parse(p, flags)
File "/usr/lib/python3.6/sre_parse.py", line 855, in parse
p = _parse_sub(source, pattern, flags & SRE_FLAG_VERBOSE, 0)
File "/usr/lib/python3.6/sre_parse.py", line 416, in _parse_sub
not nested and not items))
File "/usr/lib/python3.6/sre_parse.py", line 765, in _parse
p = _parse_sub(source, state, sub_verbose, nested + 1)
File "/usr/lib/python3.6/sre_parse.py", line 416, in _parse_sub
not nested and not items))
File "/usr/lib/python3.6/sre_parse.py", line 765, in _parse
p = _parse_sub(source, state, sub_verbose, nested + 1)
File "/usr/lib/python3.6/sre_parse.py", line 416, in _parse_sub
not nested and not items))
File "/usr/lib/python3.6/sre_parse.py", line 734, in _parse
flags = _parse_flags(source, state, char)
File "/usr/lib/python3.6/sre_parse.py", line 803, in _parse_flags
raise source.error("bad inline flags: cannot turn on global flag", 1)
sre_constants.error: bad inline flags: cannot turn on global flag at position 92 (line 4, column 20)