Reputation
Badges 1
64 × Eureka!This is actually in runtime (i.e. when running the code),
My script looks like that
` import clearml
clearml.Task.force_requirements_env_freeze(force=False, requirements_file="requirements.txt")
task = clearml.Task.init(...)
task.set_script(working_dir=dir, entry_point="my_script.py")
task.execute_remotely(queue_name='default')
rest of script goes here.... `When you refer to runtime, do you mean when the script is executed remotely, or when I run my_script.py locally (in order to ...
Im afraid there's something with my account, since it won't login on other browser as well, but logging with a different email works
Oh!
That was so silly on my side...
SuccessfulKoala55
Is it a rare use case to use a NAS as the fileserver?
What would you suggest?
Anyway, when I add the binary's path to PATH, it still won't work.
I calltask.set_base_docker(docker_image='my/docker/image', docker_setup_bash_script= ["export PATH=""/workspace/miniconda/bin:$PATH"])
just after Task.init.
On execution, the agent installs all pip requirements with python binary at /usr/bin/python3.6
but eventually the task fails with, /workspace/miniconda/bin/python3: No module named clearml_agent
/mnt/clearml_storage
is the mount point of the NAS on one linux machine.
On macos it would be /Volumes/clearml_storage,
On windows - //NAS/clearml_storage
clearml
numpy
pandas
pycocotools
tqdm
boto3
torch
torchvision
bbox_utils
xmltodict
scikit-video
onnxruntime
flask
SweetBadger76
Itās not a credential issue, because I do upload artifacts manually with tsk.upload_artifact(...)
Iāll try changing the extension, but I have to admit that in the past (I havnāt used clearml for a while and updated it recently to the latest verion) it did got this file extension right
AgitatedDove14 Yes, thats correct.
It's in my local conda environment though.
AgitatedDove14 , here's the log
Something like:
model = SomePytorchModel()
checkpoint = {'model_state_dict': model.state_dict()}
torch.save(checkpoint, āmodel.tarā)
@<1523701070390366208:profile|CostlyOstrich36>
Well, I doubt that it is the case in my situation.
Is there any API where I can get read the metrics per experiment?
SuccessfulKoala55
Were using the community server
can you test what happens if you pass the credentials in the global scope as well, i.e. here:
That didnāt help
Absolute in my hard drive?
Thanks ExasperatedCrab78
AgitatedDove14 - attached
I did found this:
from clearml.backend_api.session.client import APIClient
client = APIClient()
client.events.get_task_metrics(tasks=['some_task_id'], event_type='log')
but it returns an empty 'metrics' object
Thanks @<1523701070390366208:profile|CostlyOstrich36> .
What are considered as experiment objects?
And I'm using the latest clearml / clearml-agent
@<1523701070390366208:profile|CostlyOstrich36>
Thanks @<1523701087100473344:profile|SuccessfulKoala55> @<1523701070390366208:profile|CostlyOstrich36> .
Fixed with the RC.
It didn't work either. Still same error.
conda command was already in PATH and the conda activate
is executed properly, but it prompts to run conda init
(i.e conda wasn't configures at that shell):
` You should consider upgrading via the '/usr/bin/python3 -m pip install --upgrade pip' command.
CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
To initialize your shell, run
$ conda init <SHELL_NAME>
Currently supported shells are:
- ba...
Its mounted automatically.
The local .ssh folder is copied to a temp folder which is mounted by the agent in the docker run command -Executing: ['docker', 'run', '-t', ......., '-v', '/tmp/clearml_agent.ssh.fuu4r8ta:/root/.ssh', .... , '--rm', 'nvidia/cuda:10.1-cudnn7-runtime-ubuntu18.04', ....]
So I run the same script as part of a git repo - but unfortunately the package is still missing.
I'm not sure if it matters but 'kwcoco' is being imported inside one of the repo's functions and not on the script's header.