Reputation
Badges 1
40 × Eureka!To your point of view, it may be related with the sdk client that triggers the upload ? with urllib request ?
only a "upload failed" and no data in my S3 bucket
He ask me for credentials for the root server (minio.10.68.0.250.nip) and not for the bucket where it is stored (minio.10.68.0.250.nip/simclr) only this bucket has read/write permission.
i use a proxy and the port is 80, i need to write it ?
My configuration.json
is { "fileBaseUrl": " http://file.10.68.0.250.nip.io "}, but HTTP.FILE_BASE_URL still remains undefined
. Something is missing ?
No particular information in console( no error), no network error too.
sudo docker logs clearml-fileserver
This gives no info at all. May be i should increase the log level to debug. The only message i got is about "werkzeug" the default server module of flash that shouldn't be use as production deployement (by the way, why not use gunicorn as entrypoint in docker-compose ?)
DeterminedCrab71 You right, if i understand correctly HTTP.FILE_BASE_URL is undefined, then file to delete is describe as "misc" instead of "fc" then i guess system is unable to launch the delete of the file
As far i know, a server get a SIG_PIPE event on a socket when a client died too soon or is closed by user, but i don't know who get the broken pipe ? Is it the clearml file server ( who masters the upload, i guess) ? Is it due to my minio server ? Who is the client that died before upload is finished ?
Yes, i even got a "upload finished" message et the whole process goes to end.
I tried to modify all docker_internal_mounts point but the mount point for clearm.conf file still remains the same. May be it is defined on server side ?
I think i have my answer, this is hard coded in agent base_cmd += ( (['--name', name] if name else []) + ['-v', conf_file+':'+DOCKER_ROOT_CONF_FILE] + (['-v', host_ssh_cache+':'+mount_ssh] if host_ssh_cache else []) + ...
I saw this part of the configuration file but i don't know exactly which key is used as mount point for configuration file ?
Hello. I think an Issue should at least be opened. Modifications in my code need to be generalized before creating a pull request.
I had to set another docker volume because venvs_builds was not properly mounted to. I don't have write permission to the default one too
The mount point for the clearml.conf, i.e. '-v', '/tmp/.clearml_agent.qy2xyt21.cfg:/root/clearml.conf'
becauce the docker i use is run as user
and doesn't have acces to \root.
. May be this is defined on cleaml-server side ? I use my own server installed on another linux box using docker-compose.
Yes, i was expecting that it was already working like that. So far, i modified the code to set DOCKER_ROOT_CONF_FILE to what i want !!!
In docker-compose, image was latest allegroai/clearml:latest when i pull docker images. When i launch it, after installation i have in WebApp following informations : "WebApp: 1.3.1-169 • Server: 1.3.1-169 • API: 2.17"
The addresses seems strange, is this the hostname?
I use the nip services to have subdomains: clearml.domain api.domain and file.domain that points to the same host.
I was unable to define FILE_BASE_URL inside the docker container. I modify the HTTP constant in app.constants.ts with hard code values, compile the webapp again (npm) and replace it in my docker container and now it works....
I take a look a src/app/webapp-common/shared/entity-page/entity-delete/base-delete-dialog.effects.ts.
I see that an error is raised in the mergeMap at line 125, but i'm not familiar enough with Typescript to find why.
It seems that i should define this variable by the use of an environment variable in ConfigurationService.globalEnvironment.
By default, i put nothing in the task but when i use a ClearMLSaver like thisClearMLSaver(logger, output_uri="
")
where clearml is my bucket.
Files are stored on the same box where the docker is running. And there is a mounting point between file server docker and the host itself.
i tried so far but it was not so easy, because there is a python executable "update_from_env" that empties the configuration.json file. So i create a file in /mnt/external_files/configs and my configuration.json was read.
My artifacts are now deleted but the directories where the artifacts are stored are not deleted.
Most of the time it is due to a bad parsing of the ip adress. You need to be sure the ip adress is correctly parsed and for this, i need to specify the port used for my minio server even it is a standard http (80) port. Then 'address:80' works but not "address" alone
Ok, it works i need to specify :80 also in the output_uri from my python file !
No, no "delete" request 😞 Is there a way to put a breakpoint just before ?
When i deleted the Experiment, i obtain a the following window :