StaleButterfly40 , I'm trying to get an estimate of what you have because if the content is too large the preview isn't shown....
Reproduces for me as well. Taking a look what can be done 🙂
Do you mean reporting scalars with tensorflow OR having the reported tensorflow scalars show up on ClearML?
Yeah, but how are iterations marked in the script?
VexedCat68 , what if you simply add pip.stop() ? Does it not stop the pipeline? Can you maybe add a print to verify that during the run the value is indeed -1? Also looking from your code it looks like you're comparing the 'merged_dataset_id' to -1
Hi @<1523704207914307584:profile|ObedientToad56> , the virtual env is constructed using the detected packages when run locally. You can certainly override that. For example use Task.add_requirements - None
There are also a few additional configurations in the agent section of clearml.conf I would suggest going over
Did you download the same data directly to the NAS and it took 5 secs?
Hi IrritableJellyfish76 , it looks like you need to create the services queue in the system. You can do it directly through the UI by going to Workers & Queues -> Queues -> New Queue
Hi @<1673501397007470592:profile|RelievedDuck3> , you simply need to integrate clearml into your code.
from clearml import Task
task = Task.init(...)
More info here:
None
Hi @<1638712150060961792:profile|SilkyCrocodile89> , how did you upload them and as what?
Can you give an example of a pipeline to play with?
Are you running self deployed?
I'm not sure. Maybe @<1523703436166565888:profile|DeterminedCrab71> might have some input
VexedCat68 Hi 🙂
Please try with pip install clearml==1.1.4rc0
Can you gain access to the apiserver logs?
If you set Task.init(..., output_uri=<PATH_TO_ARTIFACT_STORAGE>) everything will be uploaded to your artifact storage automatically.
Regarding models. I to skip the joblib dump hack you can simply connect the models manually to the task with this method:
https://clear.ml/docs/latest/docs/references/sdk/model_outputmodel#connect
Regarding this one, there is actually a way. If you work on http://app.clear.ml you can share an experiment for other users to see. However, to see the experiment people getting the link would need to sign up. This could also be a pretty cool feature request to make it completely public and open. Maybe open another feature request.
JitteryCoyote63 , doesn't seem to happen to me. I'll try raising a clean server and see if this happens then. You're running with 1.2, correct?
(the API keys are exposed through environment variables)
Where are the env variables pointing? I'm interested in all CLEARML related env vars if you could add them here 🙂
The communication is done via HTTPS so relevant ports should be open.
Did you try with a hotspot connection from your phone?
Are you using the PRO or a self hosted server?
I played a bit with it and got to the value. OutrageousSheep60 , please tell me if this helps you 🙂
` >>> task.set_user_properties(x=5)
True
y=task.get_user_properties()
y
{'x': {'section': 'properties', 'name': 'x', 'value': '5'}}
y["x"]["value"]
'5' `
GrittyCormorant73 , K8s deployment will have easier time to spin up agent instances to run the tasks 🙂
Looks like you're having issues connecting to the server through the SDK. Are you able to access the webUI? Is it a self hosted server?
The highlighted line is exactly that. Instead of client.tasks.get_all() I think it would be along the lines of client.debug.ping()
It means that there is an issue with the drivers. I suggest trying this docker image - nvcr.io/nvidia/pytorch:23.04-py3
I'm not quite sure, I'll need to double check 🙂
CheerfulGorilla72 , can you point me to where in the script the reported scalars are?
I think this might be happening because you can't report None for Logger.report_scalar() so the auto logging assigns it some sort of value - 0. What is your use case? If the value of the scalar is None then why log it?
Please try like this model.tags=['Test'] and not with append