Also can you right click on the image and save it on your machine, see if it is cropped, or it is just a UI issue
at the end it's just another env var
It should work GIT_SSH_COMMAND is used by pip
Okay here is a standalone code that should be close enough? (if I missed anything let me know)
` import tempfile
from datetime import datetime
from pathlib import Path
import tensorflow as tf
import tensorflow_datasets as tfds
from clearml import Task
task = Task.init(project_name="debug", task_name="test")
(ds_train, ds_test), ds_info = tfds.load(
'mnist',
split=['train', 'test'],
shuffle_files=True,
as_supervised=True,
with_info=True,
)
def normalize_img(image, labe...
You need to mount it to ~/clearml.conf (i.e. /root/clearml.conf)
Hi ShallowCat10
What's the TB your are using?
Is this example working correctly for you?
https://github.com/allegroai/clearml/blob/master/examples/frameworks/tensorflow/tensorboard_pr_curve.py
If it cannot find the Task ID I'm guessing it is trying to connect to the demo server and not your server (i.e. configuration is missing)
I think task.init flag would be great!
👍
One thing though - I am running agent on behalf of a regular user.
Oh that might be credentials / docker service issue (i.e. the user might not have the ability to rn a docker with --gpus, but as you mentioned,, that seems like an arch thing 🙂 )
Hi @<1618056041293942784:profile|GaudySnake67>Task.create is designed to create an External task not from the current running process.Task.init is for creating a Task from your current code, and this is why you have all the auto_connect parameters. Does that make sense ?
@<1535793988726951936:profile|YummyElephant76> oh you mean like jupyter server was running, then inside the notebook you would start a new venv, in that venv "notebook" package was missing, hence it failed detecting the notebook ?
is it normal that it's slower than my device even though the agent is much more powerful than my device? or because it is just a simple code
Could be the agent is not using the GPU for some reason?
Hi @<1547028116780617728:profile|TimelyRabbit96>
You are absolutely correct, we need to allow to override configuration
The code you want to change is here:
None
You can try:
channel = self._ext_grpc.aio.insecure_channel(triton_server_address, options=dict([('grpc.max_send_message_length', 512 * 1024 * 1024), ('grpc.max_receive_message_len...then when we triggered a inference deploy it failed
How would you control it? Is it based on a Task ? like a property "match python version" ?
I suppose the same would need to be done for any
client
PC running
clearml
such that you are submitting dataset upload jobs?
Correct
That is, the dataset is perhaps local to my laptop, or on a development VM that is not in the
clearml
system, but I from there I want to submit a copy of a dataset, then I would need to configure the storage section in the same way as well?
Correct
I think you cannot change it for a running process, do you want me to check for you if this can be done ?
Hi GiganticTurtle0
I have found that
clearml
does not automatically detect the imports specified within the function decorated
The pipeline decorator will automatically detect the imports Inside the funciton, but not outside (i.e. global), to allow better control of packages (think for example one step needs the huge torch package, and the other does not.
Make sense ?
How can I tell
clearml
I will use the same virtual environment in all steps...
Could it be you have some custom SSL certificate installed, or policy ?
can you get other https sites? (for example your clearml-server)
agent.package_manager.system_site_packages
can be used to inherit packages
Correct, it is basically venv with --system-site-packages
I do not think virtualenv nesting is support, if it was then in theory you could have executed the clearml-agent from virtual environment with system_site_packages turned on and then it would inherit from it. But again I'm not sure virtualenv supports it.
BTW: the latest clearml-agent RC already have venv caching (both pip/conda) 🙂
When a remote task runs
Dataset.get()
it is not using the correct URL
BoredHedgehog47 it will get the link the data was Registered with, when creating the Dataset.
This has Nothing to do with the local configuration, it can point to any arbitrary file location on the internet.
It was created there, because at the time of the dataset creation someone (manually or via the config) set a specific host as the file location, and to that host the files were uploaded (again ...
SmugDog62 so on plain vanilla Jupyter/lab everything seems to work.
What do you think is different in your setup ?
You could change infrastructure or hosting, and now your data is associated with the wrong URL
Yeah that makes sense, so have it on a specific dns name? (this is usually the case with k8s deployments)
Just curious, if
is a value I can set, where is it used?
It is used when Creating a dataset from inside the cluster (i.e. when launching using the clearml k8s glue),
it will have No effect on what users have on their local machines
i.e. they can always point to a diff server.
That said, when users create their initial clearml.conf and copy paste the info from the web UI, this value (or it might be another one, I'll double check later) will set the initial configuration the c...
Are you running the agent in docker mode or venv mode?
Thanks for the ping ConvolutedChicken69 , I missed it 😞
from what i see in the docs it's only for Jupyter / VS Code, i didn't see anything about pycharm
PyCharm is basically SSH, which is supported 🙂
(Maybe we should mention it in the docs?)
Is there a way I could move the JWT authentication (not authorization) logic into an API Gateway or Load Balancer?
Hmm in theory, but not in practice 😞
if ClearML is following OAuth 2.0, t
This is for the SSO part, not for the API, API is only using JWT for verification, the login process itself is with external SSO (OAuth 2.0). But the open-source version does not support SSO 😞
Why are you trying to add another ELB with JWT verification on it ? ...