Hi @<1575656665519230976:profile|SkinnyBat30>
Streamlit apps are backend run (i.e. the python code drives the actual web app)
This means running your Tasks code and exposing the web app (i.e. http) streamlit.
This is fully supported with ClearML, but unfortunately only in the paid tiers 😞
You can however run your Task with an agent, make sure the agent's machine is accessible and report the full IP+URL as a hyper-parameter or property, and then use that to access your streaml...

To clarify, there might be cases where we get helm chart /k8s manifests to deploy a inference services. A black box to us.

I see, in that event, yes you could use clearml queues to do that, as long as you have the credentials the "Task" is basically just a deployment helm task.
You could also have a monitoring code there so that the same Task is pure logic, spinning the helm chart, monitoring the usage, and when it's done taking it down

It seems that I solved the problem by moving all of the local code (local repos) imports to after the Task.init

PunyPigeon71 I'm confused, how did that solve the issue on the remote machine?

My bad, there is a mixture in terms.
"configuration object" is just a dictionary (or plain text) stored on the Task itself.
It has no file representation (well you could get it dumped to a file, but it is actually stored a s a blob of text on the Task itself, at the backend side)

Are Kwargs supported in functions decorated as a pipeline component?

They are, but I think the main issue is the casting, without prior knowledge, everything will be a tring

Hmm so the SaaS service ? and when you delete (not archive) a Task it does not ask for S3 credentials when you select delete artifacts ?

Bad news, there isn't a nice interface to get the table from the Optimizer object (I will make sure we add it, no reason not to).
But you can very easily get all the information you need and more:
all_the_tasks = an_optimizer.get_top_experiments(top_k=100)Then for every task in the list you can get All the information:
for task in all_the_tasks: task_params_as_dict = task.get_parameters() task_scalars = task.get_last_scalar_metrics()Basically the Task object enables you to que...

This seems more complicated that I thought... I think you are correct, and it fails to load the entire module, let me check what I can do

And thanks again, I really appreciate testing it!

with remote machine where the code actually runs (you know this pycharm pro remote).

Are you using the pycharm plugin ? (to sync the local git changes with clearml)
https://github.com/allegroai/clearml-pycharm-plugin

can I add user properties to a scheduler configuration?

please expand, what do you mean by user property and how one would use it?

Are you sure trains-server not trains package (i.e. backend)

I think what you are looking for is clearml-agent daemon
https://clear.ml/docs/latest/docs/clearml_agent
https://clear.ml/docs/latest/docs/getting_started/video_tutorials/agent_remote_execution_and_automation

Yes 🙂 documentation is being worked on ... Anyhow we will be uploading a new documentation site soon (hopefully in a week or so), putting it all on GitHub so it will be easier for the community to edit and add more

Makes sense
we need to figure what would be the easiest way to have an "opt-in" for the demo server, that will still make it a breeze to quickly test code integration ...
Any suggestions are welcomed 🙂

maybe it can help 🙂

Could you try to clone the clearml git repo, create a new notebook in it and test ?

task.project is the project ID (not name)
task.get_project_name() will return the project name

Hi PerplexedCow66

I would like to know how to serve a model, even if I do not use any serving engine

What do you mean no serving engine, i.e. custom code?

Besides that, how can I manage authorization between multiple endpoints?

Are you referring to limiting access to all the endpoints?

How can I manage API keys to control who can access my endpoints?

Just to be clear, accessing the endpoints has nothing to do with the clearml-server credentials, so are you asking how to...

Check the examples on the github page, I think this is what you are looking for 🙂
https://github.com/allegroai/trains-agent#running-the-trains-agent

We should probably make sure it is properly stated in the documentation...

Hmm... That's what happens with the exception of None/'' if type is str... There is no way to differentiate in the UI.
This is why we opted for type=str will "cast" everything to str so you always get str, while not specifying a type will leave the variable as is... If you have an idea on how to support both, feel free to suggest 🙂

Hi FiercePenguin76

Here’s my workaround - ignore the fail messages, and manually create an SSH connection to the server with Jupyter port forwarded.

You are correct, clearml-session assumes it can SSH into the remote agent machine, from that point it automatically tunnels all other connections on top of the original SSH (well with some fancy SSH keep-alive proxy).
I'm assuming that from home you cannot connect to the SSH machine at the office, which makes sense, but out of curiosity...

Based on the log you have shared:
OSError: [Errno 28] No space left on deviceI would increase the storage ?
https://github.community/t/github-actions-failing-with-errno-28-no-space-left-on-device/18164/10
https://stackoverflow.com/questions/70175977/multiprocessing-no-space-left-on-device
https://groups.google.com/g/ansible-project/c/4U6MyvyvthQ
I would start by increasing the size of the TMPDIR folder

Hi NastyFox63 yes I think the problem was found (actually backend side).
It will be solved in the upcoming release (due after this weekend 🙂 )

Hi @<1619867971730018304:profile|WhimsicalGorilla67>
No 😞 only the "admin" (owner) of the workspace has access to it

I changed them to the one exposed to the users (the same I have in my local clearml.conf) and things work.

Nice!

But I can't really figure out why that would be the case...

So the thing is, the link to the files are generated by the clients, which means the actual code generated a link an internal link to the file server (i.e. a link that only works inside the k8s cluster). When you wanted to see the image/plot you were accessing it from outside the cluster, and the link simply ...

• This then looks for a module called foo , even though it’s just a namespaceI think this is the issue, are you using python package name spaces ?
  (this is a PEP feature that is really rarely used, and I have seen break too many times)
  Assuming you have from from foo.mod import what are you seeing in pip freeze ? I'd like to see if we can fix this, and better support namespaces

The log is missing, but the Kedro logger is print to sys.stdout in my local terminal.

I think the issue night be it starts a new subprocess, and that subprocess is not "patched" to capture the console output.
That said if an agent is running the entire pipeline, then everything is logged from the outside, so whatever is written to stdout/stderr is captured.

how can I start up the clearml agent using the clearml-agent image instead of SDK?

Not sure I follow, what do you mean instead of the SDK? and what is the "clearml-agent image" ?