Can you share the task log?

Can I ask you to open a PR with this fix? 🙂

I might be wrong about the company_id 🙂 - but in case I'm wrong, you'll see it quickly since you won't be able to login with new users to the UI 🙂

Where exactky?

GiganticMole91 how many experiments are you running concurrently? Are you reporting a lot of metrics/logs in each experiment?

I know, but we really provide the bare minimum since people usually want to try it out and I assume most are price-conscious... I guess we can explain that in the documentation 🙂

OK, that makes more sense, I guess

What's the error on the other machine?

WackyRabbit7 this is most likely a Cookie issue - your browser already had a cookie with an old Token for the previous server, and the UI failed trying to access the server using that Cookie. Clearing the cookies is always a good thing when reinstalling servers.

Cool 🙂 - let me know if there's anything else I can help with 👍

remote execution is working now. Internal worker nodes had not spun up the agent correctly 

So no issues now? 🙂

Can you add --debug ?

Also, what do you mean by another machine? Are you running the ClearML services agent daemon on another machine?

This looks like the webapp can't communicate with the apiserver

Hi UnevenDolphin73 , queues don't have any notion of configuration. Agent running tasks remotely use their own configuration file to execute tasks.

Than yes 😄

All of the data is indeed in ElasticSearch and Mongo

AverageRabbit65 you can see the full process and how to create the configuration file here: https://clear.ml/docs/latest/docs/clearml_agent

and are you sure these are the same env vars available when the agent does the same?

SucculentBeetle7 , this happens since the SDK uses the full URL of the registered image, when you use the fileserver but change its ip, you essentially break the links (this is why we recommend using a hostname, when possible). In order to change the urls to point to the new IP, you'll need to run a update_by_query API call against the ElasticSearch instance used by the server.

what's the clearml SDK version?

theoretically the entire document cannot exceed 16MB, but I'm sure that's not the realistic limit you're talking about

when I try to run it with the agent

When you try to run what, exactly?

Well, you're welcome to ask questions here. I can tell right now that most of what the ClearML server uses (ports, folders etc.) is fully configurable using both config files and environment variables.

For #1, I'll need to check 🙂

If that's of interest to you, I'll appreciate if you open an issue with a feature request in the trains GitHub page 🙂

why not use execute_remotely as well?

Hi JitteryCoyote63 ,
In the docker-compose file, you have an environment setting for the apiserver service host and port ( CLEARML_ELASTIC_SERVICE_HOST and CLEARML_ELASTIC_SERVICE_PORT ) - changing those will allow you to point the server to another ES service

Might be some other issue related to loading plots from elastic. Can you show the trains-apiserver log again after you received the error - there should be some more information there

Stop and re-run the agent