Hi @<1542316991337992192:profile|AverageMoth57> , can you please explain the use-case? I'm not sure I follow

GiganticTurtle0 did you use the project name? The project argument expects a list of project IDs...

Also, how do you restore the database exactly?

Hi @<1535069219354316800:profile|PerplexedRaccoon19> , can you share the result of running this?

python -c "from clearml import Task; print(Task._get_default_session().get_files_server_host())"

Let's try to circumvent the issue by using the environment variables. Try setting AZURE_STORAGE_ACCOUNT and AZURE_STORAGE_KEY (with your account name and account key, respectively) and commenting out the configuration in trains.conf

Well, how to you set up dictConfig?

That seems OK

Oh, my mistake - if you ran into this issue when running clearml-init , the file wouldn't have been created...

Hi ReassuredTiger98 ,
Authentication-wise, all communication with the server is authenticated using a JWT token. In the open version, fileserver is indeed not included in this scheme (however, of you use any other object storage solution such as S3/minio, that's already built-in)

Well, it's a bit tricky, but here's what you'll need to configure. Basically we'd like to see what requests are being sent to the server, and the best way to do that is using the urllib3 log used by Python's requests module. Since this log is "muted" by default (to prevent it from littering the experiment's log), you'll need to add the following in your trains.conf file:
` sdk.log.disable_urllib3_info: false
logging.loggers.urllib3 {
level: DEBUG
handlers: [urllib_co...

https://allegro.ai/docs/deploying_trains/trains_server_linux_mac/
Make sure to follow Deploying , steps 7, 8 and 9

You mean the new instance?

Exactly 😄 - however, since the server uses pyhocon to parse those vars (that's why you can provide an entire structure there, not just text), make sure to provide a value that can actually be parsed as such (i.e. the string the server should receive for each "connection" should be something like
{hosts: [{host: "127.0.0.1", port: 9200}], args { timeout: 60, dead_timeout: 10, max_retries: 3, retry_on_timeout: true}, index_version: "1"}etc, and NOT
` "{hosts: [{host: "127.0.0.1",...

I mean kill a detached subprocess on the machine executing the agent

Oh, I see. Now I understand why you want it on the same remote machine...

Well, any task that is running using the SDK actually always pings the server in the background

🙂

It's installed in the workspace 🙂 (I think 😕 )

Hi GrittyKangaroo27 ,
By default, ClearML publishes the last output model when publishing a task - the reasoning it that this is the latest model and represents the result.
Do you publish you task programmatically (i.e. by calling task.publish() )?

I see. The ClearML module, when loaded, reconfigures the Python logging module - default configuration is in the ClearML logging section of the configuration (see here ). You can try to override parts of it.

btw - if you remove the docker-compose changes, do the containers start normally?

Hi ChubbyLouse32 ,
See here: https://clear.ml/docs/latest/docs/references/api/queues#post-queuesget_all
You can simply do:
client = APIClient() queues = client.queues.get_all(name=queue_name)

We'll take a look 🙂

As far as ClearML system objects are concerned, they are overwritten and older objects are usually replaced. In come cases, they can still exist in the system, but are still associated with the task (so the same thing will apply to them as is done for all objects related to this task when you delete it, for example)

Hi JitteryCoyote63 , are these popups always for the same host/bucket? Any quick way to reproduce this behavior?

@<1523701760676335616:profile|EnviousPanda91> this issue is not related to the ssh port forwarding - it looks like you can get to the server, but the issue is that some of the server components are not starting up properly. From the apiserver log you attached it seems like the apiserver component is not able to connect to the elasticsearch database

Response contains both the raw JSON response as well as an object with properties

It might be the task was stopped/reset from the UI?

JitteryCoyote63 using multiple regions while using env vars is something that's not even supported in the AWS CLI... 🙂
If you want to mix regions, you'll have to use the clearml.conf configuration (as CostlyOstrich36 suggested)

If you ask Bash to run Bash you might get some issues 🙂

I believe so...