I built an basic nginx container
` FROM nginx

COPY ./default.conf /etc/nginx/conf.d/default.conf

COPY ./includes/ /etc/nginx/includes/

COPY ./ssl/ /etc/ssl/certs/nginx/ copied the signed certificates and the modified nginx deafult.conf `

the important part is to modify the compose file to redirect all traffic to nginx container
` reverse:
container_name: reverse
image: reverse_nginx
restart: unless-stopped
depends_on:
- apiserver
- webserver
- fil...

@<1523701070390366208:profile|CostlyOstrich36> sorry for not being clear enough
when is next version of clearml-server will be released? I can see last version is from August, is there any ETA for new release in upcoming 1-2 month?

I manage clearml-server for many users
wrote a script that pulls tasks (for review - pending deletion) and limit for specific user (saw the REST can limit only by user id )
so user will need to provide the script his own user id in advance
I guess my options are:

• maintain stastic list of users (which I get from users.get_all endpoint)
• add to my script another query that will match user name to user id, before running the tasks_get.all for specific user id

yep, that's options number 2 🙂
I will implement that

when running in debug and watch the values I get

    data = response.json()
    projects = data['data']['projects']
    all_data.extend(projects)

in each loop iteration
projects are same 500 values
all_data gets append for same 500 values in endless loop

I have bug in my code and can't find where just yet

the way I do pagination is wrong
@<1523701070390366208:profile|CostlyOstrich36> might throw some champions tip over here 🙂

I know the 500 limit and using it

but my while loop keeps pulling the same 500 ... and running endless

ohhh severe error here 🙂
I was mixed between other API I worked on .. and did not read carefully the right flag

simple adding page to the body did the work
thanks again @<1724235687256920064:profile|LonelyFly9>

I think this is the right approach, let me have a deeper look
thanks @<1724235687256920064:profile|LonelyFly9>

trying to use projects.get_all to pull all my projects into single file
and there are more then 500 ...

hmmm that's not it, same endless loop when using body

      {
        "id": "c2e33466fd9de1a0d2be8d803ad4fbed",
        "company": "d1bd92a3b039400cbafc60a7a5b1e52b",
        "name": "John Doe",
        "family_name": "John",
        "given_name": "Doe",
        "created": "2024-09-24T06:13:59.956000+00:00"
      }

this is how the object looks like

correct, but!
I wrote a script that pulls tasks and limit for user
so I'm looking for users to knows their own id in advance

yep that was my approached with no luck so far
hopefully someone from the ClearML dev team can give their input on this

for some reason it's not in REST API docs, but I used
users.get_all

how do you know the ID?
I'm looking for this ...

offset = 0
limit = 500
all_data = []

while True:
    params = {
        'offset': offset,
        'limit': limit
    }
    
    response = requests.get(url, headers=headers, params=params,verify=False)
    
    data = response.json()
    projects = data['data']['projects']

    print(f"pulled {len(projects)} projects.")

    if len(projects) == 0:
        print("no project found - exiting ...")
        break    
    
    all_data.extend(projects)
    offset += limit

print(f"pulled {le...

I had slightly similar scenario ~1 year and few versions back
there was some task that wrote a lot of tasks and mongo didn't took it nicely

I was able to identify to it only by questioning users and eventunaly one of them stopped to send and mongo started to come back and all return to normal
we did not come to any wise conclusion what is root cause or how to identify this

not sure it's same use case but I will begin to ask around people
if you have any other hint/way how to query mongo and look for potential culprit - will be glad to hear

I'd guess mongo is choking, not sure why

I think there are some experiments that are messing up mongodb
this logs unusual in clearml-mongo logs:

{"t":{"$date":"2023-09-19T12:15:50.685+00:00"},"s":"I",  "c":"COMMAND",  "id":51803,   "ctx":"conn73","msg":"Slow query","attr":{"type":"command","ns":"backend.model","command":{"distinct":"model","key":"project","query":{"$and":[{"$or":[{"company":{"$in":["d1bd92a3b039400cbafc60a7a5b1e52b",null,""]}},{"company":{"$exists":false}}]},{"user":{"$in":["197aea8467d3f471fc0db98b57ed80fa"]...

app.component.ts:138 ERROR TypeError: Cannot read properties of null (reading 'id')
    at projects.effects.ts:60:56
    at o.subscribe.a (switchMap.js:14:23)
    at p._next (OperatorSubscriber.js:13:21)
    at p.next (Subscriber.js:31:18)
    at withLatestFrom.js:26:28
    at p._next (OperatorSubscriber.js:13:21)
    at p.next (Subscriber.js:31:18)
    at filter.js:6:128
    at p._next (OperatorSubscriber.js:13:21)
    at p.next (Subscriber.js:31:18)

and I see also when trying to...

api calls behaves much better
no problem to query tasks in other projects

tried with my user and edited existing user record in apiserver.conf
it looks ClearML treated this as new user - I did not saw any of the jobs belongs to my user before the change

VivaciousPenguin66 your docs was helpful, I got SSL running but my question remained
have you kept needed http services accessible and only running the authentication via https?
api_server: "http://<my-clearml-server>:8008" web_server: " " files_server: "http://<my-clearml-server:8081"my current state is that the webserver is accessible via http and https, in 8080 & 443

you are correct and thank you for the reply @<1523701070390366208:profile|CostlyOstrich36>
going forward, I assume the clearml-server open-source releases will be continue to be released in Docker Hub

Hi VivaciousPenguin66
thanks for sharing, giving it a try now

after you set up webserver to point to 443 with HTTPS, what have you done with rest of http services clearml is using?

does weberver with 8080 remained accessible and your are directing to it in your ~clearml.conf ?
what about apiserver and file server? (8008 & 8081)

tried it from single workstation, but I get same unexpected behavior
same project/task names, same workstation running the job, anything else I should check to confirm those are identical jobs?
every new identical job start with ClearML Task: created new task id=...

same basic job not gets overwritten, but created new one every time

SuccessfulKoala55 any clue?