I believe that's correct, yes

I am guessing it could be but.. I don't feel that k8s is clearml-session's main focus/push

that's pretty darned awesome!! I didn't know we could do that 😄 😄

howdy Tim, I have tried to stay out of this, because a lot is going over my head (I am not a smart man 🙂 but, one thing I wanted to ask, are you doing the swapping in and out of code to do a/b testing with your models ?! Is this the reason for doing this ? Because if so, I would be vastly more inclined to try and think of a good way to do that. Again, this maybe wrong, I am trying to understand the use case for swapping in and out code. 🙂

I have a strange theory, that if the code is in classes, then you could include both in one .py file and then ENV["use_model"]="a" or ENV["use_model"]="b" to select between them .. in that way, you would clone the experiment and change the config and re-run

Never a problem Tim.. although it does prompt me to try and figure out a/b model testing myself ... I see everything as a "potential blog post" 😄 😄

Stupid question Tim (and I understand that maybe your code is under NDA etc but) can you show the python code that you need to a/b against ?

since this is an enterprise machine, and you don't have sudo/root, I am wondering if there is already other docker networks/composer setups running/in use

so I am not entirely sure what else you have changed Sir

👍

Hey Slava, I don't mean to be "that guy" but, I am interested in what do you think a feature store means/implies/should do. The term is still (to my mind) very open to interpretation.. so I would honestly love to hear from you (and others)

The enterprise feature store we have should probably be more named as "data store but with advanced search/update capabilities" but.. that's not as nice sounding.

If you mean feature store as 'data ingestion via a DSL with type checking' then this is no...

that's... a very good question. When I was using Feast, it was that more than one person was interested in using the ingested data, so it became that 'single source of truth'. From then on, ClearML was used to do the actual pipeline flow and training/testing/serving runs and, since it's all python shop, it worked pretty well. We used it offline, since we didn't care about online with having features at inference time. I should probably write up something about this when I have the time come t...

honestly, I don't think the feature store we have would suit your needs. It is much closer to a data store in functionality with some nice to haves, rather than a feature store that is missing some bits.
Personally, I have used Feast before with a client, but only because it's a "pip install" to get it into place. It's a much lower barrier to entry than most of the others (again, bear in mind, I am a pythonista)

to be perfectly honest, I think I stopped investigating all the stuff plotly and friends can do these days.. I am sitting here with my mouth wide open.. some of their examples are awesome eye candy 😄

the brain surface viewer (more dash than anything) .. jst.. wow

👍 good luck!!!

Sadly, I haven't, but if anyone has then please scream because I would love to pick your brain for (yet another) post/article I am writing 😄 😄

I take it you are wanting to use Airflow to replace/extend an existing Jenkins setup ??

Howdy Jevgeni, that's .. strange. I am using google colab (free edition 🙂 and doing exactly the same as you, but I don't see any uncommited changes.. hrrm.. can you try this on colab maybe ? I am wondering if it's your jupyter notebook's version of python or some other notebook extension maybe

aaahhh.. I will wager good money Sir that you are then using ipython in vscode which is probably trying to do something "fancy" with the interpreter

(when I say fancy, you are free to substitute whatever adjective you wish instead 🙂

container_name:
  logging:
    options:
      max-size: 10m

Hey Leandro, I don't think it will work with replacing elasticserver with the AWS service type 😕

clearml-deploy is clearml-serving but with other parts more intwined such as ci/cd prompts/callbacks, if you think clearml-deploy has a bit more love given to it, I believe that will put you on the right track, but at it's core, it's the same idea Sir.

the hyper datasets have always been there in the enterprise offering. It allows you to query datasets and perform functions such as updating labels on an image without an entire re-batching. I think we are trying to find a way to bring this to...

hey @<1687643893996195840:profile|RoundCat60> .. did you ever get the problem sorted ?

the one you are in 🙂

I swear, I literally made this point in my zettelkasten
- demonstration/walkthrough/first steps - would require people to register for theseso I definitely think the demo/first steps is a great idea 👍

I agree with Martin.B, it appears to be a CUDA mismatch. The version of torch is trying to use cuda 10.2 but you have
agent.default_docker.image = nvidia/cuda:10.1-runtime-ubuntu18.04that should probably be
agent.default_docker.image = nvidia/cuda:10.2-runtime-ubuntu18.04

huh.. that example animation of automated driving on their homepage https://plotly.com/ is pretty awesome.

@<1687643893996195840:profile|RoundCat60> you set it once, inside the docker-compose itself.. it will affect all docker containers but, to be honest, docker tends to log everything

so yes indeedly ..

sudo find /var/lib/ -type d -exec du -s -x -h {} \; | grep G | more

seems to give saner results.. of course, in your case, you may also want to grep M for megabyte