I am so used to pip install, I default to there 😄

mea culpa

Hello.. I don't think so. Code of ethics can obviously vary from one job to another, and of course, so can legal compliance. You obviously have something very specific in mind, if you can expand on what you are looking for specifically, we maybe able to help.

usually though, genearlly speaking, a tools ethics and legality are set by the business side - not really something software would enforce on you. I hope I understand your question.

oh it's not a problem.. if you can fling up the logs of ES after startup that's probably the next step.. along with a 'docker network list' output 👍

👍

that's... a very good question. When I was using Feast, it was that more than one person was interested in using the ingested data, so it became that 'single source of truth'. From then on, ClearML was used to do the actual pipeline flow and training/testing/serving runs and, since it's all python shop, it worked pretty well. We used it offline, since we didn't care about online with having features at inference time. I should probably write up something about this when I have the time come t...

if you see it in the community server, then I believe the answer is "yes" - although don't hold me accountable on this 😄

one last tiny thing TrickySheep9 .. please do let us know how you get on, good or bad.. and if you bump into anything unexpected then please do scream and let us know 🙂

Shameless plug here ; https://clear.ml/blog/jupyter-notebooks-used-as-clearml-workers/

this whole area is a WIP of course, but I am trying to capture some of the really interesting Q and A from here so that they don't jst disappear into the void 🙂

Hey Slava, I don't mean to be "that guy" but, I am interested in what do you think a feature store means/implies/should do. The term is still (to my mind) very open to interpretation.. so I would honestly love to hear from you (and others)

The enterprise feature store we have should probably be more named as "data store but with advanced search/update capabilities" but.. that's not as nice sounding.

If you mean feature store as 'data ingestion via a DSL with type checking' then this is no...

I would think that a combination of kubernetes (I believe the preferred way to support multiple users at once, but open to being wrong) and individual queue's is probably the solution here.

for example; in kubernetes you could setup an agent to listen to bob-queue and another agent to listen to alice-queue. In the kubernetes dashboard you could assign a certain amount of cpu/memory and if using taints, gpu or not.

you're not going to get the same performance as you would from your own dual xeon with 128gb of ram etc 🙂

honestly.. I think google are "fine" with it.. there are plenty of other (more egregious) abuses of their colab and they haven't screamed yet.

clearml-deploy is clearml-serving but with other parts more intwined such as ci/cd prompts/callbacks, if you think clearml-deploy has a bit more love given to it, I believe that will put you on the right track, but at it's core, it's the same idea Sir.

the hyper datasets have always been there in the enterprise offering. It allows you to query datasets and perform functions such as updating labels on an image without an entire re-batching. I think we are trying to find a way to bring this to...

there is a --docker flag for clearml-agent that will build containers :)

since this is an enterprise machine, and you don't have sudo/root, I am wondering if there is already other docker networks/composer setups running/in use

The takeaway from the pricing page, I think, is that clearml is free as in speech. If you want super duper support that may cost $ but the folks in the community here do an awesome job in the meantime.

honestly, I don't think the feature store we have would suit your needs. It is much closer to a data store in functionality with some nice to haves, rather than a feature store that is missing some bits.
Personally, I have used Feast before with a client, but only because it's a "pip install" to get it into place. It's a much lower barrier to entry than most of the others (again, bear in mind, I am a pythonista)

Howdy and Morning @<1687643893996195840:profile|RoundCat60> .. docker when using overlay2 doesn't have it's mount points show up in a 'df' btw, they will only appear in a 'df -a', mostly because since they are simply 'overlays', they don't (technically) consume any space (I mean, the files are still in the /var/lib but not for the space counting practices used by df)

this is why I was suggesting a find, maybe with a 'du' .. actually.. let me try that here.. 2s

the one you are in 🙂

this obviously doesn't stop bad actors from assigning (say) an alice job over to bobs-queue but.. I am not sure of anyway to solve that.

hhrrmm.. in the initial problem, you mentioned that the /var/lib/docker/overlay2 was growing large in size.. but.. 4GB seems "fine" for docker images.. I wonder .. does your nvme0n1p1 ever report like 85% or 90% used or do you think that the 4GB is a lot ? when you restart the server, does the % used noticeably drop ? that would suggest tmp files inside the docker image itself which.. is possible with docker (weird but, possible)

obviously 🙂

so I am not entirely sure what else you have changed Sir

Evening Geoffrey, sorry for getting to this later in the day, I have been rather swamped today. All good though, all good.

What you raise is a good question. A very good question.

One of things that we have been thinking of around these parts is doing deep dives and interviews with users on how they came to ClearML, what setup they are using, key technologies and languages. In short, a sort of interview which will lead to a recipe book in the spirit of cooking (I would say CookBook but I th...

I also want to stress that these don't need to be happy-path interviews/results, although those are infinitely nicer to do 🙂 So I hear you with also noting what does not work as much as what did 👍

so yes indeedly ..

sudo find /var/lib/ -type d -exec du -s -x -h {} \; | grep G | more

seems to give saner results.. of course, in your case, you may also want to grep M for megabyte

There will be a roadmap for the community up and on the blog this Monday.. It may not be as detailed as you would like but I am always happy to yak about specific requests 👍 👍

adding this to the ever expanding list of "nice to have" features 👍

can you show me the complete output from 'docker-compose ps' please ? 🙂