I swear, I literally made this point in my zettelkasten
- demonstration/walkthrough/first steps - would require people to register for theseso I definitely think the demo/first steps is a great idea 👍

I would think that a combination of kubernetes (I believe the preferred way to support multiple users at once, but open to being wrong) and individual queue's is probably the solution here.

for example; in kubernetes you could setup an agent to listen to bob-queue and another agent to listen to alice-queue. In the kubernetes dashboard you could assign a certain amount of cpu/memory and if using taints, gpu or not.

this obviously doesn't stop bad actors from assigning (say) an alice job over to bobs-queue but.. I am not sure of anyway to solve that.

(when I say fancy, you are free to substitute whatever adjective you wish instead 🙂

the one on the right, for example, has no data points at around the 19 mark

Hello.. I don't think so. Code of ethics can obviously vary from one job to another, and of course, so can legal compliance. You obviously have something very specific in mind, if you can expand on what you are looking for specifically, we maybe able to help.

usually though, genearlly speaking, a tools ethics and legality are set by the business side - not really something software would enforce on you. I hope I understand your question.

do you have code that you can share ?

from the look of those two graphs, the underlying data is totally different 😕

the part that I am concerned on is that the first pair of graphs you showed, the dataset (even from jst looking at it) are very different 😕

Hello E.K, do you have any examples handy to show us the difference ?

oh it's not a problem.. if you can fling up the logs of ES after startup that's probably the next step.. along with a 'docker network list' output 👍

So those datasets are the same but.. yes.. it's squashed/warped

so yes indeedly ..

sudo find /var/lib/ -type d -exec du -s -x -h {} \; | grep G | more

seems to give saner results.. of course, in your case, you may also want to grep M for megabyte

container_name:
  logging:
    options:
      max-size: 10m

Hey Slava, I don't mean to be "that guy" but, I am interested in what do you think a feature store means/implies/should do. The term is still (to my mind) very open to interpretation.. so I would honestly love to hear from you (and others)

The enterprise feature store we have should probably be more named as "data store but with advanced search/update capabilities" but.. that's not as nice sounding.

If you mean feature store as 'data ingestion via a DSL with type checking' then this is no...

Evening Geoffrey, sorry for getting to this later in the day, I have been rather swamped today. All good though, all good.

What you raise is a good question. A very good question.

One of things that we have been thinking of around these parts is doing deep dives and interviews with users on how they came to ClearML, what setup they are using, key technologies and languages. In short, a sort of interview which will lead to a recipe book in the spirit of cooking (I would say CookBook but I th...

I also take it you were thinking of fargate for the clearml-agent's .. which would be awesome but.. oi.. not sure how much luck you would have there 🙂 🙂

oh .. no worries at all then.. you are free to do whatever you want to with it.. but I don't think it's designed with what you are trying to do in mind sadly

if you manage to get it up and running, I would love to do a deep dive with you to understand how you did and share it on the company's blog 🙂 🙂

no pressure 😜

I know the storage can be swapped out to using S3 (obviously)

You may also want to brush up on the security and firewalls for AWS.. those always seem to be voodoo as far as I can tell 😄

Howdy and Morning @<1687643893996195840:profile|RoundCat60> .. docker when using overlay2 doesn't have it's mount points show up in a 'df' btw, they will only appear in a 'df -a', mostly because since they are simply 'overlays', they don't (technically) consume any space (I mean, the files are still in the /var/lib but not for the space counting practices used by df)

this is why I was suggesting a find, maybe with a 'du' .. actually.. let me try that here.. 2s

@<1687643893996195840:profile|RoundCat60> you set it once, inside the docker-compose itself.. it will affect all docker containers but, to be honest, docker tends to log everything

There is already a pre-built AMI fwiw 🙂

👍

Hey there waves

Not sure about plans to automate this in the future, as this is more how docker behaves and not really clearml, especially with the overlay2 filesystem. The biggest offender usually is your json logfiles. have a look in /var/lib/docker/containers/ for *.log

assuming this IS the case, you can tell docker to only log upto a max-size .. I have mine set to 100m or some such

honestly, I don't think the feature store we have would suit your needs. It is much closer to a data store in functionality with some nice to haves, rather than a feature store that is missing some bits.
Personally, I have used Feast before with a client, but only because it's a "pip install" to get it into place. It's a much lower barrier to entry than most of the others (again, bear in mind, I am a pythonista)

you can cut steak with a spoon, but that doesn't mean it's a good idea 😉 😉

👍 good luck!!!

Sadly, I haven't, but if anyone has then please scream because I would love to pick your brain for (yet another) post/article I am writing 😄 😄

I take it you are wanting to use Airflow to replace/extend an existing Jenkins setup ??