FlutteringMouse14

5 Questions, 8 Answers

Active since 18 April 2024

Last activity 6 months ago

Reputation

Badges 1

8 × Eureka!

Questions 5
Answers 8

0 Votes

6 Answers

624 Views

0 Votes 6 Answers 624 Views

Hi Guys! Anyone Else Has Trouble Caching Virtual Environments In The Agent? I Manually Installed A Single Agent On A Virtual Machine, And I'M Using This Conf For Venv Caching :

Hi guys! Anyone else has trouble caching virtual environments in the agent? I manually installed a single agent on a virtual machine, and I'm using this conf...

mlops

6 months ago

0 Votes

2 Answers

508 Views

0 Votes 2 Answers 508 Views

Hi Everyone! Thanks To The Dev For The Excellent Work, I'M Migrating Some Work Projects To Clearml And It Has Been A Great Experience So Far. There Is Just One Thing I Cannot Understand From The Documentation: Can I Set Execution Details For A Pipeline E

Hi everyone! Thanks to the dev for the excellent work, I'm migrating some work projects to ClearML and it has been a great experience so far. There is just o...

clearml

7 months ago

0 Votes

4 Answers

704 Views

0 Votes 4 Answers 704 Views

Hi Guys! Broad Question: Do You Work With A Feature Store Along With Clearml? Right Now My Workflow Is: 1) Clearml Task Fetches Data From Database/Api, Does A Rough Cleaning And Saves A "Raw" Versioned Dataset On Clearml 2) Another Clearml Task Trains A

Hi guys! Broad question: do you work with a feature store along with ClearML? Right now my workflow is: 1) clearml task fetches data from database/api, does ...

clearml

6 months ago

0 Votes

1 Answers

391 Views

0 Votes 1 Answers 391 Views

Hi Guys! I'M Getting A Weird Problem. In The Latest Project I Created, Hydra Conf Is Not Logged Automatically. In The Script Hydra Is Working Correctly, And The Code For The Clearml Part Is Copied From Another Script (That Logs Hyperpars Correctly). I Al

Hi guys! I'm getting a weird problem. In the latest project I created, Hydra conf is not logged automatically. In the script Hydra is working correctly, and ...

clearml

5 months ago

0 Votes

6 Answers

615 Views

0 Votes 6 Answers 615 Views

Hi Guys! How Do You Handle Tasks With A Complex Parametrization? For Example, A Script That Trains A Machine Learning Model, Where You Want To Parametrize Model Name, Hyperpars, Preprocessing Steps, Etc. So A Nested Configuration With Many Parameters Do I

Hi guys! How do you handle tasks with a complex parametrization? For example, a script that trains a machine learning model, where you want to parametrize mo...

clearml

7 months ago

0 Hi Everyone! Thanks To The Dev For The Excellent Work, I'M Migrating Some Work Projects To Clearml And It Has Been A Great Experience So Far. There Is Just One Thing I Cannot Understand From The Documentation: Can I Set Execution Details For A Pipeline E

Yes! I was looking at it wrong. The task execution details are only there for logging previous execution.
To modify the execution of the next run I had to modify the clearml agent. In this case I added this to the clearml.conf of my agent (I only have one worker right now):
python_binary: "/usr/local/bin/python3.10"
ignore_requested_python_version: true

7 months ago

0 Hi Guys! Broad Question: Do You Work With A Feature Store Along With Clearml? Right Now My Workflow Is: 1) Clearml Task Fetches Data From Database/Api, Does A Rough Cleaning And Saves A "Raw" Versioned Dataset On Clearml 2) Another Clearml Task Trains A

Cool thanks! So for online Feature transformation I would set up a feast server, and the tasks would use feast to create and retrieve features, right?

I know it's a huge project, but it would be great to have a feature store as part of ClearML, do you have any plans to do that?

6 months ago

I can imagine :D I'll look into it, thanks!

6 months ago

0 Hi Guys! Anyone Else Has Trouble Caching Virtual Environments In The Agent? I Manually Installed A Single Agent On A Virtual Machine, And I'M Using This Conf For Venv Caching :

@<1694157594333024256:profile|DisturbedParrot38> thanks a lot! I didn't know it. On my project, clearml can't recognize the requirements from the code, I have to pass them explicitly, either with poetry or with a requirements.txt. Does that work for you?

6 months ago

0 Hi Guys! Anyone Else Has Trouble Caching Virtual Environments In The Agent? I Manually Installed A Single Agent On A Virtual Machine, And I'M Using This Conf For Venv Caching :

@<1523701070390366208:profile|CostlyOstrich36> It's 1.8.0

6 months ago

0 Hi Guys! How Do You Handle Tasks With A Complex Parametrization? For Example, A Script That Trains A Machine Learning Model, Where You Want To Parametrize Model Name, Hyperpars, Preprocessing Steps, Etc. So A Nested Configuration With Many Parameters Do I

I had no idea you could do this, that's amazing thanks a lot!
Although after reviewing the options, probably managing everything with Hydra is the better way :D

6 months ago

Yeah that sounds great, look forward to try it! Thanks a lot :)

6 months ago

That's exactly what I was looking for! Thanks a lot.
Just two followup questions

if I connect a dict with task.connect and I edit it in the UI, will the new values be passed to the next run?
2)with connect_configuration, can I pass a python file containing the configuration (like, a configuration-as-code file, instead of a yml or a json)? Could I modify it from UI?

7 months ago