let me check when a fix can be deployed for Hydra...

DepressedChimpanzee34
What's the hydra version ?
I tested with 1.1.0dev3 and it worked for me

Hi DepressedChimpanzee34
Why do you need to have the configuration added manually ? isn't the cleaml.conf easier ? If not I think OS environments are easier no? I run run above code, everything worked with no exception/warning... What is the try/except solves exactly ?

Any chance you can zip the entire folder? I can't figure out what's missing, specifically "from config_files" , i.e. I have no packages nor file named config_files

DepressedChimpanzee34
I have sad news, it is working just fine for me 😞
Python3.6 Ubuntu hydra-core==1.1.0.dev6

Python3.8 I can quickly check, give me a minute

Noooooooooo, it is still working 🙂

Hi MinuteWalrus85
This is great question, and super important when training models. This is why we designed a whole system to manage datasets (including storage querying, balancing data, and caching). Unfortunately this is only available in the paid tier of Allegro... You are welcome to https://allegro.ai/enterprise/ the sales guys.
🙂

If you do not have a lot of workers, that I would guess console outputs

I just assumed it should only be triggered by dataset related things but after a lot of experimenting i realized its also triggered by tasks...

VexedCat68 I think you are correct, and it should only be triggered by "Dataset" Tasks, that said maybe there is a bug , in which case if there are no additional filters it will get triggered on Any change in the project. This will explain how adding the tags filter solved the issue.
wdyt?

Nice, that seems to be the issue. Any chance you can open a GitHub issue, so we do not loose track of it ?

Hmm this is odd, when you press on the parent dataset in the UI, and go to full-details, then the INFO tab. Can you copy here everything ?

clearml-task

 seems does not allow me passing the 

run

 argument without value

EnviousStarfish54 did you try --args run=True
I'm assuming run is a boolean of a sort ?

a. The submitted job would automatically download data from internal data repository, but it will be time consuming if data is re-downloaded every time. Does ClearML caching the data somewhere?

What do you mean by the agent will download the data ? are you referring to Dataset ?

Hmm that makes sense, I "think" the enterprise offering has a solution for that as well (i.e. full separation over static cluster), but probably the best way to constituent this avenue is talk to Sales (I'm assuming they'll setup a call to discuss the details)

Going back to the open source, I think that adding the credentials as part of the source code might allow to have "credentials" auto populate as part of the remote execution, wdyt?

main clearml repo?

Yep that sounds right 🙂 thank you!

will my datasets be stored on the same machine that hosts the clearml server?

By default yes, they will be stored to the files-server (but you can change it, this is an argument for both the CLI and the python interface)

SmugOx94

after having installed 

numpy==1.16

 in the first case or 

numpy==1.19

 in the second case. Is it correct?

Correct

the reason is simply that I'd like to setup an MLOps system where

I see the rational here (obviously one would have to maintain their requirements.txt)
The current way trains-agent works is that if there is a list of "installed packages" it will use it, and if it is empty it will default to the requirements.txt
We cou...

I would like to force the usage of those requirements when running any script

How would you force it? Will you just ignore the "Installed Packages" section ?

Let me check, which helm chart are you referring to ?

(I think the GCP is already up, I'll double check)

Hmmm are you saying the Dataset Tasks do not have the "dataset" system_tag as well as the type ?

checking it

Task.get_task(..).system_tags

true 😞

ReassuredTiger98 you mean when calling clearml-init ? or default value?

I'm not sure this is configurable from the outside 😞

ohh, the copy paste thing when you generate credentials ?

pip freezeworks ?

Hi @<1523701111020589056:profile|DefiantSpider5>
So there are two answers here, I'll start with the open-source version of both

Is there a way in clear ml to interactively view subsets of images based on a lasso of embedding plots

The ClearML Datasets have no "query" capabilities of the data inside the dataset. That means you can see preview images, statistics and download the datasets, but no query capabilities. On the other hand, there is no limitation on the type and format of me...

Hi EnthusiasticCoyote38

Does clearml-agent has option

Fully supported 🙂
Should work out of the box, it will always clone with --recursive and will bring all submodules