Well it seems that we have similar https://github.com/allegroai/clearml-agent/issues/86
currently we are just creating a new worker and on a separate queue

Distributor ID: Ubuntu
Description: Ubuntu 20.04.4 LTS
Release: 20.04
Codename: focal

not sure I understand
running
clearml-agent listI get
`
workers:

• company:
  id: d1bd92...1e52b
  name: clearml
  id: clearml-server-...wdh:0
  ip: x.x.x.x
  ... `

Hi SuccessfulKoala55
Thx again for your help

in case of the google colab, the values can be provided as environment variables

We still need to run the code in a colab environment (or remote client)
do you have any example for setting the environment variables?
For a general environment variable there is an example
! export MPLBACKEND=TkAgBut what would be for the clearml.conf ?
retrieving we can use

config_obj.get('sdk.google')

but how would the setting work? we did ...

Thx again for your time -

Where the experiment is being executed

Not sure I understand what you mean by this -
Assuming that we are running the ClearML on GKE (we have succeeded doing so) - and running the python code from COLAB or locally. Where do we configure the Google Storage ? how can the helm / k8s dynamically load the clearml.conf ? is it only from values.yaml ?

Where you view your experiment

In mlflow I was able to view the artifact directly (a...

This also may help with the configuration for GCS
https://clearml.slack.com/archives/CTK20V944/p1635957916292500?thread_ts=1635781244.237800&cid=CTK20V944

from the example -
since the `mp_hander`` runs

cmd = [sys.executable, sys.argv[0],
               '--counter', str(counter - 1),
               '--num_workers', str(args.num_workers),
               '--use-subprocess' if args.subprocess else '--no-subprocess']
p = subprocess.Popen(cmd, cwd=os.getcwd())

can I run another subprocess in the mp_worker ?

That is a workaround - but surly not optimal
If we want to generate a dataset from a set of files that are on a local computer (e.g. a local GPU workstation then ran some media transformation) -
then instead of creating the Dataset directly - we need to first upload them and only then use the ClearML sdk.
Do you see any option integrating this kind of workflow into clearml?

Well - that will convert it to a binary pickle format but not as parquet -
since the artifact will be accessed from other platforms we want to use parquet

Thx for investigating - What is the use case for such behavior ?
How would you use the user properties as part of an experiment?

Is current relationship only available via _get_parents() method?

Is there any settings that we need to take into account when working with session ?
in the https://clear.ml/docs/latest/docs/apps/clearml_session#accessing-a-git-repository it mentions accessing Git Repository -
Can you run clearml sessions without accessing Git? Assuming we are using ssh - what is the correct configuration?

Hi SuccessfulKoala55
Any suggestion how to progress ?

Hi AnxiousSeal95 ,
Is there an estimate when the above feature will be available?

we reinstalled the clearml-agent$clearml-agent --version CLEARML-AGENT version 1.2.3running top | grep clearmlwe can see the agent running
running clearml-agent listwe can see 2 workers
before running clearml-agent daemon --stopWe updated the clearml.conf and updated the worker_id and worker_name with the relevant name/id that we can see from clearml-agent list
and we get
` Could not find a running clearml-agent instance with worker_name=<clearml_worker_na...

ClearML key/secret provided to the agent

When is this provided? Is this during the build ?

Ok - I can see that if I ran finalize(auto_upload=True) on the dataset - I get all the information in the UI.
Way is this necessary?

CLEARML-AGENT version 1.2.2

try
None
or
None

Version: 1.6.4

yes - and removed from
clearml-agent list

Sorry -
After updating the repo I can see that the newest chart is 4.1.1
SweetBadger76 should I update to this version?

Strange
I ran
clearml-agent daemon --stopand after 10 min I ran
clearml-agent listand I still see a worker

@<1523701205467926528:profile|AgitatedDove14> -
I'm getting the following error when running the following code within the mp_worker

command = ["ffmpeg","-i",f"{url}","-vcodec","libx264", "output.mp4"]
subprocess.run(command, stderr=subprocess.STDOUT)

TypeError: fork_exec() takes exactly 21 arguments (17 given)

Any suggestions?

What key is used for this? I can only see git_user

Hi

you will have to configure the credentials there (in a local 

clearml.conf

 or using environment variables

This is the part that confuses me - is there a way to configure clearml.conf from the values.yaml ? I would like the GKE to load the cluster with the correct credentials without logging into the pods and manually updating the claerml.conf file

Also, what version are you on?

of what?

we want to use the dataset output_uri as a common ground to create additional dataset formats such as https://webdataset.github.io/webdataset/

CostlyOstrich36 - but we will use any method that will allow us to save the files as parquet.
We are not yet using clearml Dataset - i'm not sure if this is a solution

Sorry - I'm a Helm newbee
when running
helm search repo clearml --versionsI can't see version 3.6.2 - the highest is 3.5.0
This is the repo that we used to get the helm chart
helm repo add allegroaiWhat I'm I missing?