Hi @<1523701205467926528:profile|AgitatedDove14>
I'm having a similar issue.

Also notice the cleaml-agent will not change the entry point of the docker meaning if the entry point does not end with plain bash, it will not actually run anything

Not sure I understand how to run a docker_bash_setup_script and then run a python script - Do you have an example? I could not find one.
Here is our CLI command

clearml-task --name <TASK NAME> \
		--project <PRJ NAME> \
		--repo git@gi...

Hi CostlyOstrich36 ,
After verifying - I can confirm that there is no custom certificate .
any other ideas?

Thx for your reply

TimelyPenguin76 - do you have any suggestions?

Btw -after updating clearml.conf do I need to restart the agent?
I can't see the additional tab under https://clearml.slack.com/archives/CTK20V944/p1658199530781499?thread_ts=1658166689.168039&cid=CTK20V944 , and I reran the task and got the same error

i've doubled checked the Path- and it is correct

I'm looking for the bucket URI
I think my work flow needs to alter.
get the data into the bucket and then create the Dataset using the add_external_file and then be able to consume the data locally or stream And then I can use - link_entries

Hi SuccessfulKoala55
Is this section only relevant to AWS or also to GCP?

Hi AgitatedDove14
OK - the issue was the firewall rules that we had.

Now both of the jupyter lab and vscode servers are up.

But now there is an issue with the Setting up connection to remote session

After the

Environment setup completed successfully
Starting Task Execution:
ClearML results page:

https://app-clearml.XXXX/projects/XXXXXXXXXX/output/log

There is a WARNING

clearml - WARNING - Could not retrieve remote configuration named 'SSH'...

Hi SweetBadger76 ,
Well - apparently I was mistaken.
I still have a ghost worker that i'm mot able to remove (I had 2 workers on the same queue - that caused my confusion).
I can see it in the UI and when I run clearml-agent list
And although I'm stoping the worker specifically
clearml-agent daemon --stop <worker_id>I'm getting
Could not find a running clearml-agent instance with worker_name=<worker_id> worker_id=<worker_id>

Thx CostlyOstrich36 for your reply
Can't see the reverence to parquet . we are currently using the above functionality , but the pd.DataFrame is only saved as csv compressed by gz

Please advise on how to remove a worker

from the example -
since the `mp_hander`` runs

cmd = [sys.executable, sys.argv[0],
               '--counter', str(counter - 1),
               '--num_workers', str(args.num_workers),
               '--use-subprocess' if args.subprocess else '--no-subprocess']
p = subprocess.Popen(cmd, cwd=os.getcwd())

can I run another subprocess in the mp_worker ?

Well - that will convert it to a binary pickle format but not as parquet -
since the artifact will be accessed from other platforms we want to use parquet

Possibly - thinking more of https://github.com/pytorch/data/blob/main/examples/vision/caltech256.py - using clearml dataset as root path.

Hi,
You may want to consider to do the visualizing while creating the Datasets - see https://github.com/thepycoder/asteroid_example/blob/main/get_data.py#L34 logging the head() of the dataframe

clearml-3.5.0

You are getting a string - '5'
I want to get an int - 5

Thx CostlyOstrich36 for your input -
So I guess that if we try to always work with https://clear.ml/docs/latest/docs/fundamentals/hyperparameters (even if there is only have 1 parameter), we will consistently log our parameters.
Do you have a suggested different workflow?

Great - Thx TimelyPenguin76 for your input

try
None
or
None

add the google.storage parameters to the conf settings
sdk { google.storage { credentials = [ { bucket: "clearml-storage" project: "dev" credentials_json: /path/to/SA/creds/user.json }, ] } }%

yes - the pre_installations.sh runs and completes - but the pytorch/main.py file doesn't run.
so the Task completes successfully but without running the script

Sorry -
After updating the repo I can see that the newest chart is 4.1.1
SweetBadger76 should I update to this version?

Version: 1.6.4

updated the clearml.conf with empty worker_id/name ran

clearml-agent daemon --stop

top | grep clearmKilled the pidsran

clearml-agent list

still both of the workers are listed

AgitatedDove14 -
I also tried to https://github.com/allegroai/clearml-session
running the session within docker but got the same error

clearml-session --docker

--git-credentials

(there is a typo in git - --git-credent ila s -> --git-credent ials)
and still got the same error

clearml_agent: ERROR: Can not run task without repository or literal script in script.diff

Are you running the "cleamrl-session" from your machine? (i.e. not from inside a docker) ?

correct - running it locally - not inside docker . Should I try to run within a docker?

Can you send the full clearml-session console output ?

see above

AgitatedDove14 - here is the full log

will do
A work around that worked for me is to explicitly complete the task, seems like the flush has some bug
task = Task.get_task('...')
task.close()
task.mark_completed()

ds.is_final()
True