Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Profile picture
OutrageousSheep60
Moderator
27 Questions, 111 Answers
  Active since 10 January 2023
  Last activity one year ago

Reputation

0

Badges 1

103 × Eureka!
0 Votes
23 Answers
1K Views
0 Votes 23 Answers 1K Views
Hi, We have started to use clearml using the https://clear.ml/docs/latest/docs/deploying_clearml/upgrade_server_kubernetes_helm , and trying to understand ho...
gcp
3 years ago
0 Votes
1 Answers
892 Views
0 Votes 1 Answers 892 Views
Question: How can we run a task in a docker container without having to configure it manually? Assumptions with ClearML Agent: To run an experiment using Cle...
one year ago
0 Votes
1 Answers
1K Views
0 Votes 1 Answers 1K Views
Hi, Is there a way to instantiate a clearml-task while providing it a Dockerfile that it needs to build prior to executing the task? I know that there is a a...
2 years ago
0 Votes
4 Answers
1K Views
0 Votes 4 Answers 1K Views
Hi, I'm looking for an example show how clearml dataset is consumed by sklearn pipeline or pytorch datapipe. e.g. since pytorch uses torch datasets, can we w...
2 years ago
0 Votes
11 Answers
1K Views
0 Votes 11 Answers 1K Views
Hi, We have recently upgraded to > WebApp: 1.6.0-213 • Server: 1.6.0-213 • API: 2.20 I have some issues: I'm trying to delete a project - and although I don'...
2 years ago
0 Votes
7 Answers
1K Views
0 Votes 7 Answers 1K Views
Hi, Question regarding the command > clearml-agent list Were do you define the company and user that is running the Worker ? Looking at the https://clear.ml/...
2 years ago
0 Votes
9 Answers
1K Views
0 Votes 9 Answers 1K Views
Hi, I'm setting a USER PROPERTIES using the following code task.set_user_properties(MY_THRESHOLD={"type": int, "value": 0.85}, )Whet is the correct way to re...
2 years ago
0 Votes
1 Answers
1K Views
0 Votes 1 Answers 1K Views
Hi, How can I get the remote URI of a Dataset (assuming it is GS or S3). Do I need to combine the get_default_storage() with something? Since some datasets a...
2 years ago
0 Votes
4 Answers
1K Views
0 Votes 4 Answers 1K Views
Hi, working with clearml 1.6.4 what is the correct way to list all the dataset s within a specific project ?
2 years ago
0 Votes
4 Answers
1K Views
0 Votes 4 Answers 1K Views
Hi, Is there a way to add the sdk https://clear.ml/docs/latest/docs/configs/clearml_conf#sdkgooglestorage parameters when setting the https://github.com/alle...
2 years ago
0 Votes
4 Answers
1K Views
0 Votes 4 Answers 1K Views
Hi, I've added external links to the a dataset In the UI I can see that the dataset has the property of Link count with the correct number. But where can I s...
2 years ago
0 Votes
1 Answers
1K Views
0 Votes 1 Answers 1K Views
Hi, I have a question regarding the Dataset UI (chart helm version 4.1.3 )- WebApp: 1.6.0-213 • Server: 1.6.0-213 • API: 2.20. In order to create a Dataset y...
2 years ago
0 Votes
6 Answers
1K Views
0 Votes 6 Answers 1K Views
Hi,
Hi, What would be the correct way to check the hash of a file before using https://clear.ml/docs/latest/docs/references/sdk/dataset/#add_files ? is there a b...
one year ago
0 Votes
24 Answers
1K Views
0 Votes 24 Answers 1K Views
Hi, Question regarding clearml-agent How do we delete a daemon? There is an option to stop a daemon service - but what is the method to delete a specific wor...
2 years ago
0 Votes
8 Answers
1K Views
0 Votes 8 Answers 1K Views
Hi, What would be the best way to save a pandas.DataFrame as an https://allegro.ai/clearml/docs/rst/references/clearml_python_ref/task_module/task_task.html?...
2 years ago
0 Votes
6 Answers
1K Views
0 Votes 6 Answers 1K Views
Hi, Newbe question in ClearML Data Assuming my files are already in a bucket on the cloud - What would be the best practice to create a dataset and register ...
2 years ago
0 Votes
30 Answers
1K Views
0 Votes 30 Answers 1K Views
Hi We are getting the following error when we are trying to run a task on our on premis clearml-agent ( version 1.3.0) cloning: git@github.com:XXXX/sample.re...
2 years ago
0 Votes
5 Answers
1K Views
0 Votes 5 Answers 1K Views
Hi, Is it possible to run ffmpeg within clearml server as a multi processing (as in this example ?) assuming that the ffmpeg will run using a subprocess comm...
one year ago
0 Votes
5 Answers
1K Views
0 Votes 5 Answers 1K Views
Hi, We are trying to understand when the auto-logging parameters configuration. from the https://github.com/allegroai/clearml/blob/master/examples/frameworks...
2 years ago
0 Votes
3 Answers
1K Views
0 Votes 3 Answers 1K Views
Hi, I'm running Dataset.list_datasets() (Version: 1.6.4) and I can't see all the Datasets that I see in the GUI. Any suggestions?
2 years ago
0 Votes
6 Answers
1K Views
0 Votes 6 Answers 1K Views
Hi, I'm running clearml==1.8.4rc0 When creating a Dataset with the following code - the dataset that is created includes ALL the files within the bucket and ...
2 years ago
0 Votes
4 Answers
1K Views
0 Votes 4 Answers 1K Views
Hi Another issue regarding dataset whenever preview ing the dataset (which is in a parquet tabular format) the browser automatically downloads a copy of the ...
2 years ago
0 Votes
3 Answers
1K Views
0 Votes 3 Answers 1K Views
Hi i'm trying to finalize a dataset, but although the finalize(auto_upload=True) completes successfully (see image) - But the dataset is still in an uploadin...
one year ago
0 Votes
2 Answers
1K Views
0 Votes 2 Answers 1K Views
Hi Community, We are having an issue connecting our clearml server via code When I connect via a web browser - there is no problem . When trying to connect v...
3 years ago
0 Votes
9 Answers
1K Views
0 Votes 9 Answers 1K Views
Hi, We have a use case that we would like to upload a local folder into the cloud AS-IS - without compressing or breaking it up into chunks. I tried running ...
2 years ago
0 Votes
19 Answers
1K Views
0 Votes 19 Answers 1K Views
Hi, I'm trying to run clearml-agent in docker mode - however I'm having trouble saving the artifacts. 2022-08-25 05:06:21,820 - clearml.storage - ERROR - Fai...
2 years ago
0 Votes
17 Answers
1K Views
0 Votes 17 Answers 1K Views
Hi, I'm following the instructions for https://clear.ml/docs/latest/docs/guides/ide/remote_jupyter_tutorial . after > Launch interactive session [Y]/n? y I"m...
2 years ago
0 Hi, I'M Setting A

You are getting a string - '5'
I want to get an int - 5

2 years ago
0 Hi, I'M Setting A

exactly - (that is how I used it in my initial code) - but if you have to convert it back to the original data type then something is broken...

2 years ago
0 Hi, I'M Setting A

Thx for investigating - What is the use case for such behavior ?
How would you use the user properties as part of an experiment?

2 years ago
0 Hi, Is It Possible To Run

from the example -
since the `mp_hander`` runs

cmd = [sys.executable, sys.argv[0],
               '--counter', str(counter - 1),
               '--num_workers', str(args.num_workers),
               '--use-subprocess' if args.subprocess else '--no-subprocess']
p = subprocess.Popen(cmd, cwd=os.getcwd())

can I run another subprocess in the mp_worker ?

one year ago
0 Hi, Is It Possible To Run

@<1523701205467926528:profile|AgitatedDove14> -
I'm getting the following error when running the following code within the mp_worker

command = ["ffmpeg","-i",f"{url}","-vcodec","libx264", "output.mp4"]
subprocess.run(command, stderr=subprocess.STDOUT)

TypeError: fork_exec() takes exactly 21 arguments (17 given)

Any suggestions?

one year ago
0 Hi, I'M Following The Instructions For

Is there any settings that we need to take into account when working with session ?
in the https://clear.ml/docs/latest/docs/apps/clearml_session#accessing-a-git-repository it mentions accessing Git Repository -
Can you run clearml sessions without accessing Git? Assuming we are using ssh - what is the correct configuration?

2 years ago
0 Hi, I'M Running

SmugDolphin23 Where can I check the lates RC? I was not able to find it in the clearml github repo

2 years ago
0 Hi, I'M Running

SmugDolphin23 I'll be happy to checkout the RC once it is fixed

2 years ago
0 Hi, I'M Running

THx SmugDolphin23 - looks OK

2 years ago
0 Hi, I'M Following The Instructions For

Are you running the "cleamrl-session" from your machine? (i.e. not from inside a docker) ?

correct - running it locally - not inside docker . Should I try to run within a docker?

Can you send the full clearml-session console output ?

see above

2 years ago
0 Hi, I'M Following The Instructions For

I found the task in the UI -
and in the UNCOMMITTED CHANGES execution section there is

No changes logged

Any other suggestions?

2 years ago
0 Hi, I'M Following The Instructions For

Hi AgitatedDove14
OK - the issue was the firewall rules that we had.

Now both of the jupyter lab and vscode servers are up.

But now there is an issue with the Setting up connection to remote session

After the

Environment setup completed successfully
Starting Task Execution:
ClearML results page:

https://app-clearml.XXXX/projects/XXXXXXXXXX/output/log

There is a WARNING

clearml - WARNING - Could not retrieve remote configuration named 'SSH'...

2 years ago
0 Hi, I'M Following The Instructions For

Hi SuccessfulKoala55
I've run the daemon via docker
CLEARML_WORKER_ID=XXXX clearml-agent daemon --queue MY_QUEUE --docker --detached and then run the session via docker
clearml-session --docker nvidia/cuda:10.1-cudnn7-runtime-ubuntu18.04 \ --packages "clearml" "tensorflow>=2.2" "keras" \ --queue MY_QUEUE \ --verboseHowever I'm still getting the same error

clearml_agent: ERROR: Can not run task without repository or literal script in script.diff

2 years ago
0 Hi, I'M Following The Instructions For

BTW - is the CLEARML_HOST_IP relevant for the clearml-agent ?
i can see that we can create a worker with this environment variable . e.g.
CLEARML_WORKER_NAME=MY-WORKDER CLEARML_WORKER_ID=MY-WORKER:0 CLEARML_HOST_IP=X.X.X.X clearml-agent daemon --detachedmy mistake doesn't use it to create a dedicated IP

2 years ago
0 Hi, I'M Following The Instructions For

I'm checking the possibility of our firewall between the clearml-agent machine and the local computer running the session

2 years ago
0 Hi, I'M Following The Instructions For

AgitatedDove14 - here is the full log

2 years ago
0 Hi, What Would Be The Best Way To Save A Pandas.Dataframe As An

Using the https://allegro.ai/clearml/docs/rst/references/clearml_python_ref/task_module/task_task.html?highlight=upload_artifact#clearml.task.Task.upload_artifact method. It works well, but only saves it as a csv (which is very problematic since when loading the artifact none of the data types of the columns are preserved...)

2 years ago
0 Hi, What Would Be The Best Way To Save A Pandas.Dataframe As An

Thx CostlyOstrich36 for your reply
Can't see the reverence to parquet . we are currently using the above functionality , but the pd.DataFrame is only saved as csv compressed by gz

2 years ago
0 Hi, What Would Be The Best Way To Save A Pandas.Dataframe As An

CostlyOstrich36 - but we will use any method that will allow us to save the files as parquet.
We are not yet using clearml Dataset - i'm not sure if this is a solution

2 years ago
0 Hi, What Would Be The Best Way To Save A Pandas.Dataframe As An

Well - that will convert it to a binary pickle format but not as parquet -
since the artifact will be accessed from other platforms we want to use parquet

2 years ago
2 years ago
0 Hi, Question Regarding

Also, what version are you on?

of what?

2 years ago
0 Hi Another Issue Regarding Dataset

CostlyOstrich36 do you need any additional information?

2 years ago
0 Hi Another Issue Regarding Dataset

here is a video of the steps

2 years ago
0 Hi, Question Regarding

not sure I understand
running
clearml-agent listI get
`
workers:

  • company:
    id: d1bd92...1e52b
    name: clearml
    id: clearml-server-...wdh:0
    ip: x.x.x.x
    ... `
2 years ago
2 years ago
0 Hi, Question Regarding

updated the clearml.conf with empty worker_id/name ran

clearml-agent daemon --stop

top | grep clearmKilled the pidsran

clearml-agent list

still both of the workers are listed

2 years ago
Show more results compactanswers