Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
I Have A Self-Hosted Clearm-Server And And Clearml-Agent Started With

I have a self-hosted clearm-server and and clearml-agent started with clearml-agent daemon --queue default --foreground
I try to execute the following simple program:
` from clearml import Task

def main():
task = Task.init(project_name="test", task_name="test")

d = {"a": "1"}
task.upload_artifact("myArtifact", d)

if name == "main":
main() Locally it works well, I can see the artifact in the web gui. However, when I clone the experiment in the web gui and then submit it to my default queue, no artifact is generated. My agent only outputs all the install stuff and then Environment setup completed successfully

Starting Task Execution:

DONE: Running task 'ca6d1832b9e247178c35ad909c73bd50', exit status 0 `Any idea, why it is not creating the artifact?

  
  
Posted 3 years ago
Votes Newest

Answers 31


Correct :)

  
  
Posted 3 years ago

ReassuredTiger98 I'm trying to debug what's going on, because it should have worked.
Regrading Prints ...
` from clearml import Task
from time import sleep

def main():
task = Task.init(project_name="test", task_name="test")
d = {"a": "1"}
print('uploading artifact')
task.upload_artifact("myArtifact", d)
print('done uploading artifact')
# not sure if this helps but it won'r hurt to debug
sleep(3.0)
if name == "main":
main() `

  
  
Posted 3 years ago

Okay, this seems to work fine.

  
  
Posted 3 years ago

However, I cloned the experiment again via the web UI. Then I enqueued it.

  
  
Posted 3 years ago

ReassuredTiger98
Okay, but you should have had the prints ...
uploading artifactand
done uploading artifactSo I suspect something is going on with the agent.
Did you manage to run any experiment on this agent ?

EDIT: Can you try with artifacts example we have on the repo:
https://github.com/allegroai/clearml/blob/master/examples/reporting/artifacts.py

  
  
Posted 3 years ago

When I go into the GUI there are no artifacts displayed.

  
  
Posted 3 years ago

Thanks.

  
  
Posted 3 years ago

And in the web UI artifacts is still empty.

  
  
Posted 3 years ago

My agent shows the same as before:
` ...
Environment setup completed successfully

Starting Task Execution:

DONE: Running task 'aff7c6605b7243d38968f95b4351b127', exit status 0 `

  
  
Posted 3 years ago

Thanks!

  
  
Posted 3 years ago

When I add the file the to repo it works fine just like you said.

  
  
Posted 3 years ago

Yes, that looks alright. Similar to before. Local execution works.

  
  
Posted 3 years ago

Could you elaborate on that:
"So the agent failed to actually restore it from the git (files that are not added are not considered part of the git diff, this is usually git behavior)."

  
  
Posted 3 years ago

Thank you for all the help!

  
  
Posted 3 years ago

Hi ReassuredTiger98
Could you add some print ? before / after the artifact upload?
Also what's the clearml version you are using ?

  
  
Posted 3 years ago

ReassuredTiger98 when you look for task "dca2e3ded7fc4c28b342f912395ab9bc" there are no artifacts ?
Could you add some prints? this should have worked...

  
  
Posted 3 years ago

Makes sense.

  
  
Posted 3 years ago

Seems possible because I didn't know I had to specify an entrypoint somewhere. I will do some additional tests.

  
  
Posted 3 years ago

Nvm. I think I understood. When the file has never been added to repository it is not tracked.

  
  
Posted 3 years ago

Great!
I'll make sure the agent outputs the proper error 🙂

  
  
Posted 3 years ago

Okay there should not be any difference ... 😞

  
  
Posted 3 years ago

What do you mean by prints?

  
  
Posted 3 years ago

Local execution output:
ClearML Task: created new task id=855948f5d73c47e2ae37bb821385e15b ======> WARNING! Git diff to large to store (2190kb), skipping uncommitted changes <====== ClearML results page: uploading artifact done uploading artifact 2021-02-05 16:24:56,112 - clearml.Task - INFO - Waiting to finish uploads 2021-02-05 16:24:58,499 - clearml.Task - INFO - Finished uploading

  
  
Posted 3 years ago

Can you explain what you meant by entropy point file? In a new git repository my code works fine.

  
  
Posted 3 years ago

Sure, let me try.

  
  
Posted 3 years ago

Okay that look s good, now in the UI start here and then get to the artifacts Tab,
Is it there ?

  
  
Posted 3 years ago

Could you run your code not from the git repository.
I have a theory, you never actually added the entry point file to the git repo, so the agent never actually installed it, and it just did nothing (it should have reported an error, I'll look into it)
WDYT?

  
  
Posted 3 years ago

So no change in behaviour.

  
  
Posted 3 years ago

ReassuredTiger98

Can you explain what you meant by 

entropy point file?

There is no need to specify entry point file.
It is automatically detected when you run the Code manually on your machine.
My assumption was that the file "src/run_task.py" (based on your log) is just a test file, and hence was not added top the repository. So the agent failed to actually restore it from the git (files that are not added are not considered part of the git diff, this is usually git behavior). meaning the agent was not able to run your code.
Make sense ?

  
  
Posted 3 years ago

image
image

  
  
Posted 3 years ago
17K Views
31 Answers
3 years ago
7 months ago
Tags