ClearML doesnt add any yaml or lock files to your repo, actually I think it only needs read permissions.
In the full/official documentation the
clearml-data
CLI is not mentioned anywhere, so perhaps it should be refreshed
Yep, you are right, The clearml-data
is really new, and it will be in one of the next documentation version.
Not everything is manage with a git repo, if your script is a standalone, the full script will be in the uncommitted changes
section (EXECUTION tab).
The repository information is the repository location, the uncommitted changes, the branch with commit id / tag.
BTW, the full link to the docs - https://allegro.ai/clearml/docs/
For data versioning you can use the ClaerML data managemant.
Its being done with the CLI, an easy installation and you are ready to go, you can view a full example in this link - https://github.com/allegroai/clearml/blob/master/docs/datasets.md , including the installation.
Every task in ClearML includes the git repo, the changes and the full running environment.
You have some more cool things you can use (like pipelines, HPO, ClearML task CLI and more), you can find all of them here - ht...
Hi TrickySheep9 , currently when you override a parameter ( param1
in this example), you are overriding the whole parameter, but with using get_parameters
function the merge should be easy 😉
Hi SubstantialElk6 ,
Please feel free to ask anything 🙂
If nothing is reported, then nothing will be displayed.
For example, if you are reporting to Tensorboard, it will be automatically sent to the UI - https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/pytorch_tensorboard.py (run this example and view the scalers section 🙂 )
for furl
try with pip install git+
https://github.com/gruns/furl
pip install clearml
works for me now, if you like to try…
Hi NuttyOctopus69 ,
I’m getting the same, suspect its some pypi server issues (from https://status.python.org/ ).
You can install it the latest version from GitHub withpip install git+
seems like an issue with pypi, when trying to install a package with pypi server, it fails
Hi JitteryCoyote63 ,
I tried to reproduce this issue but it passed for me, what versions do you run (for trains, trains-server and trains-agent)?
can you try upgrade? I run it with trains==0.16.2rc0
Hi SmarmySeaurchin8 ,
I suspect the same, can you share an example of the path? I want to try and reproduce it on my side
Hi SmarmySeaurchin8
I tried to reproduce your issue (run https://github.com/allegroai/trains/blob/master/examples/reporting/image_reporting.py example with many report_image
calls) and got all the outputs in the debug samples section, can you share a snippet or how can I reproduce this issue?
nope, this example use local path (for the PIL image)
Trains auto-magical reports many frameworks plots: matplotlib, Tensorboard and more, maybe this is the issue? The report_media / report_image
was double reporting?
Can you send a screen grab?
Well the happy flow is to execute is locally, then clone this task and run it in the agent. If you dont want to run it locally, you can use the task.execute_remotely()
and clone this one or just start running it and kill it manually, or run one epoch to view the outputs
If you like, you can disable the auto magical for matplotlib
and report manually.
For doing so, pass auto_connect_frameworks={"matplotlib": False}
to Task.init
.
Can you send me the general flow in code?
BTW, if you want to clone a task, you can use Task.clone
Was the agent task cloned from the one works as expected (the dev task)?
When you run it locally with auto_connect_frameworks={"matplotlib": False}
, did it send the matplotlib
outputs?
Which framework do you use?
Can you try it?
Hi SmarmySeaurchin8 , Thanks, we will check that, can you share the Python version you use? 3.6?
Hi MoodyCentipede68 , do you have the repo as part of the https://clear.ml/docs/latest/docs/references/sdk/automation_controller_pipelinecontroller#pipelinedecoratorcomponent ? you can specify for each step which repo to use
Hi
I specify the repo for each step by using the ‘repo’ arguments from PipelineDecorator.component.
Here is my reference
MoodyCentipede68 do you see the repo under EXECUTION tab?
Hi GhastlySquirrel83 ,
You can specify the repository ( repo=None, repo_branch=None, repo_commit=None
parameters) in the add_function_step
for connecting the specific repo into the step, you can view all the options with some examples here - https://clear.ml/docs/latest/docs/references/sdk/automation_controller_pipelinecontroller#add_function_step