I think the only way you can get it is from the task attribute:
ds = Dataset.get(dataset_id="your dataset id") ds_uri = ds._task.artifacts.get("data").url
Hi ImmensePenguin78 ,
The problem seemed to be having a path with // and clearml not handling it well
Do you mean in the output_uri
parameter?
ThickDove42 you can get the version with
clearml-agent --version
Thanks for the information. do you get any errors? Warnings?
Hi ReassuredTiger98 , can you share your clearml
version?
I’m using some old agent I fear, since our infra person decided to use chart 3.3.0
That could be the issue, can you update to the latest version so we can check if this is the issue?
I’ll try with the env var too. Do you personally recommend docker over the simple AMI + virtual environment?
Depends, with docker you know what youll get and you can control many, venv should be quicker and you can set it before
More complete log does not add much information -
Can you sen...
Hi StaleMole4 ,
Hag Sameach 🙂 I just tried with the latest version and didn’t get any issues, can you try with it?
You can get your task logger any time with get_logger()
function, like in this example:
https://github.com/allegroai/clearml/blob/master/examples/reporting/text_reporting.py#L73
Hi FlatStarfish45 ,
In the UI you can view the packages that the task will use (under EXECUTION
tab), what do you get? Does it contains your packages in the INSTALLED PACKAGES
section?
You changed the version from 10.2 to 10.1 and nvidia-smi
output is the same? did you do a restart after the change?
The training task (child)
this is the task the HPO is cloning?
Do you have the packages in this task?
try pip install clearml==0.17.6rc1
Hi ElegantDeer55 ,
Are you referring to https://github.com/allegroai/trains-pycharm-plugin ? If so, it should sync you .git
folder to the remote machine so the task will log the git.
Basically I am confused if “remote debugging” should work / kick in automatically when running in docker mode and starting a task like this:
from trains import Task
task = Task.init(project_name=“my project”, task_name=“my task”)
task.execute_remotely()
When you are running this code from you P...
Hi NuttyOctopus69 ,
I’m getting the same, suspect its some pypi server issues (from https://status.python.org/ ).
You can install it the latest version from GitHub withpip install git+
Can you share the exception for --gpus "0,1"
?
and again - feature request - add free text there.
LethalCentipede31 can you add a new https://github.com/allegroai/clearml/issues issue with this request? Just so it won’t get lost
GiganticBat57 if you are running this code, do you get the output?
` import numpy as np
import matplotlib.pyplot as plt
from clearml import Task
task = Task.init(project_name='tests', task_name='test matplotlib')
logger = task.get_logger()
N = 50
x = np.random.rand(N)
y = np.random.rand(N)
colors = np.random.rand(N)
area = (30 * np.random.rand(N))**2 # 0 to 15 point radii
plt.scatter(x, y, s=area, c=colors, alpha=0.5)
Plot will be reported automatically
logger.report_matplotlib_figure(...
Hi ImmensePenguin78 , can you ping the server?
SubstantialElk6 you can try:
dataset_upload_task = Dataset.get(dataset_id=dataset_task) path_with_data = dataset_upload_task.get_local_copy()
Hi FlatStarfish45 ,
The HPO task will control the HPO process, means it will clone the base task (the one we are optimizing), change the parameters, enqueue it and collect the results.
The base task is the task we want to optimize.
Each one of those two tasks, have different requirements.
You can look at the https://clear.ml/docs/latest/docs/guides/optimization/hyper-parameter-optimization/examples_hyperparam_opt#set-up-the-arguments for how set the base task in the HPO task.
can this be ...
HelpfulHare30 try using CLEARML_AGENT_SKIP_PIP_VENV_INSTALL
env var with the path you your python env:CLEARML_AGENT_SKIP_PIP_VENV_INSTALL=/path/to/your/python
Hi TrickySheep9 ,
The Task.init
call will detect your repository as part of the automagic. You can disable it withauto_connect_frameworks={'detect_repository': False}
but I dont think this is the case.
Maybe clearml-task
? You can https://clear.ml/docs/latest/docs/guides/clearml-task/clearml_task_tutorial
Hi TrickySheep9 , can you copy the docker run command? do you docker install and running on the machine running the agent?
Hi GhastlySquirrel83 ,
You can specify the repository ( repo=None, repo_branch=None, repo_commit=None
parameters) in the add_function_step
for connecting the specific repo into the step, you can view all the options with some examples here - https://clear.ml/docs/latest/docs/references/sdk/automation_controller_pipelinecontroller#add_function_step
Sure, all the auto magic can be configured too - https://clear.ml/docs/latest/docs/faq#experiments , search for Can I control what ClearML automatically logs?
to view all the options 🙂
can you try upgrade? I run it with trains==0.16.2rc0
Hey PanickyMoth78 ,
my_pipeline
is wrapped with @PipelineDecorator.pipeline
? some other decorator?