Yes, it recreates the venv (or fetches it from cache) if you need your dataset, use Dataset class (it will cache it persistently, so no need to re-download)

SubstantialElk6 whats the command line you are using ?

It actually started executing your code, but it did not capture it correctly:

/root/.clearml/venvs-builds/3.10/bin/python -u /root/.clearml/venvs-builds/3.10/code/colab_kernel_launcher.py 

Which I assume means the actual Task had bad code.
What do you have under the Task execution tab in the UI (the one you were launching, i.e. enqueueing )

yes 🙂

HI FranticCormorant35 , the Reporter is internal implementation the Logger uses. In general you should use the Logger.

SmarmySeaurchin8
args=parse.parse() task = Task.init(project_name=args.project or None, task_name=args.task or None)You should probably look at the docstring 😉

:param str project_name: The name of the project in which the experiment will be created. If the project does
not exist, it is created. If project_name is None, the repository name is used. (Optional)
:param str task_name: The name of Task (experiment). If task_name is None, the Python experiment
...

I got everything working using the default queue. I can submit an experiment, and a new GPU node is provisioned, all good

Nice!

My next question, how do I add more queues?

You can create new queues in the UI and spin a new glue for the queue (basically think of a queue as an abstraction for a specific type of resource)
Make sense ?

Oh, is your pipeline code a part of a git repository ?

GreasyPenguin14 I think the default is reporting on failed tasks only? could that be?

,

remote_execute

kills the thread so the multirun stops at the first sub-task.

Hmm

task = Task.init(...)
# config some stuff
task.remote_execute(queue_name_here, exit_process=False)
# this means that the local execution will stop but when running on the remote agent it will be skipped
if Task.running_locally():
  return

Hmm let me check ...

from clearml.backend_api.session.client import APIClient c = APIClient() c.projects.update(project="project-id-here", system_tags=[])

it knows it’s a notebook and automatically adds the notebook as an artifact right?

correct

and the uncommited changes becomes the nottebook converted to a script?

correct

In one case I am seeing actual git diff coming in instead of the notebook.

it might be there is both a git repository and a notebook and the git diff will show before the notebook is detected and shown instead ? (there is a watchdog refreshing the notebook every 30sec or so)

Hi UnevenDolphin73
In theory it "might" work, I have to admit that personally I'm not a fan of what Amazon did to Mongo, i.e. forking their their code base and selling it as a service, just bad open-source practice
(The main issue might be API calls that might not fully match)
wdyt?

What do you have under the "installed packages" ?

If you are using user/pass for the git (i.e. not ssh key) we are not passing it to the pip install (and come to think about it, we probably should?!)

I tested and I have no more warning messages

if self._active_gpus and i not in self._active_gpus: continueThis solved it?

If so, PR pretty please 🙂

Oh I see the pipeline controller itself (not the components) is the one with the repo
To fix that add at the top of the script the following:
` from clearml import Task

Task.force_store_standalone_script()

@PipelineDecorator.pipeline(...) `That should do the trick

Hi ShakyJellyfish91
Check mount default here:
https://github.com/allegroai/clearml-agent/blob/e416ab526ba9fe05daa977b34c9e46b50fb214a0/docs/clearml.conf#L186
Is this what you are after, or do you actually want to change the start up script?

hmm that is odd.
Can you send the full log ?

GiganticTurtle0 we had this discussion in the wrong thread, I moved it here.
Moved from the wrong thread

Martin.B   [1:55 PM]
GiganticTurtle0  the sample mock pipeline seems to be running perfectly on the latest code from GitHub, can you verify ?

Martin.B   [1:55 PM]
Spoke too soon, sorry  🙂  issue is reproducible, give me a minute here

Alejandro C   [1:59 PM]
Oh, and which approach do you suggest to achieve the same goal (simultaneously running the same pipeline with differen...

You mean to add the extra index url?
you could use :
https://github.com/allegroai/clearml-agent/blob/5f0d51d485629e9dfc2d826622524461e3fcae8a/docs/clearml.conf#L63

BTW updating the values in grafana is basically configuration of the heatmap graph, so it is fairly easy to do, just not.automatic

AbruptHedgehog21 what exactly do you store as a Mode file ? is this a python object pickled ?

Hi @<1529271085315395584:profile|AmusedCat74>

ClearML Scheduler where it doesn't reuse the task

What do you mean by doesn't reuse the Task, do you mean you want each time the scheduler is launched to basically overwrite the previous run ?

So currently there is a limit (from the elasticsearch) of about 10k (anything above the is subsampled)
In the new version we are adding a "maximize" button, then in the full screen you will have the raw data including all ???k samples. sounds good?

In order to clone the Task it needs to complete sync, which implies closing. I guess the use case for execute remotely while still running was not considered. How / why is this your workflow? Specifically how does Jupyter get into the picture?

Could you extend on the use case of #18 ? how would you use it? what problem will it be solving ?

Yep

Hi OddShrimp85
If you pass 'output_uri=True' to task init, it will upload the model automatically, or as you said manually with outputmodel class