I mean to use a function decorated with 

PipelineDecorator.pipeline

 inside another pipeline decorated in the same way.

Ohh... so would it make sense to add "helper_functions" so that a function will be available in the step's context ?
Or maybe we need a new to support "standalone" decorator?! Currently to actually "launch" the function step, you have to call it from the "pipeline" main logic function, but, at least in theory, one could do without the Pipeline itself.....

Thank you WackyRabbit7 please feel free to remind me if it slips away during my night time (yes I do sleep , contrary to common belief :))

Your git execution needs this file, just like your machine does, to know where the server is and how to authenticate. You have to Manually pass it to your git action.

Many thanks 🙂

Hi WittyOwl57
That's actually how it works (original idea/design was borrowed from libclound), basically you need to create a Drive, then the storage manger will use it.
Abstract class here:
https://github.com/allegroai/clearml/blob/6c96e6017403d4b3f991f7401e68c9aa71d55aa5/clearml/storage/helper.py#L51
Is this what you had in mind ?

load_model will get a link to a previously registered URL (i.e. it search a model pointing to the specific URL, if it finds it, it will get you the Model object)

Hi PerfectChicken66

every X iterations and delete the older ones with

I have to ask, why not just overwrite the artifact? it is basically the same, no ?!

older ones with

delete_artifacts

from

Task

I think you are correct, when you delete the entire Task you can specify, delete artifacts, but it does not do that on delete_artifact 😞
You can manually do that with:
` task._delete_uri(task.artifacts["artifact"].url)
task.delete_artifact() ...

they are efs mounts that already exist

Hmm, that might be more complicated to restore, right ?

https://allegro.ai/clearml/docs/docs/examples/pipeline/pipeline_controller.html

seems it was fixed 🙂
MagnificentWorm7 thank you for noticing ! 🙏

You need trains-server support, so if trains v0.15 is working with older backend it will revert to "training" type

Hmm, I still wonder what is the "correct" answer for most people, is empty string in argparse redundant anyhow? will someone ever use it?

Maybe ?
https://stackoverflow.com/questions/45682010/docker-invalid-reference-format/51379726

I am thinking about just installing this manually on the worker ...

If you install them system wide (i.e. with sudo) and add agent.package_manager.system_site_packages then they will always be available for you 🙂

And then also use 

priority_optional_packages: ["carla"]

This actually means that it will always try to install the package clara first, but if it fails, it will no raise an error.

BTW: this would be a good use case for dockers, just saying :w...

ReassuredTiger98 I ❤ the DAG in ASCII!!!

port = task_carla_server.get_parameter("General/port")

This looks great! and will acheive exactly what you are after.
BTW: when you are done you can do :
task_carla_server.mark_aborted(force=True)And it will shutdown the Clara Task 🙂

p.s. any chance you can get me the nvidia driver version? I can't seem to find the one for v22 on amazon

maybe it can help 🙂

That sounds like success!

BTW: Full RestAPI reference here
https://allegro.ai/clearml/docs/rst/references/clearml_api_ref/index.html

I think this is the discussion you are after:
https://clearml.slack.com/archives/C01H5VAUZ8R/p1612452197004900?thread_ts=1612273112.002400&cid=C01H5VAUZ8R

HI QuizzicalDove0
I guess the reason is that the idea is integration is literally 2 lines, and it will take less time to execute the code on a system with working env (we assume there is one) then to configure all the git , python packages, arguments etc...
All that said you can create an experiment from code , using Task.import_task https://allegro.ai/docs/task.html#trains.task.Task.import_task

PompousBeetle71 is this ArgParser argument or a connected dictionary ?

I have one agent running on the machine. I also have only one task running. This

only

happens to us when we use pipelines

@<1724960468822396928:profile|CumbersomeSealion22> notice that when you are launching a pipeline you are actually running Two tasks, one is the "pipeline" itself (i.e. the logic) and one is the component in the pipeline (i.e. the step)
If you have one agent, I'm assuming what happens is the pipeline itself (the one that you launch on your machine)...

and I have no way to save those as clearml artifacts

You could do (at the end of the code
task.upload_artifact('profiler', Path('./fil-result/'))wdyt?

Thank you!

ConvolutedSealion94 if you do bash:
cd ~/work/repo/code/ git statuswhat are you getting ?

With pleasure, I'll make sure we officially release RC1 soon :)

PompousParrot44 the venv created in the docker always inherits form the docker system-wide packages, so in essence if you are using the same set pf python packages, nothing will get reinstalled.

Yep 🙂 but only in RC (or github)

Hi @<1691258563357315072:profile|ColorfulKitten60>
I think we need some context for this question 🙂