Thanks!
Hmm from here : None
Could it be you do not have privileges to the resource, or that you did not provide credentials ?
Did that autoscaler work before ?

Hi PompousBeetle71 , what exactly is the scenario / problem we are trying to solve ?

Hi @<1523701083040387072:profile|UnevenDolphin73>

How can I ensure tasks in a pipeline have the same environment as the pipeline itself?
...
but the tasks (executed remotely) do not use that same environment?

Just verifying, we are talking about pipeline decorators?

We also wanted this, we preferred to create a docker image with all we need, and let the pipeline steps use that docker image

You can specify the docker on the decorator itself:
[None](https://github.com/allegroai...

If you use this one for example, will the component have pandas as part of the requirement
None

def step_two(...):
    import pandas as pd
    # do stuff

If so (and it should), what's the difference, where is "internal.repo " different from pandas ?

None
This example ?

How are you setting it up?

Hi PompousBeetle71 , Trains will log all the torch.save call, I'm assuming they do not actually use it for the rest of the files on that folder.
If you like to share a code snippet we could see if we could auto-magically log it You could use artifacts and store the entire folder. It will zip it an upload it. Then you can reuse it from other experiments. https://allegro.ai/docs/task.html?highlight=artifact#trains.task.Task.upload_artifact
Example:
` task.upload_artifact('transformer', './my_...

Hi SteadyFox10 , unfortunately trains-agent currently supports only docker as a container solution (I guess they became the de-facto standard)
That said, there is the option of virtual environment, where the trains-agent installs everything inside a newly created virtual environment. That actually makes it quite easy to expand to other use cases. Essentially the docker option will spin a docker install trains-agent inside the docker and run it execute command.
Do you fee l...

We should probably add (set_task_type :))

Hi JitteryCoyote63
The NVIDIA_VISIBLE_DEVICES is set automatically for the process the trains-agent spins, so from your code, it is transparent, you can only "see" GPU 0.
(Obviously not using docker you can forcefully change the OS environment in runtime, but you should avoid that ;))

ShallowCat10 try something similar to this one, due notice that it might take a while to get all the task objects, so I would start with a single one 🙂

`
from trains import Task
tasks = Task.get_tasks(project_name='my_project')
for task in tasks:
scalars = task.get_reported_scalars()
for x, y in zip(scalars['title']['original_series']['x'], scalars['title']['original_series']['y']):
task.get_logger().report_scalar(title='title', series='new_series', value=y, iteration=...

PS. I just noticed that this function is not documented. I'll make sure it appears in the doc-string.

Check the log usually ~/.trains/venv/...

Hi PunyGoose16 ,
I think the website is probably the easiest 🙂
https://clear.ml/contact-us/
I think they get back to quite quickly

That is odd...

PompousBeetle71 so basically exclude parameters that are considered "local" only, so that other people will not accidentally use them?

WickedGoat98 if this is the case, you can check this example. Same idea only "manual":
https://github.com/allegroai/trains/blob/master/examples/automation/task_piping_example.py

Hmmm, that actually connects with something we were thinking about: introducing sections to the hyper parameters. This way we could easily differentiate between the command line arguments and other types of parameters. DilapidatedDucks58 what do you think?

Hi MagnificentPig49 unfortunately it's only in the trains-server docker, we are working on making it "presentable" and uploading it to it's repo.
It's written in Angular (v8 I think). Do you want to help out, it will definitely incentive the guys to tidy up the code and upload it :)

@<1523701099620470784:profile|ElegantCoyote26> what's the target upload? also how come you are uploading a local file and auto deleting it, and then uploading the same one as artifact ?

Hi WackyRabbit7 ,
Yes we had the same experience with kaggle competitions. We ended up having a flag that skipped the task init :(
Introducing offline mode is on the to do list, but to be honest it is there for a while. The thing is, since the Task object actually interacts with the backend, creating an offline mode means simulation of the backend response. I'm open to hacking suggestions though :)

Hi PompousParrot44
So do you mean something like:
` task_model_a = Task.get('id_a')
task_model_b = Task.get('id_b')

model_a_file = task_model_a.models['output][-1].get_local_copy()

model_b_file = task_model_b.models['output][-1].get_local_copy() `

.I am using pipeline from tasks method and not pipeline from decorator.

Wait I'm confused nowm if this is a pipeline from Tasks then the Tasks themselves should have clearml in the "installed packages", no? and if they do not, how were they created?

hmmm, somehow I have a bed feeling about it... Could you check the log, it should say something like "Collecting torch==1.6.0.dev20200421+cu101 from https://"
It should be right at the top of the installation. What do you have there?

Hi @<1523704757024198656:profile|MysteriousWalrus11>

"parents": [

  "step_two", 

  "step_four" 

], 

Seems like step 5 depends on steps 2+4 , how did you create it? what did the console say ?
Could it be your not actually passing any output from step3 ? how is it dependent on it ?

RestAPI refrence:
https://allegro.ai/docs/references/trains_api_ref/trains_api_ref.html
https://allegro.ai/docs/references/trains_api_ref/trains_api_ref.html

if I use 

report_image

 can I get a URL to it somehow?

Let me check ...

HealthyStarfish45 this sounds very cool! How can I help?

docker mode. they do share the same folder with the training data mounted as a volume, but only for reading the data.

Any chance they try to store the TensorBoard on this folder ? This could lead to "No such file or directory: 'runs'" if one is deleting it, and the other is trying to access, or similar scenarios

(I am not an expert on UI to be honest)

Same here 🙂 lol

we can implement this externally

What do you mean by that?