Okay Jake, so that basically means I don't have to touch any server configuration regarding the file-server on the trains server. It will simply get ignored and all I/O initiated by clients with the right configuration will cover for that?

🤞

no need to do it again, I ahve all the settings in place, I'm sure it's not a settings thing

So just to correct myself and sum up, the credentials for AWS are only in the cloud_credentials_*

I'm using ip address show

btw my site packages is false - should it be true? You pasted that but I'm not sure what it should be, in the paste is false but you are asking about true

So regarding 1, I'm not really sure what is the difference

When running in docker mode what is different the the regular mode? No where in the instructions is nvidia docker a prerequisite, so how exacly will tasks on GPU get executed?

I feel I don't underatand enough of the mechanism to (1) understand the difference between docker mode and not and (2) what is the use casr for each

the curl refers to the master

the output above is what the agent has as it seems... obviously on my machine I have it installed

I'll share some screenshots

awesome martin, thanks

its like ps + grep together 😄

I only found Project ID, which I'm not sure what this refers to - I have the project name

In the larger context I'd look on how other object stores treat similar problems, I'm not that advanced in these topics.

But adding a simple force_download flag to the get_local_copy method could solve many cases I can think of, for example I'd set it to true in my case as I don't mind the times it will re-download when not necessary as it is quite small (currently I always delete the local file, but it looks pretty ugly)

even if it is so

TimelyPenguin76 if I build a custom image, do I have to host it on dockerhub for it to run on the agent? If not how do I make the agent aware of my custom image?

SuccessfulKoala55 The simplest thing i can think of is on Task.execute_remotely to be able to append ot the docker_init_bash_script

That was the issue then

So could you re-explain assuming my piepline object is created by pipeline = PipelineController(...) ?

iwhat if

(it works now, with 20 GB)

Hahahah thanks for the help SuccessfulKoala55 & CostlyOstrich36

I really do feel it would be a nice to have the ability to easily configure the Cleanup Service to cleanup only specific projects / tasks as its a common use case to have a project dedicated for debugging and alike

and the machine I have is 10.2.

I also tried nvidia/cuda:10.2-base-ubuntu18.04 which is the latest

It works!

I want to collect the dataframes from teh red tasks, and display them in the pipeline task

I assume we are talking about the IP I would find here right?
https://www.whatismyip.com/

this means that you supply the machine to execute the actions specified

I know tasks have a name and an ID - but projects?

Okay, looks interesting but actually there is no final task, this is the pipeline layout

💪