nvidia/cuda:10.1-base-ubuntu18.04

I prefer we debug on my machine (tell me what you want to check) than create a snippet

Yes, I have the controller task in hand

I don't think the problem is setting that variable, I think it has something to do with it but not that obvious... Because it did work for me in the past, since then we docker-compose up/downed a few times, changed some other things etc... Can't figure out what made it get to this point

to fix it, I excluded this var entirely from the docker-compose

so it is not defined

SuccessfulKoala55

Not sure I understand, if i run pipe.start_locally(run_pipeline_steps_locally=True|False) what is the difference betwee ntrue and false? assuming I want to execute locally

I did, but I only find task name, I'm looking to filter by the project

and we want to continue xecuting locally

Hahahah thanks for the help SuccessfulKoala55 & CostlyOstrich36

I really do feel it would be a nice to have the ability to easily configure the Cleanup Service to cleanup only specific projects / tasks as its a common use case to have a project dedicated for debugging and alike

Will try this out and report

thx TimelyPenguin76
skimming over this, I can't find how to filter by project name or something similar

I only found Project ID, which I'm not sure what this refers to - I have the project name

Gotcha

I showed you this phenomenon in the UI photos in the other thread

what does project ID mean?

glad I managed to help back in some way

Oh I get it, that also makes sense with the docs directing this at inference jobs and avoiding GPU - because of the 1-N thing

For example, this is one execution:

I'm saying that because in the task under "INSTALLED PACKAGES" this is what appears

moreover, in each pipeline I have 10 different settings of task A -> Task b (and then task C), each run 1-2 fails randomly

if I called .start_locally

can you tell me which API call exactly are you using for spinning up? I would like to debug and try to use boto3 myself in order to spin up an instance, so I can understand where the problem is coming from

This is the pip freeze of the environment I don't know why it differs from what the agent has... the agent only has a subset of these google libs

Okay martin, I'll try that

Do you need other part?

I don't have ifconfig

and also in the extra_vm_bash_script variables, I ahve them under export TRAINS_API_ACCESS_KEY and export TRAINS_API_SECRET_KEY

👍