Reputation
Badges 1
533 × Eureka!UptightCoyote42 - How are these images avaialble to all agents? Do you host them on Docker hub?
cool, didn't know about the PAT
Wait but I don't want to execute it
` # define pipeline
pipe = clearml.PipelineController(
name=TASK_NAME,
project=PROJECT_NAME,
version='0.0.1',
add_pipeline_tags=False,
)
pipe.set_default_execution_queue('default')
Adding steps
pipe.add_step(name=f'{start_date_train}_{end_date_train}_choose_best',
base_task_project=CHOOSE_PROJECT_NAME,
base_task_name=CHOOSE_TASK_NAME,
parameter_override=params_override,
...
Okay so at the first part of the code, we define some kind of callback that we add to our steps, so later we can collect them and attach the results to the pipeline task. It looks something like this
` class MedianPredictionCollector:
_tasks_to_collect = list()
@classmethod
def collect_description_tables(cls, pipeline: clearml.PipelineController, node: clearml.PipelineController.Node):
# Collect tasks
cls._tasks_to_collect.append(node.executed)
@classmethod...
how do I run this wizard? is this wizard train's or aws's?
Is tehre anything specific about the logs we're looking for? Because if I just dumop them it will take me a while to see no sensitive data and naming is there
so basically - if she has new commits locally that werent pushed it won't work
But if she did not commit her latest changes, and now she enqueues - it will work?
If this includes scheduling through pipelines, in my opinion there should be an option to execute a pipeline without an agent. Sometimes for development I just want to execute a pipeline on my local machine just as I would a task...
I mean usually it would read if cached_file: return cached_file
its like ps
+ grep
together 😄
Okay, so if my python script imports some other scripts I've written - I must use git?
Maybe even a dedicated argument specifically for apt-get
packages, since it is very common to need stuff like that
I might, I'll look at the internals later cause at a glance I didn't really get the logic inside get_local_copy
... the if
there is ending with if ... not cached_file: return cached_file
which from reading doesn't make much sense
BTW is the if not cached_file: return cached_file
is legit or a bug?
Oh I get it, that also makes sense with the docs directing this at inference jobs and avoiding GPU - because of the 1-N thing
the worst part of debugging this is waiting for the docker to install tensorflow each time over and over again 😞
AgitatedDove14 this is stillnot fixed for me, even though I upgraded to server 1.1... Does the client require an update as well? Should I open an issue about this?
Cool - so that means the fileserver which comes with the host will stay emtpy? Or is there anything else being stored there?