-_- why there isn't a link to source on the docs?

For example, this is one execution:

💪

I am noticing that the files are saved locally, is there any chance that the files are over-written during the run or get deleted at some point and then replaced?
Yes they are local - I don't think there is a possibility they are getting overwritten... But that depends on how clearml names them. I showed you the code that saves the artifacts, but this code runs multiple times from a given template with different values - essentially it creates like 10 times the same task with different param...

Legit, if you have a cached_file (i.e. exists and accessible), you can return it to the caller

I agree, so shouldn't it be if cached_file: return cached_file instead of if not cached_file: return cached_file

CostlyOstrich36

nope - I'm talking about here

https://github.com/allegroai/trains/blob/98ea965e6d0a39019b31e5fec146ab55da0d52e8/trains/storage/manager.py#L41

👍

Another thing I noticed now it happens on my personal computer, when I execute the same pipeline from the exact same commit with exact same data on another host it works without these problems

Thanks very much

Now something else is failing, but I'm pretty sure its on my side now... So have a good day and see you in the next question 😄

Do you have any idea as to why does that happen SuccessfulKoala55

👍

Searched for "custom plotly" and "log plotly" in search, didn't thinkg about "report plotly"

(I'm working with maria)

essentially, what maria says is when she has a script with uncomitted changes, when executing remotely, the script that actually runs on the remote machine is without the uncomitted changes

e.g.:
Her git status is clean, she makes some changes to script.py and executes it remotely. What gets executed remotely is the original script.py and not the modified version she has locally

Now I see the watermarks are 2gb

I guess the AMI auto updated

when spinning up the ami i just went for trains recommended settings

what should I paste here to diagnose it?

and also in the extra_vm_bash_script variables, I ahve them under export TRAINS_API_ACCESS_KEY and export TRAINS_API_SECRET_KEY

and then how would I register the final artifact to the pipelien? AgitatedDove14 ⬆

Wait, suddenly the UI changed to 0.16.1, seems like I was shown a cached page

` # Python 3.8.10 (default, Jun 2 2021, 10:49:15) [GCC 9.4.0]
clearml == 1.0.5
hyperopt == 0.2.5
matplotlib == 3.4.3
numpy == 1.21.2
pandas == 1.3.2
plotly == 5.3.0
python_dateutil == 2.8.2
scikit_learn == 0.24.2
statsmodels == 0.12.2
tqdm == 4.62.2

Detailed import analysis

**************************

IMPORT PACKAGE clearml

tasks/data_projection.py: 9

tasks/hp_optimization.py: 6

tasks/hpo_n_best_evaluation.py: 6

tasks/pipelines/monthly_predictions.py: 4

IMPORT PACKAGE hypero...

Anyway I checked the base task, and this is what it has in installed packages (seems like it doesn't list all the real packages in the environment)

This is what I meant should be documented - the permissions...

SuccessfulKoala55 ?

TimelyPenguin76 , this can safely be set to s3:// right?

👍

SuccessfulKoala55

I only want to save it as a template so I can later call it in a pipeline

Okay so that is a bit complicated

In our setup, the DSes don't really care about agents, the agents are being managed by our MLops team.
So essentially if you imagine it the use case looks like that:
A data scientists wants to execute some CPU heavy task. The MLops team supplied him with a queue name, and the data scientist knows that when he needs something heavy he pushes it there - the DS doesn't know nothing about where it is executed, the execution environment is fully managed by the ML...