Thanks Martin! 🙌

I need to fetch a dataset for some simple tests but since it doesn’t have credentials to the self-hosted server it wont find the dataset

Thanks for the info AgitatedDove14 !

I’ll show you what I have through PM!

On the server through the command line?

Hey AgitatedDove14 after playing around seems that if the callback filepath points to an hdf5 file it is not uploaded.

Sure! I enqueue the experiment from my local machine:
python -m src.train model=my_model loss=my_loss dataset=my_dataset

Then I go to the server and run the experiment and create a copy to run with a new model. On the copy, I go to the script path and modify it to be:
-m src.train model=my_other_model loss=my_loss dataset=my_dataset

The new experiment, even though the script path has my_new_model default, starts training using my_model .

I can also see ...

Hey AgitatedDove14 does this work for you?
` from argparse import ArgumentParser
from tensorflow.keras import utils as np_utils
from tensorflow.keras.datasets import mnist
from tensorflow.keras.layers import Dense
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.callbacks import ModelCheckpoint

import tensorflow as tf
from clearml import Task

class Linear(tf.keras.Model):
def init(self, in_shape=(784,), num_classes=10):
super().init()
self.l...

Yes! What env variables should I pass

Thanks SuccessfulKoala55 i’ll try this out!

Thanks so much AgitatedDove14 !

Thanks AgitatedDove14 ! seems to be subclassed model + extension

Thanks AgitatedDove14 🙌

AgitatedDove14 Thanks! Im trying to figure out how to create a minimum working example! I am also working with Hydra so that may be a thing. The extension is whats causing it to fail (haven’t figured out why).

If you try:
ModelCheckpoint('best_model.hdf5', save_best_only=True)does it work too?

Thanks AgitatedDove14 !

Managed to get:

clearml_agent: ERROR: Command '['/home/ramon/.clearml/venvs-builds/3.9/bin/python', '-m', 'pip', '--disable-pip-version-check', 'install', '-r', '/var/tmp/requirements_tb0x2i3j.txt', '--extra-index-url', '

 died with <Signals.SIGKILL: 9>.

while building the task with the id on the agent

For option 2 do I have to configure it on all agents or on the server?

AgitatedDove14 its on the checkpoint

I just want to retrieve the weights on a script that tests models I have trained in the past

Side note: When running src.train as a module the server gets the command as src and has to be modified to be src.train

Also, should I allow 8080 , 8008 , and 8081 on ingress and egress on GCP or is only egress enough?

Thanks Martin! I’ll keep checking 👌

Thanks SuccessfulKoala55 !

Yes AgitatedDove14 ! I’ll PM you

AgitatedDove14 Downloading a dataset would not be possible using this right? I want to be able to access the data just avoid reporting the experiment results

Yes Martin! I have a package installed from github but its using the pypi version

Awesome AgitatedDove14 Thanks a lot 🙌

It works perfectly! AgitatedDove14 There is something weird on my side 😢

SuccessfulKoala55 on both 8080 and 8008 I get: Safari can’t open the page http://<External IP>:80XX because Safari can’t establish a secure connection to the server http://<External IP>:80XX .