Hi CleanWhale17 let me see if I can address them all

Email Alert for finished Job(I'm not sure if it's already there).

Slack integration will be public by the end of the weekend 🙂
It is fully customization / extendable, I'll be happy to help .

DVC

Full dataset tracking is supported using the artifacts and the ability to integrate to any central storage (shared folders/ S3 / GS / Azure etc.)
From my experience, it is easier to work with artifacts from Data-Processing Tasks...

model upload and registration i should pass something like

'xgboost': False

or

'xgboost': False, 'scikit': False

?

Exactly! which framework are you using ?

about 2, I refer to the names of the models.

Hmm that is a good point to test, usually this is based on the Task name (I think), so if the Task name contains the HPO params in the name it should be the same on the model name. Do you see the HPO params on the Task name ? Should we open a Gi...

TroubledHedgehog16 generally speaking you can expect about 10 api calls per minute if you have many reports, and about 3 per minute on low report. We just optimized the sdk so in cases there are lots of consequential reports they are better batched, I would recommend the latest RC

Wait I might be completely off.
Is this line "hangs" ?

task.execute_remotely(..., exit_process=True)

Hi TrickyRaccoon92
Yes please update me once you can, I would love to be able to reproduce the issue so we could fix for the next RC 🙂

When is clearml-deploy coming to the open source release?

Currently available under clearml-serving (more features are being worked on, i.e. additional stats and backends)
https://github.com/allegroai/clearml-serving

Latest RC 🙂

Yes this is a misleading title

PompousParrot44 please try to reply on the thread, so we do not create a mess in the main channel 🙂
What's the "working directory" in the execution section? Do you have package "test" in the installed packages?

BTW: you can always set different config files by with an environment variable:
CLEARML_CONFIG_FILE="path/to/cobfig/file

👍

Correct

Nice 🙂

Hi RipeGoose2
I think it "should" take of uploading the artifacts as well (they are included in the zip file created by the offline package)
Notice that the "default_output_uri" on the remote machine is meaningless as it stored them locally anyhow. It will only have an effect on the machine that actually imports the offline session.
Make sense ?

That should not be complicated to implement. Basically you could run 'clearm-task execute --id taskid' as the sagemaker cmd. Can you manually launch it on sagemaker?

I mean this blob is then saved on the fs

It can if you do:
temp_file = task.connect_configuration('/path/to/config/file', name='configuration object is a config file')Then temp_file is actually a local copy of the text coming from the Task.
When running in manual mode the content of '/path/to/config/file' is stored on the Task When running remotely by the agent, the content from the Task is dumped into a temp file and the path to the file is returned in temp_file

Maybe combining the two, with an unload gRPC api we could have that ability moved to the "preprocessing" logic, wdyt?

okay let me check...

MysteriousBee56 what do you mean by "local repository"?
Like no git server, or local commit before pushing it ?

Hi OddShrimp85
If you pass 'output_uri=True' to task init, it will upload the model automatically, or as you said manually with outputmodel class

It seems the code is trying to access an s3 bucket, could that be the case? PanickyMoth78 any chance you can post the full execution log? (Feel free to DM so it won't end up being public)

I'm trying to achieve a workflow similar to the one

You mean running everything on a single machine (manually)?

at means I need to pass a single zip file to 

path

 argument in 

add_files

 , right?

actually the opposite, you pass a folder (of files) to add_files. Then add_files remembers the files location (and pre calculates the hash of the files content). When you call upload it will actually compress the files that changed into a zip file (or files depending on the chunk size), and upload the files to the destination (as specified in the upload call...

Any reason not to ?

ElegantCoyote26 It means we need to have a keras logger that logs everything to trains, then we need to hook it automatically.
Do you feel like PR-ing the logger (the hooking I can take care of 🙂 )?

so all models are part of the same experiment and has the experiment name in their name.

Oh that explains it, (1) you can use the model filename to control the model name in clearml (2) you can disable the autologging and manually upload the model, then you can control the model name
wdyt?

PanickyMoth78 RC is out
pip install clearml==1.6.3rc1🤞

Thanks OutrageousGiraffe8
Any chance you can expand the example code to be a fully a reproducible toy code? (I would really like to make sure we fix it)

Hi MiniatureShells8
The torch.save triggers the model creation.
If you are using the same filename, then the same model in the system will be used.
New filenames will create new models.
What exactly is your use case ?

there is probably some way to make an S3 path open up in the browser by default

You should have a pop-up asking for credentials ...
Could you check that if you add the credentials in the profile page it works ?