Hi CleanWhale17 let me see if I can address them all

Email Alert for finished Job(I'm not sure if it's already there).

Slack integration will be public by the end of the weekend 🙂
It is fully customization / extendable, I'll be happy to help .

DVC

Full dataset tracking is supported using the artifacts and the ability to integrate to any central storage (shared folders/ S3 / GS / Azure etc.)
From my experience, it is easier to work with artifacts from Data-Processing Tasks...

Hi QuaintJellyfish58

like we want set using UI but limited option on dropdown list

You mean like limit the ability of users to choose specific values? (so they do not mess things up?)

No, they're not in Tensorboard

Yep that makes sense

Logger.current_logger().report_scalar("test", test_metric, posttrain_metrics[test_metric], 0)

That seems like a great solution

GiddyTurkey39
BTW: you can always add the missing package via code:
Task.add_requirements('torch', optional_version)

We host in a private (self-hosed) git-Lab, which can only be cloned through the SSH, and we would like to import packages by compiling it from a public git-Hub (or by installing it using wheels with find-links).

Try removing force_git_ssh_protocol: true
If you do not provide user/pass , and assuming the repo link on the task is internal, hence SSH, it should leave links as they are (ssh for ssh http for http), this should solve the issue (I hope)

@<1523716917813055488:profile|CloudyParrot43> yes server upgrades deleted it 😞 we are redeploying a copy, should take a few min

Hmm so the Task.init should be called on the main process, this way the subprocess knows the Task is already created (you can call Task.init twice to get the task object). I wonder if we somehow can communicate between the sub processes without initializing in the main one...

sounds good, CheerfulGorilla72 could I ask you to open a github issue and suggest it? just so we do not forget ?

is it planned to add a multicursor in the future?

CheerfulGorilla72 can you expand? what do you mean by multicursor ?

Now I need to figure out how to export that task id

You can always look it up 🙂
How come you do not have it?

Hi FiercePenguin76
Is catboost actually using TB or is it just writing to .tfevent on its own ?

This is an odd error, could it be conda is not installed in the container (or in the Path) ?
Are you trying with the latest RC?

make sense ?

I also found that you should have a deterministic ordering

before

you apply a fixed seed

Not sure I follow ?

Hi MinuteWalrus85
This is great question, and super important when training models. This is why we designed a whole system to manage datasets (including storage querying, balancing data, and caching). Unfortunately this is only available in the paid tier of Allegro... You are welcome to https://allegro.ai/enterprise/ the sales guys.
🙂

You can always force it with environment variable CUDA_VERSION=10.1

Scenario 1 & 2 are essentially the same from caching perspective (the face B != B` means they have different caching hashes, but in both cases are cached).
Scenario 3 is the basically removing the cache flag from those components.

Not sure if I'm missing something.

Back to the @<1523701083040387072:profile|UnevenDolphin73>

From decorators - when the pipeline logic is very straightforward ...

Actually I would disagree, the decorators should be used when the pipeline Logic is not a D...

ok, yes, but this will install the package of the branch specified there.

Correct

So If im working on my own branch and want to run an experiment, I would have to manually put in the git path my current branch name.

When you say your own branch you mean local (i.e. not pushed to remote git repo) ?

VexedCat68 I think this is the issue described here:
https://github.com/allegroai/clearml/issues/491
Can you test with the latest RC:
pip install clearml==1.1.5rc1

it is using the wrong URL - an https:// instead of an s3:// so it can't upload the preprocess.py file

What do you see as the link in the UI for that specific model ?
What is the full model add command ?

I'm a bit confused between the distinction / how to use these appropriately --

Task.init

does not have

repo

/

branch

args to set what code the task should be running.

It detects it automatically at run time 🙂 based on what is actually being used

My ideal is that I do exactly what

Task.create

does, but the task only goes into the pipeline section rather than making a new one in the experiments section.

Do y...

Hi JumpyPig73

import data from old experiments into the dashboard.

what do you mean by "old experiments" ?

(This is why we recommend using pip, because it is stable and clearml-agent takes care of pytorch/cuda verions)

I'll make sure we fix it

Yes clearml is much better 🙂
(joking aside, mlops & orchestration in clearml is miles better)
CheerfulGorilla72 What are you looking for?

Hi @<1523704157695905792:profile|VivaciousBadger56>
No these are 3 different ways of building pipelines.
Creating from decorators is recommended when each component can be easily packages into a single function (every function can have an accompanying repository).
Here the idea it is very easy to write complex execution logic, basically the automagic does serialization/deserialization so you can write pipelines like you would code python.

Creating from Tasks is a good match if you need to ...

Hi @<1691620877822595072:profile|FlutteringMouse14>
Yes, feast has been integrated by at least a couple if I remember correctly.
Basically there are two ways offline and online feature transformation. For offline your pipeline is exactly what would be recommended. The main difference is online transformation where I think feast is a great start

web-server seems okay, could you send the logs from the api-server?
Also if you can, the console logs from your browser, when you get the blank screen. Thanks.

Hi @<1549202366266347520:profile|GorgeousMonkey78>

how do I integrate sagemaker with clearml ,

you mean to launch an experiment, or just to log it?

Give me a minute