BTW: if you feel like pushing forward with integration I'll be more than happy to help PRing new capabilities, even before the "official" release

JitteryCoyote63 while it's running, could you give me a few details on the setup, maybe I can reproduce it.
Is it using pytorch distributed ?
Are all models uploaded to S3 ?
etc.

Hi NastyFox63
What do you mean not all of them are shown?
Do they have diff series/titles, are they plots or scalars ? How are you reporting them ?

Awesome ! thank you so much!
1.0.2 will be out in an hour

It was set to true earlier, I changed it to false to see if there would be any difference but doesn’t seem like it

I would actually just add:
Task.add_requirements('google.cloud')Before the Task.init call (Notice, it has to be before the the init call)

Hi SubstantialElk6
Could you test with the latest RC6 ?
pip install clearml==0.17.5rc6

Won't it be too harsh to have system wide restriction like that ?

Hi MagnificentSeaurchin79
This sounds like a deeper bug (of a sort), I think the best approach is to open a GitHub issue with some code that can reproduce this behavior, or at least enough information so that we could try to catch the bug.
This way we will make sure it is not forgotten.
Sounds good ?

So if everything works you should see "my_package" package in the "installed packages"
the assumption is that if you do:
pip install "my_package"
It will set "pandas" as one of its dependencies, and pip will automatically pull pandas as well.
That way we do not list the entire venv you are running on, just the packages/versions you are using, and we let pip sort the dependencies when installing with the agent
Make sense ?

This is only in the preview right ?

Ohh, sorry 🙂
:param run_pipeline_steps_locally: (default False) If True, run the pipeline steps themselves locally as a subprocess (use for debugging the pipeline locally, notice the pipeline code is expected to be available on the local machine)

So if I pass a function that pulls the most recent version of a Task, it'll grab the most recent version every time it's scheduled?

Basically you function will be called, that's it.
What I'm assuming is that you would want that function to find the latest Task (i.e. query based & filter based on project/name/tag etc), clone the selected Task and Enqueue it,
is that correct?

@<1577468638728818688:profile|DelightfulArcticwolf22>

How can I tell clearml-agent not to run pip install unless my requierments.txt file was changed.

the agent has built in cache, it will reuse the previous venv if nothing changed (cache local on the agent's machine).
Make sure this is line is not commented :
None

Notice the order here:
Task.add_requirements("tensorflow") task = Task.init(...)

SmarmySeaurchin8 could you test with the latest RC
pip install clearml==0.17.5rc2

Hi DefeatedCrab47
Not really sure, and it smells bad ...
Basically you can always use the TB logger, and call Task.init.

Should work, follow the backup process, and restore into a new machine:
None

I try to add it to ClearML Serving, but it call

forward

method by default

If this is the case, then the statement above is odd to me, if this is a custom engine, who exactly is calling " forward " ?
(in you code example you specifically call generate, as you should)

Is there a solution for that?

Hi DisturbedElk70
Well assuming you mount/sync the "temp" folder of the offline experiment to a storage solution, then have another process (on the other side), syncing these folders, it will work and you will get "real-time" updates 🙂
Offline Folder:
get_cache_dir() / 'offline' / task_id

Hi GiganticTurtle0
Sure, OutputModel can be manually connected:
model = OutputModel(task=Task.current_task()) model.update_weights(weights_filename='localfile.pkl')

Hi StaleButterfly40

but if I sync more than once I get a duplication of each line in log

Hmm.. let me check if we can "force" overwriting (it might require you to have a more stateful code for the sync process)

sometime we resume training

How would that work in offline mode? The offline process cannot sync with the backend... Are you saying you would like to get a new capability, "continue-offline-session" ?

StaleButterfly40 just making sure I understand, are we trying to solve the "import offline zip file/folder" issue, where we create multiple Tasks (i.e. Task per import)? Or are you suggesting the Actual task (the one running in offline mode) needs support for continue-previous execution ?

Hi RoughTiger69

but still get the semantics of knowing when an (external) file changed?

How would you know it changed?
This implies you have a way to verify hash, which means you download the data , no?

Hi DeliciousBluewhale87
You mean per Task? Is it reporting? Is it like the project overview?

Will the new fix avoid this issue and does it still requires the 

incremental

 flag?

It will avoid the issue, meaning even when incremental is not specified, it will work
That said the issue any other logger will be cleared as well, so, just good practice ...

From the 

logging

 documentation ...

Hmmm so I guess Kedro should not use dictConfig ?! I'm not sure on the exact use case, but just clearing all loggers seems like a harsh approach

https://github.com/allegroai/clearml-serving

Task.set_offline(True)
https://clear.ml/docs/latest/docs/references/sdk/task#taskset_offline

(BTW: draft means they are in edit mode, i.e. before execution, then they should be queued (i.e. pending) then running then completed)

Thanks CynicalBee90 I appreciate the discussion! since I'm assuming you will actually amend the misrepresentation in your table, let me followup here.
1.

SPSS license may be a significant consideration for some, and so we thought it was important to point this out clearly.

SPSS is fully open-source compliant unless you have the intention of selling it as a service, I hardly think this is any users consideration, just like anyone would be using mongodb or elastic search without think...

They inherit from one another, so it does make sense. Also the add_tags is on the "main" Task and not the backend parent