@<1535793988726951936:profile|YummyElephant76>

Whenever I create any task the "uncommitted changes" are the contents of

ipykernel_launcher.py

, is there a way to make ClearML recognize that I'm running inside a venv?

This sounds like a bug, it should have the entire notebook there, no?

Change to add_missing_installed_packages=False, here, and see if you end up with git diff
https://github.com/allegroai/clearml/blob/1f82b0c4010799be6157f5c845c7f6ac48e71c0c/clearml/backend_interface/task/populate.py#L158

*Actually looking at the code, when you call Task.create(...) it will always store the diff from the remote server.
Could that be the issue?

To edit the Task's diff:
task.update_task(dict(script=dict(diff='DIFF TEXT HERE')))

I think so...

Thank you!

one thing i noticed is that it's not able to find the branch name on >=1.0.6x , while on 1.0.5 it can

That might be it! let me check the code again...

Thanks ShakyJellyfish91 ! please let me know what you come up with, I would love for us to fix this issue.

CooperativeSealion8

when it first asks me to enter my full name

Where? in the Web?

Hi @<1564785037834981376:profile|FrustratingBee69>

It's the previous container I've used for the task.

Notice that what you are configuring is the Default container, i.e. if the Task does not "request" a specific container, then this is what the agent will use.
On the Task itself (see Execution Tab, down below Container Image) you set the specific container for the Task. After you execute the Task on an Agent, the agent will put there the container it ended up using. This means that ...

Hi @<1729309120315527168:profile|ShallowLion60>
How did you create those credentials ?

Hi RipeGoose2
Just to clarify, the issue with the html stuck in cache is a UI, thing, basically the webapp needs to tell the browser not to cache the artifacts, it has nothing to do with how the artifacts are created.
Regardless we love improvements so feel free to mass around with the code and PR once you get something useful 😉
Specifically this is where the html conversion happens
https://github.com/allegroai/clearml/blob/9d108d855f784e1fe7f5691d3b7bf3be64576218/clearml/backend_in...

None

Glad to hear! 🎉

 I want to schedule bulk tasks to run via agents, so I'm running 

create

I see, that makes sense.

specially when dealing with submodules,

BTW: submodule diff should always get stored, can you provide some error logs on fail cases?

Before manually modifying the diff:
If you have local commits (i.e. un-pushed) this might fail the diff apply, in that case you can set the following in your clearml.conf
store_code_diff_from_remote: truehttps://github.com/allegroai/clear...

Good question 🙂
https://clear.ml/docs/latest/docs/clearml_agent#dynamic-gpu-allocation

The latest updated help will always be here as well 🙂
clearml-agent daemon --help

(Just a thought, maybe we just need to combine Kedro-Viz ?)

there is probably some way to make an S3 path open up in the browser by default

You should have a pop-up asking for credentials ...
Could you check that if you add the credentials in the profile page it works ?

that is because my own machine has 10.2 (not the docker, the machine the agent is on)

No that has nothing to do with it, the CUDA is inside the container. I'm referring to this image https://allegroai-trains.slack.com/archives/CTK20V944/p1593440299094400?thread_ts=1593437149.089400&cid=CTK20V944
Assuming this is the output from your code running inside the docker , it points to cuda version 10.2
Am I missing something ?

I guess this is from clearml-server and seems to be bottlenecking artifact transfer speed.

I'm assuming you need multiple "file-server" instances running on the "clearml-server" with a load-balancer of a sort...

Does StorageManager.upload and upload_artifact use the same methods?

Yes they both use StorageManager.upload

Is the only difference is task being async?

Two differences:
Upload being async Registering the artifact on the experiment. StorageManager will only upload, where as upload_artifact will make sure the file is registered as an artifact on the experiment, together with all of the artifacts properties.

Could you give an example of such configurations ?
(e.g. what would be diff from one to another)

Glue machine or K8S Worker machine?

The K8s worker machine.
You could also configure an ingest service as part of the template, so they always have an external port mapped into the port.

• At its simplest, this could just mean checking that all of the steps and the pipeline itself have completed successfully (by checking their “Task status”).If a pipeline step ends with "failed" status in the pipeline execution function an exception will be raised, if the exception is not caught, the pipeline itself will also fail

run

pipeline_script.py

which contains the pipeline code as decorators.

So in theory the following should actually work.
Let's assume you ...

How are you running it?

Hi WickedElephant66

Setting the pipeline controller with pipeline_execution_queue as None

is actually launching the pipeline controller on your "dev" machine, not sure why you have two of them?

ColossalDeer61 FYI all is fixed now 🙂

I can share some code

Please do 🙂

Hi JitteryCoyote63 ,
These properties are usually not available on the UI and are used internal, hence the lack of documentation. Regrading parent property, it will hold a parent Task.id (str) , that said it has no real effect on the Task itself. You can however search for Tasks with a specific parent ID (For examples, this is how the the hyper parameter class is using this property)

Well it is there, do you have it in your docker-compose as well?
https://github.com/allegroai/trains-server/blob/master/docker-compose.yml#L55

Hi @<1528908687685455872:profile|MassiveBat21>

However

no useful

template

is created for down stream executions - the source code template is all messed up,

Interesting, could you provide the code that is "created", or even better some way to reproduce it ? It sounds like sort of a bug? or maybe a feature support that is missing.

My question is - what is a best practice in this case to be able to run exported scripts (python code not made availa...

SuccessfulKoala55 please post here once the code is available in your pytorch_ignite 🙂