Hi DeliciousBluewhale87
Yes that should have worked, can you verify the task status ?
Print(Task.get_task(...).get_status())

btw:
If you need to access it, just bash into the running docker
docker exec -it <container_name> /bin/bash

I can probably have a python script that checks if there are any tasks running/pending, and if not, run docker-compose down to stop the clearml-server, then use boto3 to trigger the creating of a snapshot of the EBS, then wait until it is finished, then restarts the clearml-server, wdyt?

I'm pretty sure there is a nice way, let me check soemthing

. Are there any option to remove the example projects?

So sorry just realized I missed your message
Yes, but I'm not sure it will have an effect, see here

why the memory usage of the elastic search still persist on 32 gb after removing experiments?

did you restart the server after removing the experiments?

5 seconds will be a sleep between two consecutive pulls where there are no jobs to process, why would you increase it to a higher pull freq ?

The task pod (experiment) started reaching out to an IP associated with malicious activity. The IP was associated with 1000+ domain names. The activity was identified in AWS guard duty with a high severity level.

BoredHedgehog47 What is the pod container itself ?
EDIT:
Are you suggesting the default "ubuntu:18.04" is somehow contaminated ?
https://hub.docker.com/layers/library/ubuntu/18.04/images/sha256-d5c260797a173fe5852953656a15a9e58ba14c5306c175305b3a05e0303416db?context=explore

https://github.com/allegroai/clearml-serving

ReassuredTiger98 I can verify the code snippet reproduces the issues with packages missing from "installed package".
If you feel this is important, please open a GitHub issue.
Also, you can manually add packages:
Task.add_requirements('package_name_here', 'optional version here')
So when you manually load the package you can make sure it will be listed, do remember to call it before the Task.init call.

ConfusedPig65 could you send the full log (console) of this execution?

SubstantialElk6 I know they have full permission control in the enterprise edition, if this is something you need I suggest you contact http://allegro.ai 🙂

WackyRabbit7 I guess we are discussing this one on a diff thread 🙂 but yes, should totally work, that's the idea

Hi @<1593413673383104512:profile|MiniatureDragonfly17>
These are the specific model input/output layers name.
The way Triton analyses PyTorch model is usually
input__0 then input__1 for the input layers and output__0 and so on for the results:
You can see an example here:
None

--input-size 1 28 28 --input-name "INPUT__0" --input-type float32    --output-size -1 10 --output-name "OUTPUT__0" --outpu...

The package is just subdir by the way. So it should not be in installed packages anyways, right?

Correct, also when the agent is spinning the code it will automatically add the root of the git repository to the pythonpath so you should be able to load the package.

ReassuredTiger98 maybe we should add an option to send a text next to the abort?
(Actually it is just a matter of passing the argument)
wdyt?

My pleasure 🙂

Hi PerfectChicken66

every X iterations and delete the older ones with

I have to ask, why not just overwrite the artifact? it is basically the same, no ?!

older ones with

delete_artifacts

from

Task

I think you are correct, when you delete the entire Task you can specify, delete artifacts, but it does not do that on delete_artifact 😞
You can manually do that with:
` task._delete_uri(task.artifacts["artifact"].url)
task.delete_artifact() ...

ShallowGoldfish8 I believe it was solved in 1.9.0, can you verify?
pip install clearml==1.9.0

You are correct, the agent will clone the git and install the requirements, as written in the task installed packages section. Regrading the git branch, notice it will pull the specific commit id as stated in the execution section, and it will apply any uncommitted changes. You can edit the execution section and change the commit to the latest in a specific version (you should probably also clear the uncommitted changes of you do that)

Hi CleanPigeon16
Yes there is, when you are cloning the pipeline in the UI, go to the Configuration/Pipeline/continue_pipeline and change it to True

Hi GreasyPenguin14
Quick question, any reason not to use a 2D scatter ? or a histogram (or any other non time-series plot)?

I did not start with python -m, as a module. I'll try that

I do not think this is the issue.
It sounds like anything you do on your specific setup will end with the same error, which might point to a problem with the git/folder ?

You can disable it with:

Task.init('example', 'train', auto_connect_frameworks={'pytorch': False})

owning the agent helps, but still it's much better if the credentials don't show up in logs,

They are not, they are always filtered out,

• how does force_git_ssh_protocol help please? it doesn't solve the issue of the agent simply not having accessIt automatically maps the host .ssh into the container, so that git can use SSH to clone.
  What exactly is not working?
  and how are you configuring it?

HI PlainSquid19 could you add a bit more information? Are you running trains-agent ? is it in docker/venv mode ? what's the trains/trains-agent/trains-server versions ?

I think EmbarrassedSpider34 is correct.
When you pass the requirements to clearml-task, actually the agent depending on how it was configured (conda / pip) will do the installation.
That said, maybe it is worth adding support to provide the env.yml in the CLI ?
(Notice that adding specific channels needs to be configured on the agent, they are not stored per Task)
AlertCamel57 wdyt?

PlainSquid19 No worries 🙂
btw: If you could see if the mangling of workings / script path happens with the latest trains, that will be appreciated, because if you were running the script in the first place from "stages/" then the trains should have caught it ...

what do you say that I will manually kill the services agent and launch one myself?

Makes sense 🙂

An example for something like spacy would be useful for the community.

That awesome, any chance you can PR something? (no need for it to be perfect, we can take it from there)

we made two tb versions of / task and wrote in parallel.
And I wanted to know if it is possible here as well.

Basically you will have different series (based on the TB log file) on the same graph so you can compare 🙂 all automatically

Thank you AttractiveWoodpecker16 !
Removing the uncommitted changes so that you can launch it from an agent? Or is it visual only?