The fact is that I use docker for running clearml server both on Linux and Windows.

My question was on running the agent, is it running with --docker flag, i.e. docker mode

Also, just forgot to note, that I'm running clearml-agent and clearml processes in virtual environment - conda environment on Windows and venv on Linux.

Yep that answers my question above 🙂

Does it make any sense to chdnge 

system_site_packages

 to 

true

 if I r...

Hi ZanyPig66

I used tensorboard as clearml claims to auto-capture tensorboard outputs, but it was a no go.

The auto TB logging should work out of the box, where is it failing ?

Also,
task = Task.current_task()Why aren't you using Task.init in the original script?
The idea is that you run your code on your machine (where the environment works), ClearML auto detects code + python packages + args etc.
Then you clone it in the UI and launch it on a remote machine.
What am I missing ...

Yep this will work. BTW check the new pipeline it might have a more flexible solution
https://github.com/allegroai/clearml/blob/master/examples/pipeline/full_custom_pipeline.py

Did you change the commit ID ?

Hi @<1571308003204796416:profile|HollowPeacock58>
could you share the full log ?

ReassuredTiger98 oh wow I did not realize you actually call importlib to import your libraries (any reason not to call import ?)
And yes, I think we will miss it as the package analysis is actually static text analysts of the code

I "think" the IAM should only have the ability to create an EC2 instance (querying instances is done through the trains platform)

do I still need to specify a OutputModel

No need, only if you want to upload a local model file (but I assume in this case, no new model is created)

If you want to quickly test it:
pip install clearml-agent
Then assuming Task id is aabbcc
Run
clearml-agent execute --id aabbccYou should be able to trace if the package was installed

ResponsiveHedgehong88 so I would suggest using execute_remotely in your code, basically you start locally you make sure everything is passed as intended, then from within the code you call task.execute_remotely(...) which will stop the current process and enqueue the Task on the selected queue for the agent to execute.
https://github.com/allegroai/clearml/blob/0397f2b41e41325db2a191070e01b218251bc8b2/examples/advanced/execute_remotely_example.py#L127
This way you can both easily test...

Hi DilapidatedDucks58 just making sure, the link is pyrorch nightly artifactory? Or is it a direct link to the package? Reason for asking, I was not aware they have proper artifactory... When the task runs the trains agent will update the installed packages with all the installed packages it used. Could you verify you have the correct version?

Regarding the extra files, you are correct, the docker container is reset every run, so they will get lost. What are those files for? Could you add ...

The configuration tab -> configuration objects -> pipeline is empty

That's the reason it is doing nothing 😞
How come it is empty if you Cloned the local one?

So a bit of explanation on how conda is supported. First conda is not recommended, reason is, is it very easy to create a setup on conda that is un-reproducible by conda (yes, exactly that). So what trains-agent does, it tries to install all the packages it can first with conda (not one by one, because that will break conda dependencies), then the packages that it failed to install from conda, it will install using pip.

So there is a hack for it:
CLEARML_OFFLINE_MODE=1 python3 my_main.pyWhich is the same as calling Task.set_offline
Then inside the code After the Task.init call:
` task = Task.init(...)

not sure what the if here is?!

Task.debug_simulate_remote_task(task_id="offline-1") `This will make things act as if this is running remotely , i.e. your logic Task.running_remotely() will be called.
Do notice that in remote mode, all the arguments / data is read from the clearml-server into the cod...

It should have been:
output_uri="s3://company-clearml/artifacts/bethan/sales_journeys/artifacts/examples/load_artifacts.f0f4d1cd5eb54795b11508dd1e739145/artifacts/filename.csv.gz/filename.csv.gz

what if cleanup service is launched using ClearML-Agent Services container

The easiest is to use the container args and pass the AWS credentials as env variables:
-e AWS_ACCESS_KEY_ID=abcd -e ....Make sense ?

or by trains

We just upload the image as is ... I think this is SummaryWriter issue

ReassuredTiger98

It seems like clearml is not able to fetch the dependencies correctly when 

importlib

 is used.

If you have an example please let me know we'll try to fix it :)

Is it possible to read the dependencies manually from a conda environment.yml?

You can set detect_with_conda_freeze: true in clearml.conf, it will just use the entire conda env
https://github.com/allegroai/clearml/blob/28b85028fe4da3ab963b69e8ac0f7feef73cfcf6/docs/clearml.conf#L170

in Your Additional ClearML Configuration (which is basically clearml.conf configuration)
Add the following:
environment { GOOGLE_APPLICATION_CREDENTIALS="~/gs.cred" } files { gsc { contents: "<this is your GCP storage credentials file>" path: "~/gs.cred" } }Reference:
https://github.com/allegroai/clearml-agent/blob/a5a797ec5e5e3e90b115213c0411a516cab60e83/docs/clearml.conf#L421
https://github.com/allegroai/clearml-agent/blob/a5a797ec5e5e3e90b115213c0411a...

DeliciousSeal67
are we talking about the agent failing to install the package ?

Hi SubstantialElk6

Unicodeencodeerror:'ascii' codec can't encode characters in position 296-297: ordinal not in range (128) (edited)

I'm assuming this is the usual UTF8 missing from the container.
Can you try to launch it with PYTHONIOENCODING=utf-8 ?

In the "installed packages" section you should have "nvidia-dali-cuda110" In the agent's clearml.conf you should add:extra_index_url: [" ", ]https://github.com/allegroai/clearml-agent/blob/master/docs/clearml.conf#L78
Should solve the issue

If you want to rename it (any pipeline), click on the "Full details" in the "Run Info" (right hand side panel), then in the full detail of the Pipeline Task you will be able to rename the pipeline execution
(Is renaming useful? should we add a right click to rename ?)

HealthyStarfish45 what exactly did you have in mind, in terms of the widget ?

Should work, follow the backup process, and restore into a new machine:
None

last iteration is no reset and I still have a gap in my scalars

Hmm is this reproducible ? can you check with the latest clearml version (1.10.3) ?
btw: I'm assuming continue_last_task=0

I think I found the issue, the fact the agent is launching it causes it to ignore the "overridden" set_initial_iteration

looks like a great idea, I'll make sure to pass it along and that someone reply 🙂

or at least stick to the requirements.txt file rather than the actual environment

You can also for it to log the requirements.txt with
Task.force_requirements_env_freeze(requirements_file="requirements.txt") task = Task.init(...)

Correct 🙂

Okay that seems to explain it. Now the question is why it installed it in the wrong place.