Hi CluelessElephant89

hey guys, I believe 

clearml-agent-services

 isn't necessary right?

Generally speaking, yes you are corrected 🙂
Specifically, this is the "services" queue agent, running your pipeline logic, services etc.
But it is not a must to get the server to work, and you can also spin it on a different host

I remember there were some issues with it ...

I hope not 😞 Anyhow the only thing that does matter is the auto_connect arguments (meaning if you want to disable some, you should pass them when calling Task.init)

It seems like you are correct, everything should just work. Are you still getting the error? What's the clearml agent version?

It takes 20mins to build the venv environment needed by the clearml-agent

You are Joking?! 😭
it does apt-get install python3-pip , and pip install clearml-agent, how is that 20min?

https://clear.ml/docs/latest/docs/references/sdk/task#taskimport_offline_session

Hi @<1533620191232004096:profile|NuttyLobster9>
base_task_factory is a function that gets the node definition and returns a Task to be enqueued ,
pseudo code looks like:

def my_node_task_factory(node: PipelineController.Node) -> Task:
  task = Task.create(...)
  return task

Make sense ?

Hi PompousParrot44
You can check the cleanup service example.
It sleeps for 24 hours then spins up and does its thing.
You can always launch this service tasks on the services queue, its purpose is to run those services on the trains-server as additional CPU services. They will also be registered as service nodes, so you have visibility into which service is running.
In order to clone a task and wait for its completion.
Use the TrainsJob https://github.com/allegroai/trains/blob/65a4a...

Lol yeah Hydra is great. Notice you still have the ability to override Hydra from the UI so you really have the best of the two worlds

Full markdown edit on the project so you can create your own reports and share them (you can also put links to the experiments themselves inside the markdown). Notice this is not per experiment reporting (we kind of assumed maintaining a per experiment report is not realistic)

Thanks HelpfulHare30 , I would love know know what you find out, please feel free to share 🙂

UnsightlyShark53 Awesome, the RC is still not available on pip, but we should have it in a few days.
I'll keep you posted here :)

Ohh if this is the case, you might also consider using offline mode, so there is no need for backend
https://clear.ml/docs/latest/docs/guides/set_offline#setting-task-to-offline-mode

fyi: hot fix for 1.3.0 (smoothing graphs) was just released see v1.3.1

I am actually considering rolling back to 1.1.0,

Can you share why?
JitteryCoyote63 notice from the release notes of 1.2:

Important Note!
This release requires a MongoDB migration from previous versions. Please see

for more information.

I'm not sure you can downgrade that easily ...

SpotlessFish46 unless all the code is under "uncommitted changes" section, what you have is a link to the git repo + commit id

So on the ec2 instance (with the agent running), just install prior to running the agent:
apt-get install poppler-utils

See example here:
https://demoapp.trains.allegro.ai/projects/97f6b5b53a0243c196d6f49c221cbdca/compare-experiments;ids=cdc2cc156ae042f08dab2b66756f468a,0aa6737817d0408ba22090a8cb076fdd/hyper-params/graph?hyper-params=graph

Hi @<1545216070686609408:profile|EnthusiasticCow4>
hmm this seems odd, and definitely looks like a bug, please report on GH 🙏

I prepared my own image and want use this venv

No worries, it creates a "transparent" venv, it uses everything from the docker (the penalty of create a new venv is negligible 🙂 , you end up with the exact same set of packages)

Hi VirtuousFish83
Apologies for the documentation in the docs 🙂 It sounds complicated but actually should be relatively simple. Based on what I understand, you already have the server setup and you code integrated. The question is "can you see an experiment in the UI"? If you do, then you can right click it, clone the experiment , edit parameters and send for execution (enqueue). If the experiment is not in the UI you can either (1) run the code with the Task.init call, it ill automatica...

but I belive it should have work with 0.14.1 as well

Correct

It seems to follow a structure specific to clearml,

Actually plotly.js 🙂

Can you also make sure you did not check "Disable local nachine git detection" in the clearml PyCharm plugin?

I'm assuming your are looking for the AWS autoscaler, spinning EC2 instances up/down and running daemons on them.
https://github.com/allegroai/clearml/blob/master/examples/services/aws-autoscaler/aws_autoscaler.py

https://clear.ml/docs/latest/docs/guides/services/aws_autoscaler

So why is it trying to upload to "//:8081/files_server:" ?
What do you have in the trains.conf on the machine running the experiment ?

Hi GrievingTurkey78
task.models['output'][-1] should return the last stored model.
What do you have under under task.models['output'][-1].url

Documentation:
https://allegro.ai/clearml/docs/rst/references/clearml_python_ref/model_module/model_outputmodel.html?highlight=model#model-outputmodel

with ?

                     multipart: false
                     secure: false

If so, can you post here your aws.s3 section of the clearml.conf? (of course replacing the actual sensitive information with *s)

Hmmm, that actually connects with something we were thinking about: introducing sections to the hyper parameters. This way we could easily differentiate between the command line arguments and other types of parameters. DilapidatedDucks58 what do you think?

MysteriousBee56 I would do Task.create()
you can get the full Task internal representation with task.data
Then call task._edit(script={'repo': ...}) to edit/update all the Task entries.
You can check the dull details of the task object here: https://github.com/allegroai/trains/blob/master/trains/backend_api/services/v2_8/tasks.py#L954
BTW: when you have a sample script working, consider PR-ing it, I'm sure it will be useful for others 🙂 (also a great way to get us involved with debuggin...

So the issue is that you have two reference branches on the local git, one to gitlab one to gitea and it fails to understand which on is the correct remote ...
I wonder if "git ls-remote --get-url" will always work ?!

DepressedChimpanzee34 <character> will almost always be converted into \ because otherwise it will not support \t or \n etc.
What I'm looking here is some logic that will allow us not to break backwards compatibility on the one hand, but still will allow you to have something like "first\second" entry.
WDYT? any ideas? (I really want to make sure we fix it as soon as possible)