Do you report with LightningModule log function? something like:
self.log('train_loss', loss)?

Hi EcstaticBaldeagle77 ,

The comment says “Connecting ClearML with the current process, from here on everything is logged automatically.”

this comment means that every framework is now patched and will report to ClearML too, this can be configure (per task) with auto_connect_frameworks in your Task.init call (example can be found here - https://clear.ml/docs/latest/docs/faq#experiments )

Q2: Can I dump this logged keys & values as local files ? (edited)

Not sure ...

I get your point, can you please open a https://github.com/allegroai/trains/issues with this request? 🙂

max_spin_up_time_min - the maximum time for an instance to spin up
max_idle_time_min - the maximum time for an instance to stay up with worker on it (the time the ec2 instance is finished running a task with the agent and still up with an worker running on it listening to the queue)

So once I enqueue it is up?

If the trains-agent is listening to this queue (services mode), yes.

Docs says I can configure the queues that the auto scaler listens to in order to spin up instances, inside the auto scale task - I wanted to make sure that this config has nothing to do to where the auto scale task was enqueued to

You are right, the auto scale task has nothing to do to where the auto scale task is enqueued

btw. why do I need to give my git name/pass to run it if I serve an agent from local?

The main idea is that you can run the agent in any machine (local, cloud) and all should be done out of the box.

If your code is running as part of a git repository, the clearml agent will have to clone it, and for doing so it will use credentials.
Git name and pass are one way for it, but you can also use ssh - if you dont have the git name and password in the configuration, the clearml-agent will t...

EagerStork23 However, the issue is only in the presentation of the graph (occurs when all subplots have the same label), so you can use the following workaround for solving it:

` import matplotlib.pyplot as plt

def plot_subplots():
fig = plt.figure()
ax1 = fig.add_subplot(2, 2, 1)
ax2 = fig.add_subplot(2, 2, 2)
ax3 = fig.add_subplot(2, 2, 3)
ax4 = fig.add_subplot(2, 2, 4)
x = range(10)
y = range(10)
ax1.plot(x, y)[0].set_label("label1")
ax2.plot(x, y)[0]...

Hi VexedCat68

You can use argparse and all the parameters will be log automagically to the hyperparameters section like in https://github.com/allegroai/clearml/blob/master/examples/frameworks/keras/keras_tensorboard.py#L56 example, or just connecting any dict like in https://github.com/allegroai/clearml/blob/master/examples/frameworks/ignite/cifar_ignite.py#L23 example

can you try it?

The AWS autoscaler is running as a https://github.com/allegroai/trains-server#trains-agent-services-- .
For services, we use a trains-agent configure to be in services mode. Now, each “service task” you insert to the queue should run until you stop it.

So, you need the AWS autoscaler keep running, which the services agent provide you. The “service task” is still a task, so it should have a project and a task name (just one for ALL your projects).

You can always run things locally wit...

 thanks for the answer, so for example (to make sure I understand) with the example you gave above when I’ll print the config I’ll see the new edited parameters?

Correct

What about the second part of the question, would it be parsed according to the type hinting?

It should

Hi FlatStarfish45 ,

The HPO task will control the HPO process, means it will clone the base task (the one we are optimizing), change the parameters, enqueue it and collect the results.
The base task is the task we want to optimize.
Each one of those two tasks, have different requirements.
You can look at the https://clear.ml/docs/latest/docs/guides/optimization/hyper-parameter-optimization/examples_hyperparam_opt#set-up-the-arguments for how set the base task in the HPO task.
can this be ...

The training task (child)

this is the task the HPO is cloning?
Do you have the packages in this task?

the packages should be list in this section. if they arent, it wont try to install them

Hi FlatStarfish45 ,

In the UI you can view the packages that the task will use (under EXECUTION tab), what do you get? Does it contains your packages in the INSTALLED PACKAGES section?

We can certainly add a trains.conf brief, thanks for the feedback  🙂

Hi HighCentipede85 ,

You can pass the git repo detection with auto_connect_frameworks={"detect_repository": False} :
task = Task.init(project_name="Your project name", task_name="Your task name", auto_connect_frameworks={"detect_repository": False})Notice: This will leave empty all the sections for the execution part.

Hi UnevenDolphin73 , the fix is ready, can you try it with the latest rc?

pip install clearml==1.4.2rc0

Hi MysteriousBee56 ,

Can you share a the code you are using with the APIClient?

The api version is selected automatically, let me check that for you

👍

Did you try task.data.hyperparams or task.hyperparams ?

Can you check the api version?
from trains.backend_api import Session print(Session.api_version)

can you try
print(task.data.hyperparams)instead of the last line?

You can get your task logger any time with get_logger() function, like in this example:
https://github.com/allegroai/clearml/blob/master/examples/reporting/text_reporting.py#L73

Hi VexedCat68 ,

How do you create it? with Dataset.create ?

HelpfulHare30 try using CLEARML_AGENT_SKIP_PIP_VENV_INSTALL env var with the path you your python env:
CLEARML_AGENT_SKIP_PIP_VENV_INSTALL=/path/to/your/python

WackyRabbit7 can you try upload the artifact with wait_on_upload=True ?

When you run it locally with auto_connect_frameworks={"matplotlib": False} , did it send the matplotlib outputs?

Hi PompousParrot44 , the distinction based on matplotlib, imshow display the data as an image, and images are going to debug section.

what do you think?