hmmm I see...
It seems to miss the fact that your process do uses the GPU.
Maybe it only happens later, that the GPU is used?
Does that make sense ?

Merged 🙂

Hi LovelyHamster1 ,
you mean totally ignore the "installed packages" section, and only use the requirements.txt ?

@<1541954607595393024:profile|BattyCrocodile47> first let me say I ❤ the dark theme you have going on there, we should definitly add that 🙂

When I run

python set_triggers.py; python basic_task.py

, they seem to execute, b

Seems like you forgot to start the trigger, i.e.
None
(this will cause the entire script of the trigger inc...

Is is across the board for any Task ?
What would you expect to happen if you clone a Task that used the requirements.txt, would you ignore the full "pip freeze" and use the requirements .txt again, or is this thime we want to use the "installed packages" ?

So I can set output_uri = "s3://<bucket_name>/prefix" and the local models will be loaded into the s3 bucket by ClearML ?

Yes, magic 🙂

Back to the feature request, if this is taken care of (both adding a missed package, and the S3 upload), do you still believe there is a room for this kind of feature?

I think there is a bug on the UI that causes series with "." to only use the first part of the series name for the color selection. This means "epsilon 0" and "epsilon 0.1" will always get the same color, and this will explain why it works on other graphs

it is just local copy so you can rerun and reconfigure

I think it was just pushed, including nested call you have to use the new argument for the decorator, helper_function
https://github.com/allegroai/clearml/blob/400c6ec103d9f2193694c54d7491bb1a74bbe8e8/clearml/automation/controller.py#L2392

My pleasure, and apologies 🙂

pip cache & git cache & venvs cache
Are all supported, you just need to map the folders.
If you do not want to spin a PVC with NFS mount, you can just mount an S3 bucket with s3fs as part of the container extra bash script,
https://github.com/allegroai/clearml-agent/blob/b39b54bbafab39e6731cb742fdf317bc6dcae54a/docs/clearml.conf#L140

s3 FUSE fuse filesystems:
https://github.com/kahing/goofys
https://github.com/s3fs-fuse/s3fs-fuse

WDYT?

Hi @<1610083503607648256:profile|DiminutiveToad80>

<h1>Request Entity Too Large</h1>

What's the size of the file? how are you running your clearml-server?

GreasyLeopard35 from the implementation:
https://github.com/allegroai/clearml/blob/fcad50b6266f445424a1f1fb361f5a4bc5c7f6a3/clearml/automation/parameters.py#L215
Which basically returns the "self.base" (default) 10 to the power of the selected value:
10**-3 = 0.001
So how would I get a negative value ?

GreasyLeopard35 I think you are on to something, I think UniformParameterRange just misses a min value:
https://github.com/allegroai/clearml/blob/fcad50b6266f445424a1f1fb361f5a4bc5c7f6a3/clearml/automation/parameters.py#L168
Should be:
[self.min_value + v*step_size for v in range(0, int(steps))]

` from clearml.automation.parameters import LogUniformParameterRange
sampler = LogUniformParameterRange(name='test', min_value=-3.0, max_value=1.0, step_size=0.5)
sampler.to_list()

Out[2]:
[{'test': 1.0},
{'test': 3.1622776601683795},
{'test': 10.0},
{'test': 31.622776601683793},
{'test': 100.0},
{'test': 316.22776601683796},
{'test': 1000.0},
{'test': 3162.2776601683795}] `

But the missing implementation of LogUniformRange for hpbandster still causes problems.

wdym?

GreasyLeopard35
I can update that the fix to UniformIntegerParameterRange should be pushed with tomorrows release 🙂
(which would fix in turn LogUniformParameterRange)

Releasing an RC

Hi ReassuredOwl55
The easiest is to configure it as default output_uri in the clearml.conf of file the agent, wdyt?
https://github.com/allegroai/clearml-agent/blob/ebb955187dea384f574a52d059c02e16a49aeead/docs/clearml.conf#L430

there is a semaphore warning, not sure if it’s related

Can you resend it?
Is the Task marked as closed when the process ends ?

Wouldn't it make sense to use a single one ?

Why are you initializing 3 diff Tasks ?

Let me check that for a sec

UnevenDolphin73 something like this one?
https://github.com/allegroai/clearml/pull/225

we need to evaluate the result across many random seeds, so each task needs to log the result independently.

Ohh that kind of makes sense to me 🙂
Yes I'm also getting:

/usr/local/lib/python3.6/multiprocessing/semaphore_tracker.py:143: UserWarning: semaphore_tracker: There appear to be 74 leaked semaphores to clean up at shutdown
  len(cache))

Not sure about that ...

Let me check, see what can be learned ...

GrievingTurkey78 I have to admit I can't see the difference, can you help me out 🙂

Not sure on the cause but if you do:

mp.set_start_method('fork', force=True)

There is no semaphore leakage

Since I can't use the

torchrun

comand (from my tests, clearml won't use it on the clearm-agent), I went with the

@<1556450111259676672:profile|PlainSeaurchin97> did you check this example?
None