Hi CleanPigeon16
Put the specific git into the "installed packages" section
It should look like:
... git+ ...(No need for the specific commit, you can just take the latest)

Hi CleanPigeon16

can I make the steps in the pipeline use the latest commit in the branch?

Yes:
manually clone the stesp's Task (in the UI), and in the UI edit the Execution section and change to "last sommit on branch" and specify the branch name programmatically (as the above, clone+edit)

ValueError: Could not parse reference '${run_experiment.models.output.-1.url}', step run_experiment could not be found

Seems like the "run_experiment" step is not defined. Could that be ...

With remote_execution it is 

command="[...]"

 , but on local it is 

command='train'

 like it is supposed to be.

I'm not sure I follow, could you expand ?

hurray

Hi SubstantialElk6

Generically, we would 'export' the preprocessing steps, setup an inference server, and then pipe data through the above to get results. How should we achieve this with ClearML?

We are working on integrating the OpenVino serving and Nvidia Triton serving engiones, into ClearML (they will be both available soon)

Automated retraining

In cases of data drift, retraining of models would be necessary. Generically, we pass newly labelled data to fine...

if we look at the host machine we can see a single python process that is actually busy

Only one?! can you see the other python processes ?

GiddyTurkey39

as others will also be running the same scripts from their own local development machine

Which would mean trains ` will update the installed packages, no?

his is why I was inquiring about the 

requirements.txt

 file,

My apologies, of course this is supported 🙂
If you have no "installed packages" (i.e. the field is empty in the UI) the trains-agent will revert to installing the requirements.txt from the git repo itself, then it...

PompousBeetle71 , the reason I'm asking is the warning you see is due to the fact it cannot detect the filename you are saving your model to ... I'm trying to figure out how that actually happened .
BTW: in the next version we will probably remove this warning altogether, but I'm still curious on how to reproduce 🙂

Hi SubstantialElk6

We can't seem to find a way for the end user to pass in their git credentials when they run their codes in both agent and non-agent scenarios. Any advice here?

The bottom line is the agent needs to have read-only access to all the repositories so it can launch any Task. I would recommend to create an agent git user with read-only credentials and configure the agent to use it. wdyt?

Hi RoughTiger69
A. Yes makes total sense . Basically you can use Task.export Task.import to do achieve this process (notice we assume the dataset artifacts links are available on both, usually this is the case)

B. The easiest way would be to use Process , then one subprocess is exporting from dev , where the credentials and configuration is passed with os environment. The another subprocess imports it to the prod server (again with os environment pointing to the prod server). Make sense?

Is trains-agent using docker-mode or virtual-env ?

ShinyLobster84

fatal: could not read Username for '

': terminal prompts disabled

This is the main issue, it needs git credentials to clone the repo code, containing the pipeline logic (this is the exact same behaviour as pipeline v1 execute_remotely(), which is now the default, could it be that before you executed the pipeline logic, locally ?)
WackyRabbit7 could the local/remote pipeline logic could apply in your case as well ?

` from time import sleep
from clearml import Task
import tqdm

task = Task.init(project_name='debug', task_name='test tqdm cr cl')
print('start')
for i in tqdm.tqdm(range(100)):
sleep(1)
print('done') `The above example code will output a line every 10 seconds (with the default console_cr_flush_period=10) , can you verify it works for you?

Thank you DilapidatedDucks58 for the ping!
totally slipped my mind 😞

IrritableOwl63 in the profile page, look at the bottom right corner

great 🙂
two things:
I'm not sure argparse supports dict as a type (I mean it will take anything but I'm not sure it will parse your arguments as dict) I know there was an issue with argparsing, but I think it was solvedbtw: Basically the way clearml-agent works, it does not actually pass the arguments in commandline but directly to the argparser at runtime
What happens if you clone the Task (the one with Args showing and without the explicit task.connect(_args) and send it to the age...

Hi @<1529633468214939648:profile|CostlyElephant1>

Is it possible to get user ID of the current user

On the Task.data object itself there should be a filed named " user " that's the user ID of the owner (creator) of the Task.
You can filter based on this id with

Tasks.get_tasks(..., task_filter={'user': ["user-id-here"]})

wdyt?

HandsomeCrow5 Seems like the right place would be in the artifacts, as a summary of the experiment (as opposed to on going reporting), is that the case?
If it is then in the Artifacts tab clicking on the artifact should open another tab with your summary, which sounds like what you were looking for (with the exception of the preview thumbnail 🙂

Hi SpotlessWorm70

OMP: Error #15: Initializing libiomp5.dylib, but found libomp.dylib already initialized.
OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program.

This seems like OpenMP issue
I would assume something is off with the local environment (not really connected to clearml but to one of the frameworks, for example TF, Keras, etc.)

Should be under Profile -> Workspace (Configuration Vault)

GiganticTurtle0 what's the Dataset Task status?

🤞

GrievingTurkey78

maybe since the package is not directly imported in my code it is possible to get a different version to what I have locally (?).

If these are derivative packages (i.e. imported by other packages) they are not automatically logged when executing the Task manually (in order to keep the "installed packages as lean as possible on the one hand but specify also specify the important packages for you)
That said, when the "trains-agent" executed the task it will store nack...

Hi ReassuredOwl55

a dialogue box that opens with a “deleting” bar swishing across, but then it just hangs, and becomes completely unresponsive

I believe this issue was fixed in the latest server version, seems like you are running 1.7 but the latest is 1.9.2. May I suggest an upgrade ?

So the only difference is how I log in into machine to start clear-ml

the only different that I can think of is the OS Environments in the two login types:
can you run export in the two cases and check the diff between them?
export

SolidSealion72 EcstaticGoat95 I'm hoping the issue is now resolved 🤞
can you verify with ?
pip install git+

Thanks SmallDeer34 !
This is exactly what I needed

I think the main issue is that for some reason the container running changed one of the files inside the temp folder. then the host machine is "stuck" with a file that the root user owned/changed, and now it cannot reuse / delete the temp folder.
I think the fix is to make sure the container deleted the temp folder when it is done

Hmm I would have the docker file contain the default Azure credentials/output_uri, and then have the users clearml credentials passed as env variable in runtime. wdyt?
(I'm checking if you can pass the azure credentials as env in a minute)

my pleasure