Reputation
Badges 1
25 × Eureka!What do you have under the "installed packages" ?
Thanks CleanPigeon16
Could you verify Task "d1d361d1059c4f0981200f59d7683773" exists (and not archived)?
Hi UpsetCrocodile10
execute them and return scalars.
This should be a good start (I hope π )
` for child in children:
put the Task into an execution queue
Task.enqueue(child, queue_name='my_queue_here')
wait for the task to finish
child.wait_for_status(status=['completed'])
reload all the metrics
child.reload()
get the metrics
print(child.get_last_scalar_metrics()) `
Hi JitteryCoyote63 ,
When you shutdown the task (manually with close() or when the process finish) it wait for the uploads...
Why do you need to specifically wait for all the artifacts upload? (currently you can stop the artifacts upload thread and wait for all the artifacts, but that seems like a bad hack)
Oh I think that I understand what's going on, @<1523701260895653888:profile|QuaintJellyfish58> let me check how to update the cron scheduler while it is running (I really like this idea, so if this is not already supported I'l like us to add this capability π )
. However, despite having imported the required types from theΒ
typing
Β library in the script where the function decorated withΒ
PipelineDecorator.component
Β is defined, later in the generated script theΒ
typing
Β library is not imported outside the scope of the function
Actually the typing part is not passed to the "created step" , because there are no global imports, for eexample:
` def step(a: pd.DataFrame):
import pandas as pd
...
Hi @<1657918706052763648:profile|SillyRobin38>
I have included some print statements
you should see those under the Task of the inference instance.
You can also do:
import clearml
...
def preprocess(...):
clearml.Logger.current_logger().report_text(...)
clearml.Logger.current_logger().report_scalar(...)
, specifically within the containers where the inferencing occurs.
it might be that fastapi is capturing the prints...
[None](https://github.com/tiangolo/uvicor...
2023-02-15 12:49:22,813 - clearml - WARNING - Could not retrieve remote configuration named 'SSH'
This is fine, it means it uses the default identity keys
The thing is - when I try to connect with normal SSH there are no issues
Now I'm lost, so when exactly do you see the issue ?
clearml doesn't change the matplotlib backend under the hood, right? Just making sure
if the agent is running it, of course it does π otherwise where is the automagic, it would break the moment you run it on a remote machine
Hi AgitatedTurtle16 could you verify you can access the API server with curl?
RobustGoldfish9
I think you need to set the trains-agent docker to be aware of the host, so it knows how to mount data/cache/configurations into the sibling docker
It should look something like:TRAINS_AGENT_DOCKER_HOST_MOUNT="/mnt/host/data:/root/.trains"
So if running a docker:docker run -e TRAINS_AGENT_DOCKER_HOST_MOUNT="/mnt/host/data:/root/.trains" ...
What do you have in "server_info['url']" ?
Hi WickedGoat98 ,
I think you are correct π
I would guess it is something with the ingress configuration (i.e. ConfigMap)
Hi @<1523722267119325184:profile|PunySquid88> I guess it's a good thing we talk, because I believe that what you are looking for is already available :)
Logger.current_logger().report_media('title', 'series', iteration=1337, local_path='/tmp/bunny.mp4')
This will actually work on any file, that said, the UI might display the wrong icon (which will be fixed in the next version).
We usually think of artifacts as data you want to reuse, so all the files uploaded there are accessibl...
Correct the serving Task ID is the clearml serving session. It is the instance that holds all the information of this specific setup and models
Hi MagnificentSeaurchin79
Yes this is a bit confusing π
Datasets are stored as delta changes from parent versions.
A dataset contains a list of files and list of artifacts where these files exist. This means that if we add a new file to a dataset we create a new dataset from a parent dataset and want to add a file, we have to add a link to the file, and have a new artifact containing just the delta (i.e. the new file) from the parent version When you delete a file you just remove the li...
Oh in that case add --remote-gateway <external_ip>
It will connect to the provided address instead of the local one. (you can also add --public-ip
which will automatically resolve the public IP of the server
Are you trying to upload an artifact post execution ?
Hi SteadyFox10
Yes we changed the Web UI, to something more intuitive (but after you get used to the original design , I guess not that obvious).
After selecting a bunch of experiment, right click one of them, you will be able to archive them all (it will display the number of experiments you are about to archive)
I might gave an idea, could you test with:
` from clearml import Task
Task._report_subprocess_enabled = False
...
real code here `
Creating a dataset sounds like a good idea, but that does not seem to be the issue.
Can you verify you can manually clone using the same link (notice the log should specify the exact clone it is using, with the password replaced with *)
Hurray π
BTW: the next version will have a project level "readme alike" markdown embedded in the UI, so hopefully you will be able to add all the graphs there :)
Okay verified, it's the 'Agg' backend
TrickySheep9 is this a conda package or a wheel you are installing manually ?
the separate experiments are not starting back at iteration 0
What do you mean by that?
I like the idea of using the timeit interface, and I think we could actually hack it to do most of the heavy lifting for us π
Hi @<1523704198338711552:profile|RoughTiger69>
From this scenario can we assume the "selection" will be tagging the model manually?
Also, how would an human operator decide on the best model, that is what is the input to base the decision on?
BurlyRaccoon64 by default if .ssh exists in the host user folder it should mount it to the container (actually mount a copy of it). do you have a log of two tasks from two diff machines, one failing one passes? because this is quite odd (assuming the setup itself is identical)