Hi @<1523701868901961728:profile|ReassuredTiger98> , I think you can achieve a similar affect using add_external_files - None
Hi @<1648134232087728128:profile|AlertFrog99> , I don't think there is an automatic way to do this out of the box but I guess you could write some automation that does that via the API
Hi PunyWoodpecker71 ,
Regarding your questions:
We have an existing EKS cluster. So I'm wondering if I should deploy ClearML on the EKS cluster, or deploy on EC2 using AMI. Is there an advantage of one over the other?I think it's a matter of personal preference. Maybe SuccessfulKoala55 , can add some information.
We have a pipeline that need to run once a month to train a model. Is there an scheduler option we can config to enqueue the pipeline once a month? (It look like the Pro plan has ...
Hi GrievingDeer61 ,
The packages are automatically detected during the run and used by the worker. You can force the worker to use the requirements file with the following method:
https://clear.ml/docs/latest/docs/references/sdk/task#taskadd_requirements
Tell me if this helps 🙂
Hi! Hmmm, good question. I think it's asynchronous since most of the uploading processes are usually async. Is there a specific use case you're thinking of?
I recall a big fix to plots in server version 1.6.0, can you try upgrading to see if it fixes the issue?
I'm not sure. Maybe TimelyPenguin76 can elaborate.
Looking at the docs for:
https://clear.ml/docs/latest/docs/references/sdk/task#get_reported_scalars
I can see it has a max_samples argument
Please note it is better to disable a specific framework who's model you don't want to auto log
My guess other agents are sitting on different machines, did you verify that the credentials are the same between the different clearml.conf files? Maybe @<1523701087100473344:profile|SuccessfulKoala55> might have an idea
TimelyPenguin76 can you assist?
Hi @<1658643479691005952:profile|TroubledLobster8> , about the agents in the web UI, they usually clear out in 10-15 minutes from there
Hi OutrageousSheep60 ,
I guess ClearML has an internal function to check hash. You can follow the code and see what is being used and use something similar to validate.
How does you dataframe.shape look?
Hi HappyDove3 ,
I think you would need to go through the various sections of the API ( https://clear.ml/docs/latest/docs/references/api ) and see what needs to be moved. Basically you need to export all tasks and their events and everything else that might be on your workspace
I am using v.1.3.2
The SDK I assume.
Is this a self hosted version? What is the server version?
Hi @<1523717803952050176:profile|SmoothArcticwolf58> , can you describe a bit about the network between the agent and the new server?
Hi NarrowLion8 , do you have a snippet? Just changing the file_server attribute caused that?
VirtuousFish83 Hi 🙂
What versions are you running with? ClearML, ClearML-Agent, Torch, Lightning. Which OS are they run on and with what python version.
Do you maybe have a snippet to play around with to try and reproduce the issue?
You'll need ES/Mongo to run the ClearML server
Hi @<1524922424720625664:profile|TartLeopard58> , you mean the side bar on the left with projects/datasets/etc... ?
I'm afraid not, but I think it would be a cool feature request on GitHub 😉
This is the env variable you're looking for - CLEARML_AGENT_SKIP_PYTHON_ENV_INSTALL
Hi @<1750327614469312512:profile|CrabbyParrot75> , why use the StorageManager module and not the Datasets to manage your data?
Hi @<1730033904972206080:profile|FantasticSeaurchin8> , can you provide a sample script that reproduces this behaviour?
@<1706116294329241600:profile|MinuteMouse44> , did you set this in the clearml.conf of the agent as well as the machine of the SDK?
Try setting it outside of any section. Basically set an environment section by itself
In the webUI, when you go to the dataset, where do you see it is saved? You can click on 'full details' in any version of a dataset and see that in the artifacts section