Hi @<1710827348800049152:profile|ScantChicken68> , I'd suggest first reviewing the onboarding videos on youtube:
None
None

After that, I'd suggest just adding the Task.init() to your existing code to see what you're getting reported. After you're familiar with the basics then I'd suggest going into the orchestration/pipelines features 🙂

Hi @<1523702786867335168:profile|AdventurousButterfly15> , are the models logged in the artifacts section?

Hi @<1673501387578675200:profile|AdventurousLizard97> , can you please provide the full log of such a run?

You certainly can do it with the python APIClient OR through the requests library

Hi WickedCat12 ,

During Task.init() you can specify auto_connect_frameworks=False for the framework you're working with. However please note that this will stop auto reporting scalars etc

https://clear.ml/docs/latest/docs/references/sdk/task#taskinit

Hi GiganticMole91 , what version of ClearML server are you using?
Also, can you take a look inside the elastic container to see if there are any errors there?

Hi RoundMosquito25 , you can reset a failed task and this will bring it back to draft mode

o, if I pull this file from s3 bucket, I can conclude which chunk I should download to get a specific file. Am I wrong?

I think you're right. Although I'm not sure if you can decompress individual chunks - worth giving it a try!

I also though clearML writes this mapping (

state.json

) into one of its databases: Mongo, Redis, Elasticsearch.

I think the state.json is saved like an artifact so the contents aren't really exposed into one of the dbs

Hi @<1536881167746207744:profile|EnormousGoose35> , you can integrate ClearML into your existing code with the two simple lines of

from clearml import Task
task = Task.init(...)

To see how it works and looks 🙂

Hi @<1523713932588486656:profile|PerplexedWalrus3> , I'm not sure about the exact configuration of your setup but I'm quite sure you could do this fairly easily with pipelines and datasets in ClearML. Have you tried playing with Datasets to get the feeling of how it works?

Hi @<1523701062857396224:profile|AttractiveShrimp45> , can you please add the configuration of your HPO app and the log?

Do you see any errors in the dev tools console (F12)?
Also are there any errors in elastic?

https://clear.ml/docs/latest/docs/references/sdk/automation_controller_pipelinecontroller#get_processed_nodes

FrustratingShrimp3 , it could be quite simple to do yourself as well. You can take a look at the example bindings here:
https://github.com/allegroai/clearml/tree/master/clearml/binding
It shouldn't be difficult to implement a new framework

Hi VivaciousReindeer64 , I think you can simply edit the files_server to point to the correct port 🙂

Hi @<1547752799075307520:profile|ZippyCamel28> , to address your points

• What do you mean by 'reload'?
• You need to go into the project and archive the experiments in order to delete the project + experiments in the archive
• There are some configurations you can play with to report 'less' metrics. For example sdk.metrics.plot_max_num_digits You should read here - None . To get an idea of the size of an experiment think of an...

mongo/elastic/redis

Hi @<1523701842515595264:profile|PleasantOwl46> , I'm afraid that such a capability doesn't really exist in ClearML. You could technically populate an experiment using the API.

I'm however curious - what is your use case for this?

Hi @<1523711002288328704:profile|YummyLion54> , can you please add a full log of both runs for reference?

@<1556812486840160256:profile|SuccessfulRaven86> , what is the base docker image you mention? Did you check that this docker has python 3.9?

Hi @<1661904968040321024:profile|SpotlessOwl43> , you can achieve this using the REST API of ClearML - None

I think as long as they have different hashes you will have two different files

Hi @<1523702932069945344:profile|CheerfulGorilla72> , I think you need to map out the relevant folders for the docker. You can add docker arguments to the task using Task.set_base_docker

Hmm maybe @<1523701087100473344:profile|SuccessfulKoala55> might have an idea

Hi @<1539780284646428672:profile|PoisedElephant79> , I think you need to have the gitlab runner able to connect to your VPN. Otherwise, how do you expect it to connect to the server if only people on your VPN can connect to it?

The assumption is that the server and serving don't run on the same machine. The ClearML server is just a control plane whereas the serving solution actually does computation.

Hi @<1566959349153140736:profile|ShinyChicken29> , when you try to access the image on your browser, the browser tries access the S3 bucket directly - This is why you get the popup. Data never goes through ClearML backend. Makes sense?

Hi @<1547028031053238272:profile|MassiveGoldfish6> , I think you can access the description by fetching the dataset and searching inside the different attributes. You can use dir() in python to see what's attached. I think you can look for Dataset.data after you fetch it

And you're sure that clearml.conf points to the correct server with the right credentials?

Hi ReassuredOwl55 , can you please elaborate on your use case or exactly what you're trying to achieve?