o, if I pull this file from s3 bucket, I can conclude which chunk I should download to get a specific file. Am I wrong?
I think you're right. Although I'm not sure if you can decompress individual chunks - worth giving it a try!
I also though clearML writes this mapping (
state.json
) into one of its databases: Mongo, Redis, Elasticsearch.
I think the state.json is saved like an artifact so the contents aren't really exposed into one of the dbs
I'm thinking maybe you can do it by chaining tasks somehow, but I think that is not the correct way (Never tried it myself, as I said, in the end it all abstracts to a single pipeline).
Maybe check out pipelines with functions or even better decorators, this might be an easier solution for you and you can create very complex pipelines with it. Think about using loops and if statements to create steps and progress your pipeline
Hi DeliciousKoala34 , is there also an exceptionally large amount of files in that Dataset? How do you create the dataset? What happens if you use something like s3 if you have available?
Hi @<1603560525352931328:profile|BeefyOwl35> , The agent uses it's own entry point, so yes you do need to specify it even if it's in the dockerfile 🙂
Can you post it?
Hi @<1590514584836378624:profile|AmiableSeaturtle81> , can you please add the log? What version of clearml are you using?
Hi @<1691258549901987840:profile|PoisedDove36> , did you do all the db migrations during the upgrade or did you go straight to 1.5 form 1.0?
I think you also might find this video useful:
None
Hi @<1547028079333871616:profile|IdealElephant83> , this is what the community channel is for, support, news & discussions related to ClearML OS 🙂
Regarding the github issue - Can you send the docker-compose you used for 1.9 that works and 1.10 that doesn't work for you?
Backup what you currently have
As you mentioned it seems that the authentication you're using is not created as part of the boto object. Is there a specific reason you don't want to use access/secret pairs for authentication?
In the HPO application I see the following explanation:
'Maximum iterations per experiment after which it will be stopped. Iterations are based on the experiments' own reporting (for example, if experiments report every epoch, then iterations=epochs)'
Hi @<1535069219354316800:profile|PerplexedRaccoon19> , not sure what you mean. Can you please share the full log, a screenshot of the two experiments and some snippet that re-creates this for you?
JitteryCoyote63 , heya, yes it is :)
You can save the entire folder as an artifact.
Can you add a code snippet that reproduces this for you please?
DilapidatedDucks58 , I think this is what you're looking for
https://github.com/allegroai/clearml/blob/master/docs/clearml.conf#L69
Hi @<1637624975324090368:profile|ElatedBat21> , do you have a code snippet that reproduces this? You can also manually log a model to the system using the OutputModel - None
Can you please add the full log of the execution?
StickySheep96 , Is it possible you raised the server locally on your machine and not the EC2 instance?
Then you can define the git credentials that can clone these repositories
https://clear.ml/docs/latest/docs/references/sdk/task#taskenqueue
Is this what you're looking for?
Also you can enqueue it through the API
https://clear.ml/docs/latest/docs/references/api/tasks#post-tasksenqueue
Is there anything special about the parent dataset?
Hi @<1695969549783928832:profile|ObedientTurkey46> , this is supported in the Scale/Enterprise licensees of ClearML (external IdP support). API access is always done using credentials.
@<1702492411105644544:profile|YummyGrasshopper29> , I suggest you take a look here - None
You can see the version if you go to settings page. It is in the bottom right of the screen 🙂
Hi @<1739093605621960704:profile|LovelySparrow29> , do you see any errors in mongo or elastic containers?