Reputation
Badges 1
25 × Eureka!RoundMosquito25 do notice the agent is pulling the code from the remote repo, so you do need to push the local commits, but the uncommitted changes clearml will do for you. Make sense?
Makes sense
we need to figure what would be the easiest way to have an "opt-in" for the demo server, that will still make it a breeze to quickly test code integration ...
Any suggestions are welcomed π
GrievingTurkey78 I see,
Basically the arguments after the -m src.train
in the remote execution should be ignored (they are not needed).
Change the m in the Args section under the configuration. Let me know if it solved it.
But in credentials creation it still shows 8008. Are there any other places in docker-compose.yml where port from 8008 to 8011 should be replaced?
I think there is a way to "tell" it what to out there, not sure:
https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server_config#configuration-files
Hi RoundMosquito25
however they are not visible either in:
But can you see them in the UI?
Let me check... I think you might need to docker exec
Anyhow, I would start by upgrading the server itself.
Sounds good?
SpotlessFish46 unless all the code is under "uncommitted changes" section, what you have is a link to the git repo + commit id
BoredGoat1
Hmm, that means it should have worked with Trains as well.
Could you run the attached script, see if it works?
it is shown in the recording above
It was so odd, I had to ask π okay let me see if we can reproduce
I donβt have any error message in the browser console - Just an empty array returned on events.get_task_logs. This bug didnβt exist on version 1.1.0 and is quite annoyingβ¦
meaning the RestAPI returns nothing, is that correct ?
Any chance you can PR a fix to the docs?
Hi MoodyCentipede68 , I think I saw something like it, can you post the full log? The triton error is above, also I think it restarted the container automatically and then it worked
Meanwhile check CreateFromFunction(object).create_task_from_function(...)
It might be better suited than execute remotely for your specific workflow π
Tested with two sub folders, seems to work.
Could you please test with the latest RC:pip install clearml==0.17.5rc4
Thanks MinuteGiraffe30 , fix will be pushed later today
But once i see it on the UI means it is already launched somewhere so i didn't quite get you.
The idea is you run it locally once (think debugging your code, or testing it)
While running the code the Task is automatically created, then once in the system you can clone / launch it.
Also, I want to launch my experiments on a kubernetes cluster and i don't actually have any docs on how to do that, so an example can be helpful here.
We are working on documenting the full process, ...
WackyRabbit7 my apologies for the lack of background in my answer π
Let me start from the top, one of the goal of the trains-agent is to reproduce the "original" execution environment. Once that is done, it will launch the code and monitor it. In order to reproduce the original execution environment, trains-agent will install all the needed python packages, pull the code, and apply the uncommitted changes.
If your entire environment is python based, then virtual-environment mode is proba...
SubstantialElk6 feel free to tweet them on their very inaccurate comparison table π
Hi LazyFox65
So the idea is that you add two lines of code to your codebase :from clearml import Task task = Task.init(project_name='examples', task_name='change me')
And you run it once, then it will create the experiment, environment arguments etc.
Now that you have it in the UI you can clone / change all the fields and send for execution.
That said you can also create an experiment from CLI (basically pointing to a repo and entry point)
You can read here:
https://github.com/allegroa...
My apologies you are correct 1.8.1rc0 π
for example train.py & eval.py under the same repo
Yep I changed it
This means it will totally ignore the overrides and just take the OmegaConf, this is by design. You either use the overrides, or you configure the OmegaConf. LovelyHamster1 Does that make sense ?
Hi @<1684010629741940736:profile|NonsensicalSparrow35>
So sorry I missed this thread π
Basically your issue is the load balancer that prevents the post command, you can change that, just add to any clearml.conf the following line:
api.http.default_method: "put"
Hi ExuberantParrot61 the odd thing is this, message
No repository found, storing script code instead
when you are actually running from inside the repo... (
is it saying that on a specific step, or is it on the pipeline logic itself?
Also any chance you can share the full console output ?
BTW:
you can manually specify a repo branch for a step:
https://github.com/allegroai/clearml/blob/a492ee50fbf78d5ae07b603445f4983feb9da8df/clearml/automation/controller.py#L2841
Example:
https:/...
Yes, experiments are standalone as they do not have to have any connecting thread.
When would you say a new "run" vs a new "experiment" ? when you change a parameter ? change data ? change code ?
If you want to "bucket them" use projects π it is probably the easiest now that we have support for nested projects.
Hi ReassuredTiger98
Are you referring to the UI (as much as I understand there was an improvement, but generally speaking, it still needs the users to have the S3 credentials in the UI client, not backend)
Or are you asking on the cleanup service ?
. And I saw that it upload the notebook it self as notebook. Does it is normal? There is a way to disable it?
Hi FriendlyElk26
Yes this is normal, it backups your notebook as well as converts it into python code (see "Execution - uncommitted changes" so that later the clearml-agent will be able to run it for you on remote machines.
You can also use task.connect({"param": "value")
to expose arguments to use in the notebook so that later you will be able to change them from the U...