The -m src.train is just the entry script for the execution all the rest is be taken care by the Configuration section (whatever you pass after it will be ignored if you are using Argparse as it is auto-connects with ClearML)
Make sense ?

Are you sure trains-server not trains package (i.e. backend)

What's the trains-server version ?

using caching where specified but the pipeline page doesn't show anything at all.

What do you mean by " the pipeline page doesn't show anything at all."? are you running the pipeline ? how ?
Notice PipelineDecorator.component needs to be Top level not nested inside the pipeline logic, like in the original example

@PipelineDecorator.component(
        cache=True,
        name=f'append_string_{x}',
    )

Also, I would upgrade the backend 0.15.1 a few bugs were fixed since 0.14.x some have to do with the plots...

Hi @<1534706830800850944:profile|ZealousCoyote89>

We'd like to have pipeline A trigger pipeline B

Basically a Pipeline is a Task (of a specific Type), so you can have pipeline A function clone/enqueue the pipelineB Task, and wait until it is done. wdyt?

BroadMole98

I'm still exploring what trains is for.

I guess you can think of Trains as Experiment manager + MLOps tied together.

The idea is to give a quick and easy way to move from coding/running on one machine to scaling it to multiple remote machines, with everything that comes with it.

In some ways it is like snakemake, it setups your environment and execute the code. Snakemake also allows you to setup data, which in Trains is done via code (StorageManager), pipelines are also...

Update us if it solved the issue (for increased visibility)

Hmm would uploading it as YAML string be better?

Hi ProudChicken98
How about saving it as a local YAML and upload the file itself as an artifact?

Hmm, so the way the configuration works is it loads the default configuration (equivalent to the example in the docs) then it adds the ~/clearml.conf on top. That means that you can tell your users to just copy paste the credentials from the UI into a template you make. How is that ?

It is http btw, i don't know why it logged https://

This is odd could it be it automatically forwards to https ?
I would try the certificate check thing first

This port is odd, it should be 8008
8015

Hi OutrageousSheep60

Is there a way to instantiate a

clearml-task

while providing it a

Dockerfile

that it needs to build prior to executing the task?

Currently not really, as at the aned the agent does need to pull a container,
But you can cheive basically the same by adding the "dockerfile" script as --docker_bash_setup_script Notice of course that this is an actual bash script not Docker script, so no need for "RUN" prefix.
wdyt?

The 'on-premise' server fails to connect to the ClearML server because of the VPN I think

I think you are correct.
You can quickly test it, try ti run curl http://local-server:8008 see if that works

According to you the VPN shouldn't be a problem right?

Correct as long as all parties are on the same VPN it should work, all the connections are always http so basically trivial communication

And is it talking to the correct server ?

SmarmyDolphin68 what's the error ?

curl -vvv

Could it be the credentials are actually incorrect? because it seems like you can access the server? (I assume you were able to browse to it and generate credentials. right?)

Just making sure I understand, basically same ArgParser support we already have, but for python-fire (which is the ability to automatically log the arguments, and then change them when executed by trains-agent), correct?
If this is the case, are you familiar with the implementation of python-fire ? What I'm looking for is where exactly the parsing happens, so we could patch it, and log/override values

I could take a look and figure that out.

This will greatly accelerate integration 😉

GrievingTurkey78 can you send the entire log?

Hmm, Is it http or https ?

seems okay to me

Hi GracefulDog98
As UnevenDolphin73 pointed you might be looking for https://clear.ml/docs/latest/docs/references/sdk/task#execute_remotely
Which will stop the current local process, and enqueue the task on the "default" queue, for the agent to execute.
Is this what you are looking for ?
The idea is you can run your code once in "development" mode, so you know everything is working, then from the UI (or programmatically) you can clone the experiment, edit the configuration (or anythin...

GrievingTurkey78 whats the repository link you see in the UI? Does it start with ssh:// or https://
Did you add git_user/git_pass to the trains.conf of the trains-agent ? (if you did it should replace any ssh:// link with https:// user/pass link)

I look forward to your response on Github.

Great, I would like to make this discussion a bit more open and accessible so GitHub is probably better

I'd like to start contributing to the project...

That will be awesome!

Hi RoughHedgehog31
I'm assuming your git diff is just too big to be stored as is (probably some binary files)
it should not really have any effect on the execution, it just means the clearml-agent will not be able to reproduce the uncommitted changes.
Make sense ?

Which would mean the error is because of a company firewall/self-signed certificate.
The easiest solution,Disable SSL certificate check for ClearML.
Create the ~/clearml.conf manually:
` #disable SSL certificate check
api.verify_certificate: False

copy paste the credentials section from the UI

it should look something like:

api {
# web_server on port 8080
web_server: " "

# Notice: 'api_server' is the api server (default port 8008), not the web server.
api_server: ...