Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Profile picture
GorgeousShrimp11
Moderator
10 Questions, 42 Answers
  Active since 06 January 2024
  Last activity one year ago

Reputation

0

Badges 1

37 × Eureka!
0 Votes
5 Answers
1K Views
0 Votes 5 Answers 1K Views
Hi there, I am having issues executing a pipeline on a schedule . If I run the schedule with a simple function using schedule_function , it executes on the q...
one year ago
0 Votes
5 Answers
2K Views
0 Votes 5 Answers 2K Views
Hi, I have a worker running in services mode with --services-mode and using a Docker image. I did this as I need to run a pipeline with this agent running in...
one year ago
0 Votes
12 Answers
1K Views
0 Votes 12 Answers 1K Views
Hi there, we have a ClearML server and i have spun up an agent using a docker image (as we have private packages on AWS CodeArtifact that need installing). I...
one year ago
0 Votes
2 Answers
1K Views
0 Votes 2 Answers 1K Views
How does one add_tags to a pipeline? There is this function here: None But, I cannot figure out how to use it. I would like to automatically add_tags in the ...
one year ago
0 Votes
3 Answers
1K Views
0 Votes 3 Answers 1K Views
one year ago
0 Votes
14 Answers
1K Views
0 Votes 14 Answers 1K Views
one year ago
0 Votes
5 Answers
1K Views
0 Votes 5 Answers 1K Views
Hi there, I am running a clearml-agent and now trying to run a pipeline on a schedule. When I try run the schedule using the queue assigned to the worker I'v...
one year ago
0 Votes
13 Answers
1K Views
0 Votes 13 Answers 1K Views
How do you start an agent on the ClearML server with a specific clearml.conf file? ie. I am running something like this: clearml-agent daemon --queue forecas...
one year ago
0 Votes
4 Answers
1K Views
0 Votes 4 Answers 1K Views
Hi there, I want to get the URLs that are generated for the results and pipeline pages, as are logged here, as I want to use these elsewhere to reference the...
one year ago
0 Votes
6 Answers
978 Views
0 Votes 6 Answers 978 Views
one year ago
0 Hi There, I Am Having Issues Executing A

I have an agent running and assigned to the queue megan-testing :
image

one year ago
0 Hi, I Have A Worker Running In Services Mode With

Thanks, I'll have a look at the youtube videos. I've been going over the documentation a lot and haven't found much about actually running and deploying pipelines, and a lot of details seem to be missing.

one year ago
0 Hi There, I Want To Get The Urls That Are Generated For The Results And Pipeline Pages, As Are Logged Here, As I Want To Use These Elsewhere To Reference The Specific Pages. How Can I Get These Generated Urls From A Currently Running Pipeline?

For reference for anyone else, this is what I did:

current_task = Task.current_task()
model_run_url = Task.get_task_output_log_web_page(
        task_id=current_task.id,
        project_id="abc")
one year ago
one year ago
0 Hi There, Does Anyone Have Suggestions For Best Practice For Deploying A Pipeline So That It Can Run Remotely On A Clearml Server Using A Docker Image? I Am Finding The Clearml Docs And Videos Insufficient To Get The Pipeline To Actually Run To Completion

Ok, thanks! Going to try this now. I included an entry point from reading some other messages on Slack here when trying to figure out how to use Docker for running remotely.

one year ago
0 Hi There, I Am Having Issues Executing A

Error when executing schedule with a pipeline task ID defined:

Launching job: ScheduleJob(name='Simple Pipeline Schedule Run Test', base_task_id='4cf5b603e3ae4593a8c2dd6c34190c6d', base_function=None, queue='megan_testing', target_project=None, single_instance=False, task_parameters={}, task_overrides={}, clone_task=True, _executed_instances=None, execution_limit_hours=None, recurring=True, starting_time=datetime.datetime(1970, 1, 1, 0, 0), minute=2, hour=None, day=None, weekdays=['mon...
one year ago
0 Hi There, I Am Having Issues Executing A

Schedule testing code:

# Schedule for running the pipeline daily

from clearml import Task
from clearml.automation import TaskScheduler


def simple_function():
    print('This code is executed in a background thread, '
          'on the same machine as the TaskScheduler process')
    # add some logic here
    print('done')

if __name__ == "__main__":

    scheduler = TaskScheduler(force_create_task_name='megan-test-remote-pipeline')

    scheduler.add_task(
        name='Simple Pipeli...
one year ago
0 Hi There, We Have A Clearml Server And I Have Spun Up An Agent Using A Docker Image (As We Have Private Packages On Aws Codeartifact That Need Installing). I Have Created The

Hi @<1523701070390366208:profile|CostlyOstrich36> , I've gotten past the environment set up phase - I have the agent running in --services-mode for the pipeline, but the pipeline step is failing and I can't see why from the error message. When I run this locally it works (ie. if I add PipelineDecorator.run_locally() ).

Environment setup completed successfully

Starting Task Execution:


1710160150496 direct-relief:cpu:0:service:5062dce7ff1d49cfbaf36c96abe3282c DEBUG ClearML res...
one year ago
0 Hi, We Are Using Clearml For Our Experiment Tracking But Now Investigating Using The Pipeline Functionality As Well For Scheduling. We Also Want To Be Able To Trigger A Pipeline Run When There Is New Data In An External Database. Is This Possible? From Wh

Hi @<1523701070390366208:profile|CostlyOstrich36> , another quick question, can you run a pipeline on a schedule or are schedules only for Tasks? We are battling to figure out how to automate the pipelines.

one year ago
0 Hi, I Have A Worker Running In Services Mode With

Ah ok, so if I have two steps, this means 3 queues with 3 agents, one for the controller and one each dedicated to the steps, just to confirm?

one year ago
0 Hi There, I Want To Get The Urls That Are Generated For The Results And Pipeline Pages, As Are Logged Here, As I Want To Use These Elsewhere To Reference The Specific Pages. How Can I Get These Generated Urls From A Currently Running Pipeline?

Thanks, @<1523701070390366208:profile|CostlyOstrich36> .

If I use Task.current_task() , I then get a task object back that looks like <clearml.task.Task object at 0x7f7723b90a00> . How does one get the url from this?

one year ago
0 Hi There, We Have A Clearml Server And I Have Spun Up An Agent Using A Docker Image (As We Have Private Packages On Aws Codeartifact That Need Installing). I Have Created The

I've figured out that this is because I have a config.yaml file with secrets in it in the repository. This is not committed to git. So, when running remotely, the file is no present. Is the recommendation to put this in the docker image and then I have to specify an entry point in the dockerfile? Previously, I was hoping to just get away with creating a docker image with the installed packages for the agent, not with the repository code as well. Is this not the recommended approach?

one year ago
0 Hi There, We Have A Clearml Server And I Have Spun Up An Agent Using A Docker Image (As We Have Private Packages On Aws Codeartifact That Need Installing). I Have Created The

Yes, I started the agent in docker mode with clearml-agent daemon --queue direct-relief-forecasting --docker directrelief_ml_clearml --cpu-only -d

one year ago
0 Hi There, We Have A Clearml Server And I Have Spun Up An Agent Using A Docker Image (As We Have Private Packages On Aws Codeartifact That Need Installing). I Have Created The

I found this issue but not sure if it's the same thing as it's from awhile back: None and also not sure what the final solution here was.

one year ago
0 Hi There, I Am Running A Clearml-Agent And Now Trying To Run A Pipeline On A Schedule. When I Try Run The Schedule Using The Queue Assigned To The Worker I'Ve Got Running, I Get The Following Error About

I've tried changing the clearml.conf file to specify the python_binary with:

 python_binary: "usr/bin/python3"

But, still getting the same error.

one year ago
0 Hi There, We Have A Clearml Server And I Have Spun Up An Agent Using A Docker Image (As We Have Private Packages On Aws Codeartifact That Need Installing). I Have Created The

Ok, so the image needs ssh in order to clone the repo, not just the server.

This is my dockerfile:

# Use a slim Python image as the base image
FROM python:3.9.9-slim as builder

ARG CODEARTIFACT_AUTH_TOKEN
ARG POETRY_HTTP_BASIC_ARTIFACT_USERNAME
ARG POETRY_HTTP_BASIC_ARTIFACT_PASSWORD

# Set environment variables for Poetry
ENV POETRY_HOME=/opt/poetry \
    POETRY_NO_INTERACTION=1 \
    POETRY_VIRTUALENVS_CREATE=false \
    PATH="/opt/poetry/bin:$PATH"\
    PYTHONPATH="/ml_pipeline:${...
one year ago
0 Hi There, I Am Having Issues Executing A

Oh my word 🙈 Yes, I just saw this just and came back to reply on here. Thank you!

one year ago
0 Hi There, We Have A Clearml Server And I Have Spun Up An Agent Using A Docker Image (As We Have Private Packages On Aws Codeartifact That Need Installing). I Have Created The

Sorry, was trying to figure out how to do this. So, looks like I can't as get an error NameError: name 'ssh' is not defined

one year ago
0 Hi, We Are Using Clearml For Our Experiment Tracking But Now Investigating Using The Pipeline Functionality As Well For Scheduling. We Also Want To Be Able To Trigger A Pipeline Run When There Is New Data In An External Database. Is This Possible? From Wh

Hi @<1523701205467926528:profile|AgitatedDove14> , thanks for your reply. We want to use the Scheduler but to run a pipeline. Looking at this example here, it looks like it only works with tasks: None

So, in my code example above, where I have executing_pipeline as the pipeline function created with the decorator, can this be scheduled to run with the TaskScheduler , ie. used as the function...

one year ago
0 How Does One

Ah, I see, this makes sense now. Thank you!

one year ago
0 Hi There, I Am Running A Clearml-Agent And Now Trying To Run A Pipeline On A Schedule. When I Try Run The Schedule Using The Queue Assigned To The Worker I'Ve Got Running, I Get The Following Error About

This is schedule.py where the schedule_task_id is the id for a pipeline that has previously been run and now we want to schedule it:

from clearml import Task
from clearml.automation import TaskScheduler

if __name__ == "__main__":

    scheduler = TaskScheduler(force_create_task_name='megan-test-remote-pipeline')

    scheduler.add_task(
        name='Simple Pipeline Schedule Run Test',
        schedule_task_id='94a7e898bb3f4494a31828187710f5bd',
        queue='megan_testing',
...
one year ago
Show more results compactanswers