Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Profile picture
IrritableGiraffe81
Moderator
8 Questions, 29 Answers
  Active since 10 January 2023
  Last activity one year ago

Reputation

0

Badges 1

15 × Eureka!
0 Votes
2 Answers
784 Views
0 Votes 2 Answers 784 Views
Hi there, How can I set the model metadata using code? The Model object has the Model.set_all_metadata , but I am not sure how to access it from the Task .
one year ago
0 Votes
12 Answers
648 Views
0 Votes 12 Answers 648 Views
Hi there, I have a package called feast[redis] in my requirements.txt file. When I run locally everything works, but from UI it does not list in INSTALLED PA...
one year ago
0 Votes
2 Answers
664 Views
0 Votes 2 Answers 664 Views
Hi guys, So, the docker-compose available here: https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server_linux_mac Deploys all clearml stack? inclu...
2 years ago
0 Votes
3 Answers
752 Views
0 Votes 3 Answers 752 Views
Hi there, As a last step of the model training pipeline, I upload it to ClearML and set the auto_delete_file filepath = f'models/{model_id}/model.sav' joblib...
one year ago
0 Votes
4 Answers
664 Views
0 Votes 4 Answers 664 Views
Hi there, I have a pipeline that query data from a Neo4J database. When I run it using PipelineDecorator.debug_pipeline() it runs just fine, but when I use P...
one year ago
0 Votes
3 Answers
661 Views
0 Votes 3 Answers 661 Views
Hi community, I’ve just posted my first blog post about MLOps. I am open to any suggestions. https://cpatrickalves.com/mlops-what-it-is-and-why-it-matters
2 years ago
0 Votes
2 Answers
641 Views
0 Votes 2 Answers 641 Views
Hi there, I have a batch prediction Task that load a model published on ClearML. input_model = InputModel(model_id=model_id) model_path = input_model.get_loc...
one year ago
0 Votes
1 Answers
694 Views
0 Votes 1 Answers 694 Views
Hi there, Has anyone running clearml-agent inside a docker container? Would you mind to share your Dockerfile?
one year ago
0 Hi There, I Have A Package Called

AgitatedDove14 Worked!

But a new error raises:

` File "kgraph/pipelines/token_join/train/pipeline.py", line 48, in main
timestamp = pd.to_datetime(data_timestamp) if data_timestamp is not None else get_latest_version(feature_view_name)
File "/root/.clearml/venvs-builds/3.8/task_repository/Data-Science/kgraph/featurestore/query_data.py", line 77, in get_latest_version
fv = store.get_feature_view(fv_name)
File "/root/.clearml/venvs-builds/3.8/lib/python3.8/site-packages/feast/u...

one year ago
0 Hi There, I Have A Batch Prediction Task That Load A Model Published On Clearml.

Thanks Martin, your suggestion solves the problem.
👍

one year ago
0 Hi Community, I’Ve Just Posted My First Blog Post About Mlops. I Am Open To Any Suggestions.

SubstantialElk6
Only today I've saw your comments (did not get notified for some reason)
Thanks for you suggestions

one year ago
0 Hi There, I Have A Package Called

AgitatedDove14 Thanks for the explanation
I got it.
How I can use force_requirements_env_freeze with PipelineDecorator()
as I do not have the Task object created.
@PipelineDecorator.pipeline(name='training', project='kgraph', version='1.2') def main(feature_view_name, data_timestamp=None, tk_list=None): """Pipeline to train ...

one year ago
0 Hi There, I Have A Package Called

I've build a container using the same image used by agent.
Training ran with no errors

one year ago
0 Hi There, I Have A Package Called

Go it!
Thanks a lot AgitatedDove14
I will try !

one year ago
0 Hi There, I Have A Package Called

I don’t think so AgitatedDove14
I’ve tested with:

PipelineDecorator.debug_pipeline() PipelineDecorator.run_locally() Docker
I’ve got no error

one year ago
0 Hi There. When Trying To Launch My Specific Docker, It Fails Launching Clientml-Agent Inside The Container Due To This...

Hi MotionlessCoral18
Are you running the agent inside a container?
Would you mind to share your Dockerfile?

one year ago
0 Hi There, I Have A Package Called

Thanks for the reply, I will send you soon.

one year ago
0 Hi. I'M Running This Little Pipeline:

Got it.
Thanks for explanation AgitatedDove14 ! 😀

one year ago
0 Hi. I'M Running This Little Pipeline:

this will cause them to get serialized to the local machine’s file system, wdyt?

I am about the disk space usage that may increase over time.
I just prefer do not worry about that

one year ago
0 Hi. I'M Running This Little Pipeline:

that makes sense, so why don’t you point to the feature store ?

I did, the first step of the pipeline query the feature store. I mean, I set the data version as a parameter, then this steps query the data and return it (to be used in the next step)

one year ago
0 Hi. I'M Running This Little Pipeline:

These are the steps of the pipeline

one year ago
0 2. Is There A Case-Study Or Ref. Architecture For Interacting With Ci/Cd I.E. Exposing Mature Pipelines To Be Triggered Upon Code Pushes (Taking Latest Git Hash) Or With Manual Ci Triggers?

AgitatedDove14 , thanks for the quick answer.

I think this is the easiest way, basically the CI/CD launches a pipeline (which under the hood is another type of Task), by querying the latest “Published” pipeline that is also Not archived, then cloning+pushing it to execution queue

Do you have an example?

UI when you want to “upgrade” the production pipeline you just right click “Publish” on the pipeline

I’ve did saw this “publish” option for pipelines, just for models, is thi...

one year ago
0 2. Is There A Case-Study Or Ref. Architecture For Interacting With Ci/Cd I.E. Exposing Mature Pipelines To Be Triggered Upon Code Pushes (Taking Latest Git Hash) Or With Manual Ci Triggers?

AgitatedDove14
How do you recommend to perform this task?
I mean, have a CI/CD (e.g Github Actions) thats update my “production” pipeline on ClearML UI, so a Data Scientist can start to experiment things and create jobs from the UI.

one year ago
0 Hi There, As A Last Step Of The Model Training Pipeline, I Upload It To Clearml And Set The

This is not a valid parameter: https://clear.ml/docs/latest/docs/references/sdk/task#taskinit

Also I did not find any usage example of the setup_upload method

Thanks anyway

one year ago
0 Hi. I'M Running This Little Pipeline:

The transformation has nome parameters that we change eventually
I could merge some steps, but as I may want to cache them in the future, I prefer to keep them separate

one year ago
0 Hi. I'M Running This Little Pipeline:

I see now.
I didn’t know that each steps runs in a different process
Thus, the return data from step 2 needs to be available somewhere to be used in step 3.

one year ago
0 Hi. I'M Running This Little Pipeline:

So, how wrap the returns in a dict could be a solution?
It will serialize the data on the dict? (leading to the same result, data storage somewhere)

one year ago
0 Hi. I'M Running This Little Pipeline:

Hi there,

PanickyMoth78
I am having the same issue.
Some steps of the pipeline create huge datasets (some GBs) that I don’t want to upload or save.
Wrap the returns in a dict could be a solution, but honestly, I don’t like it.

AgitatedDove14 Is there any better way to avoid the upload of some artifacts of pipeline steps?

The image above shows an example of the first step of a training pipeline, that queries data from a feature store.
It gets the DataFrame, zip and upload it (this one i...

one year ago
0 Hi. I'M Running This Little Pipeline:

Pipelines runs on the same machine.
We already have the feature-store to save all data, that’s why I don’t need to save it (just a reference of version of dataset).

I understand your point.
I can have different steps of the pipeline running on different machines. But this is not my use case.

one year ago
0 Hi There, I Have A Package Called

I've also tried with clearml-1.6.5rc2, got same error
I am lost 😔

one year ago
0 Hi There, I Have A Pipeline That Query Data From A Neo4J Database. When I Run It Using

Found the issue.
For some reason, all parameters on the main functions are passed as strings.

So I have these parameters:

@PipelineDecorator.pipeline(name='Build Embeddings', project='kgraph', version='1.3') def main(tk_list=[], ngram_size=2): ...
The ngram_size variable is a int when using PipelineDecorator.debug_pipeline() and it is a string when I used PipelineDecorator.run_locally()

I’ve add Python type hints and it fixed the issues:
` def main(tk_list:list = [], ngram...

one year ago
0 Hi There, I Have A Pipeline That Query Data From A Neo4J Database. When I Run It Using

AgitatedDove14 is that the expect behavior for Pipelines?

one year ago