AgitatedDove14

49 Questions, 8122 Answers

Active since 10 January 2023

Last activity one year ago

Reputation

Badges 1

25 × Eureka!

Answers 8122

0 Hello, I Am Getting `Valueerror: Could Not Get Access Credentials For '

And the correct region ?

4 years ago

0 Hello, I Am Getting `Valueerror: Could Not Get Access Credentials For '

Any chance ?

4 years ago

0 Another Issue Is The Agent Uses Python 2 For Some Reason Even Though Locally I’M Using Python 3 And The Agent Is Supposed To Use A Python 3 Venv.

Can you send the full log? This is odd, it will by default use the python executable it (the agent) is running with.
Regardless you can specify the python executable to be used here:
https://github.com/allegroai/clearml-agent/blob/bd411a19843fbb1e063b131e830a4515233bdf04/docs/clearml.conf#L44

4 years ago

0 Hello, I Am Getting `Valueerror: Could Not Get Access Credentials For '

What's the exact error you are getting ?
(Maybe this is privilege error on the cache folder, what are the folders it is using, you can see in the configuration as well)

4 years ago

0 Hi Everyone. I Have An Issue With The Simple Pipeline - It Runs Two Similar Nn Training Steps (Tf2.3, Windows10, Python 3.7) With Only Difference Is A Batch Size. I'M Running First Separately Each Step To Have Them In Clearml Project Page. Then I Run Pipe

BattyLion34 let me see if I understand.
The same base_task_id when cloned by the UI and enqueues on the same queue as the pipeline, will work but when the pipeline runs the same Task it fails?!
Could it be that you enqueue them on different queues ?

4 years ago

0 Given These Are Settled.. Another Question I Have Is About The Job Scheduling Based On Cron Style.. E.G. Run Training Every Night At 2 Am Etc.

Meanwhile you can just sleep for 24hours and put it all on the services queue. it should work 🙂
Example here:
https://github.com/allegroai/trains/blob/master/examples/services/cleanup/cleanup_service.py

5 years ago

0 I’M Using Catboost For Training, But Sadly It Does Not Have A Native Integration With Clearml (Xgboost And Lightgbm Do Have Integrations). But Catboost Writes Down Training Logs In Tensorboard Format (Into A

Hmm I think everything is generated inside the c++ library code, and python is just an external interface. That means there is no was to collect the metrics as they are created (i.e. inside the c++ code), which means the only was to collect them is to actively analyze/read the tfrecord created by catboost 😞
Is there a python code that does that (reads the tfrecords it creates) ?

4 years ago

0 Hello! Thank You All For Your Work! I Have A Question (Which Is Probably Not Clearml Related At All). I Am Using Clearml-Agent Running In Docker Mode On Several Machines With Gpu In Our Local Network And Get Different Behaviour Depending On How I Logged I

I think the main issue is that for some reason the container running changed one of the files inside the temp folder. then the host machine is "stuck" with a file that the root user owned/changed, and now it cannot reuse / delete the temp folder.
I think the fix is to make sure the container deleted the temp folder when it is done

3 years ago

0 Hi All, I'Ve Successfully Run A Task Locally, And Now I'M Trying To Clone It And Send It To A Queue. It Looks Like The Environment Is Built Successfully, But It Hangs Here:

Retrying (Retry(total=239, connect=240, read=240, redirect=240, status=240)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)'))': /auth.login

OH that makes sense I'm assuming on your local machine the certificate is installed but not on remote machines / containers
Add the following to your clearml.conf:

api.verify_certificate: false

[None](https...

one year ago

0 Hey, Don'T Really Understand Why The Clearml Worker Needs To Pull The Repository Where My Pipeline (Defined With Decorators) Is Written Is Since Apparently A Temporary Python File (Containing At Least The Code And Imports For The Executed Component) Seems

This is odd I was running the example code from:
https://github.com/allegroai/clearml/blob/master/examples/pipeline/pipeline_from_decorator.py
It is stored inside a repo, but the steps that are created (i.e. checking the Task that is created) do not have any repo linked to them.
What's the difference ?

2 years ago

0 Hi, Is It Intented Behavior That Models That Are Saved By A Clearml-Agent Will Have The Clearml-Agents User (So The User Of Which Generated The Api Credentials For The Agent) In The "User" Field Of The Model Instead Of The User Who Started The Task?

There is no way to create an artifact/model/dataset without a task, right?

Models are a an entity of it's own, and you can actually create one without a Task.

(just for my own interest: how much does the enterprise version divert from the open source version? It it just extended or are there core changes to the enterprise version)

It adds a few security layers on top, and adds a few features that are just not part of the open source (RBAC, hyper-datasets, advanced scheduling, cu...

3 years ago

0 Hello All! I Have Some Trouble With Running Remotely Task With Code From Gitlab Repo With Ssl Cert. On The Machine Where Clearml Agent Installed Cert Is Added And Repo Cloning Successfully, But When I Tried To Run Task - It Failing With Git Repo Cloning F

that should have worked, do you want send the log?

one year ago

0 Hey I Use Allegro With Docker Mode. But I Do Not Have Access To Paths Where The Data Are(Data I Use For Training). How Can I Use "Volume Mount" With Allegro?

Hi UnsightlySeagull42
Basically you can get the agent to always add additional arguments for the docker run, such as -v for mounting:
https://github.com/allegroai/clearml-agent/blob/948fc4c6ce1ecf33a74619ad570d69b8188f6db9/docs/clearml.conf#L133

4 years ago

0 Hello! Getting Credential Errors When Attempting To Pip Install Transformers From Git Repo, On A Gpu Queue.

Let's try:
` echo 'Binary::apt::APT::Keep-Downloaded-Packages "true";' > /etc/apt/apt.conf.d/docker-clean ; chown -R root /root/.cache/pip ; export DEBIAN_FRONTEND=noninteractive ; export CLEARML_APT_INSTALL="$CLEARML_APT_INSTALL libsm6 libxext6 libxrender-dev libglib2.0-0" ; [ ! -z $(which git) ] || export CLEARML_APT_INSTALL="$CLEARML_APT_INSTALL git" ; declare LOCAL_PYTHON ; for i in {10..5}; do which python3.$i && python3.$i -m pip --version && export LOCAL_PYTHON=$(which python3.$i) && b...

4 years ago

0 Logger.Report_Image And Logger.Report_Matplotlib_Figure Not Working

👍

3 years ago

0 Another Issue Is The Agent Uses Python 2 For Some Reason Even Though Locally I’M Using Python 3 And The Agent Is Supposed To Use A Python 3 Venv.

that clearml-agent needs to be installed from system python mentioned anywhere in the docs, if not I suggest it gets added.

You are right, I will check and fix if not 🙂

Thank you so much for helping.

My pleasure

4 years ago

0 Hey - Is There A Known Issue With Tf.Summary.Image Not Being Logged When There'S A 'Description' Argument?

is there a code that can reproduce it ?

4 years ago

0 Hi All, We Have A Weird Inconsistency. We Have A Clearml Server Installed On-Prem And Started Playing With It. Using The Dataset.Create Command And The Subsequent Add_Files, And Upload Commands I Can See The Upload Action As An Experiment But The Data Is

Using the dataset.create command and the subsequent add_files, and upload commands I can see the upload action as an experiment but the data is not seen in the Datasets webpage.

ScantCrab97 it might be that you need the latest clearml package installed on the client end (as well as the new server with the UI)
What is your clearml package version ?

3 years ago

0 Hi! I Have A Freshly Deployed Clearml Instance. In The Docs I Found A Phrase

Hi @<1547390415320125440:profile|SilkySparrow85>

because it is trying to send a debug-sample to fileserver!

Yes, you should always configure the "files server" to point to your minio S3, basically:
None

files_server: "

But do not forget to also configure the credentials here:
[None](https://github.com/allegroai/clearml/blob/40c6db9d95016382c721546d42...

2 years ago

0 Hi There, I Have A Batch Prediction Task That Load A Model Published On Clearml.

Hi IrritableGiraffe81
Can you share a code snippet ?
Generally I would try
task = Task.init(..., auto_connect_frameworks={"pytorch': False, 'tensorflow': False)

2 years ago

0 I Have No Prior Devops Experience. I'Ve Been Able To Set Up A Simple Continuous Training Setup Using Clearml. I Wanted To Ask What Should I Learn Which Would Help Me Move A Project From Mlops Level 0 To Level 1, And Then Level 2, Using Clear Ml. I Would A

I'm kind of at a point where I don't know a lot of what to even search for.

we feel you 💗 , yes there still isn't a very good source of information on where to get started...
This is because the entire field is constantly changing and evolving, and one solution will usually only apply to specific use case...
I would start with the mlops community slack channel, and youtube talks (specifically those by companies describe how they built their own internal infrastructure, i...

3 years ago

0 Hello! I Get The Following Error In Results->Console After A Task Is Sent For Remote Execution (Using Sdk):

Yey?

2 years ago

0 Hey Guys, I'M Hosting A Private Server Configured To Link To A S3 Bucket. I'M Having Difficulties Identifying The Reason For An Error In A Worker Recurring Repeatedly (Shown In The Screenshots Attached). It Basically Uses The Same Clearml.Conf File As The

Hi AverageBee39
It seems the json is corrupted, could that be ?

3 years ago

0 Hi Clm Community, I Am Having An Issue With A Private Package Install When Using Clearml-Agent By Env (Not Docker). Things I Have Tried: Adding The Package Repository To "Extra_Index_Url" Adding The Conda Env To "Python_Binary" Listing The Libraries To I

FlatStarfish45

In the parent task, the libs appear installed.

What do you mean by "parent Task"? Is this the base task we are optimizing (i.e. the experiment / model we are optimizing) ?
Or is it the "Optimization Task" itself?

3 years ago

0 Hi There, I'Ve Encountered A Problematic Behavior In Python. When Defining An Argument A Default Value Of

PompousBeetle71 is this ArgParser argument or a connected dictionary ?

5 years ago

0 I Am Looking For A "Hello World" Example To Do 3 Tasks: Data=Preprocessdata() //Local Processing Model=Trainmodel(Data) // On Cloud Processing On A Custom Container Say Using Pytorch Or Keras Results=Evaluatemodel(Model, Testdata) //Local Processing

LazyLeopard18 you can point the artifact directly to your azure object storage and have StorageManager download and cache it for you:

5 years ago

another option is the download fails (i.e. missing credentials on the client side, i.e. clearml.conf)

3 years ago

0 I'M Using Tensorboard Summarywriter To Add Scalar Metrics For The Experiment. If Experiment Crashed, And I Want To Continue It From Checkpoint, For Some Reason It Plots Metrics In A Really Weird Way. Even Though I Pass Global_Step=Epoch To The Summarywrit

I think we should open a GitHub Issue and get some more feedback, maybe we should just add support in the backend side ?

3 years ago

0 Hi There, I'Ve Encountered A Problematic Behavior In Python. When Defining An Argument A Default Value Of

Hmm, I still wonder what is the "correct" answer for most people, is empty string in argparse redundant anyhow? will someone ever use it?

5 years ago

0 Can Anyone Point Me To Web Ui Source Code Out, I’M Wondering How Can I Customizing Web App Ui A Little Bit

Hi MagnificentPig49 unfortunately it's only in the trains-server docker, we are working on making it "presentable" and uploading it to it's repo.
It's written in Angular (v8 I think). Do you want to help out, it will definitely incentive the guys to tidy up the code and upload it :)

5 years ago

Show more results