hi DizzyHippopotamus13
Yes you can generate a link to the experiments using this format.
However I would suggest you to use the SDK for more safety :
task = Task.get_task(project_name=xxx, task_name=xxx)
url = task.get_output_log_web_page()

Or in one line
url = Task.get_task(project_name=xxx, task_name=xxx).get_output_log_web_page()

hi ApprehensiveSeahorse83
i am working too on your issue. It seems that there is a wrong behavior here, so we need to get a bit deeper to understand what's happening. We will keep you updated asap, thanks for your contribution ! 🙏

Agent is a process that pulls task from a queue and assigns ressources (worker) to them. In the pipeline, when not runned locally, steps are enqueued tasks

Hi MotionlessCoral18
You need to run some scripts when migrating, to update your old experiments. I am going to try to find you soem examples

Hi PanickyMoth78
There is indeed a versioning mechanism available for the open source version 🎉

The datasets keep track of their "genealogy" so you can easily access the version that you need through its ID

In order to create a child dataset, you simply have to use the parameter "parent_datasets" when you create your dataset : have a look at
https://clear.ml/docs/latest/docs/clearml_data/clearml_data_sdk#datasetcreate

You also alternatively squash datasets together to create a c...

Hi EnormousWorm79

The Pycharm testrunner wraps the script into a local cript, and thats what you are getting.
(jb pytest runner). Because it is local, you lose the source info

Let me check if I have a workaround or solution for you. I keep you updated

Hi Max
you can configure a clearml agent to pull your docker image from ECR and run the experiment into it. Is that answering your question ?

Have you tried try to set your agent in conda mode ( https://clear.ml/docs/latest/docs/clearml_agent#conda-mode ) ?

btw here is the content of the imported file:

import torch
from torchvision import datasets, transforms
import os
MY_GLOBAL_VAR = 32

def my_dataloder ():
return torch.utils.data.DataLoader(
datasets.MNIST(os.path.join('./', 'data'), train=True, download=True,
transform=transforms.Compose([
transforms.ToTensor()
` ...

Hey LuckyKangaroo60
So far there isnt a CLI command to check the conf file format : if there is an error, it is detected from the beginning of the execution and the program fails. Here is what i use as a conf for accessing my local docker based minio :

`
s3 {
# S3 credentials, used for read/write access by various SDK elements

        # Default, used for any bucket not specified below
        region: ""
        # Specify explicit keys
        key: "david"
...

can you check that your server ports are opened ?

When the pipeline or any step is executed, a task is created, and it name will be taken from the decorator parameters. Additionally, for a step, the name parameter is optional : if not provided, the function name will be used instead.

It seems to me that your script fails creating the pipeline controller task because it fails pulling the name parameter. which is weird ... Weird because in the last error line, we can see that name !

hi MoodySheep3
I think that you use ParameterSet the way it is supposed to be 🙂
When I run my examples, I also get this warning - which is weird ! because
This is just a warning, the script continues anyway (and reaches end without issue) Those HP exists - and all the sub tasks corresponding to a given parameters set find them !

DepravedSheep68 you could also try to add the port to your uri.
Output_uri: "s3://...... : port"

Hi CheerfulGorilla72
You have an example of implementation here :
https://github.com/allegroai/clearml/tree/master/examples/services/monitoring

Hope it will help 🙂

can you also check that you can access the servers ?
try to do curl http://<my server>:port for your different servers ? and share the results 🙂

it is basically auto-generated when you do clearml-init
there are a bench of optional configurations that are not in the auto generated file though.
Have a look here it is pretty detailed https://clear.ml/docs/latest/docs/configs/clearml_conf

hi AbruptHedgehog21
clearml-serving will use your clearml.conf file
Configure it to access your s3 bucket - that is the place for bucket, host etc

Hey
unfortunatly there is no way to manage the CPU cores allocation for an agent

Hi JitteryParrot8
clearml-data is a tool for improving datasets management, including versioning. However it is not tracking modifications at the file level , so it would not track the evolution of a single file.
The idea is to create a dataset with your excel file, and each time you will change that file, you will create a new version of the dataset with the updated file. So that you could register the different versions of your file through the dataset version.

JuicyFox94

hi GrittyStarfish67
"Hi, love what you guys did with the new datasets!" Thanks 🙂 !

you can squash the datasets together : it will result in the creation of a child dataset, that will contain its parents data merged together. Note that there will be no duplicate upload of the parents data : when a dataset inherits from parents datasets, it receives the references to the data uploaded by the parents.
SDK: https://clear.ml/docs/latest/docs/clearml_data/clearml_data_sdk#datasetsquash
...

Of course. Here it is
https://github.com/allegroai/clearml/issues/684
I'll keep you updated

hey Maximilian,
which version of clearml are you using ?

hi FiercePenguin76
Can you also send your clearml packages versions ?
I would like to sum your issue up , so that you could check i got it right

you have a task that has a model, that you use to make some inference on a dataset you clone the task, and would like to make inferences on the dataset, but with another modelthe problem is that you have a cloned task with the first model....

How have you registered the second model ? Also can you share your logs ?

Hi UnevenDolphin73
you can find AMI that could satisfy your needs in the AWS website. Simply enter a console, choose your region (upper-right) , enter EC2 -> launch instance
There you can choose your os, or type "deep learning" if you need GPU machines

If you need some help to find more specific machines, do not hesitate to ask me :)

Great !
Concerning the running status :
in the first case, the program failed so the server has no way to be informed of a status change in the second case, this is not a task status but a dataset, so the status would change when you will publish the datasetThe fix is in last phases of testing, I hope that it will be released very soon

yes i have the behavior. I think that we have a bug. We will release a fix and will keep you updated 🙂
can you please open a github issue ? Of course if it is a problem, i can do it for you, just let me know 🙂
http://github.com/allegroai/clearml/issues

can you show the logs ?