The tl;dr is that some of our users like poetry and others prefer pip . Since pip install git+.... stores the git data, it seems trivial to first try and install based on pip , and only later on poetry , since the pip would crash with poetry as it stores git data elsewhere (in poetry.lock )

For example:

my-repo @ git+

Using an on-perm clearml server, latest published version

Looks great, looking forward to the all the new treats 😉
Happy new year! 🎉

Internally yes, but in Task.init the default argument is a boolean, not an int.
We don't want to close the task, but we have a remote task that spawns more tasks. With this change, subsequent calls to Task.init fail because it goes in the deferred init clause and fails on validate_defaults .

Parquet file in this instance (used to be CSV, but that was even larger as everything is stored as a string...)

SuccessfulKoala55 TimelyPenguin76
After looking into it, I think it's because our AMI does not have docker, and that the default instance suggested by ClearML auto scaler example is outdated

QuaintPelican38 did you have a workaround for this then? Some cleanup service or similar?

JitteryCoyote63 please do not get used to it :D there's an open ticket/feature request to either revert this or let the user/server choose the most comfortable way

I just ran into this too recently. Are you passing these also in the extra_clearml_conf for the autoscaler?

I'm not sure, I'm not getting anything (this is the only thing I could fin that's weird about this project).
It has a space in the name, has no subprojects, and it just doesn't show up anywhere 🤔

Perfect now 👌 (also nice cleanup of default_new_data_root duplicate code :D)

Would be good if that's mentioned explicitly in the docs 😄 Thanks!

IIRC, get_local_copy() downloads a local copy and returns the path to the downloaded file. So you might be interested in e.g.
local_csv = pd.read_csv(a_task.artifacts['train_data'].get_local_copy())
With the models, you're looking for get_weights() . It acts the same as get_local_copy() , so it returns a path.

EDIT: I think also get_local_copy() for a model should work 👍

Follow-up question/feature request (out of interest) - could the WebUI show the matching commit message?

In which repo?:)

Also I can't select any tasks from the dashboard search results 😞

2. I guess? 🤔 I mean the same filter option one has for e.g. tags in the table view. In the "all experiments" project I think it would make sense for one to be able to select the projects of interest, or even filter for textual matches.

3. Sorry I meant the cards indeed :)

Also (sorry for all of these!) - could be nice to have a direct "task comparison" link in the UI somewhere, that would open a comparison with no tasks and the user can add them manually using the "add experiments" button. :)

I can't seem to manage the first way around. If I select tasks in different projects, I don't get the bottom bar offering to compare between them

Hey AgitatedDove14 🙂
Finally managed; you keep saying "all projects" but you meant the "All Experiments" project instead. That's a good start 👍 Thanks!

Couple of thoughts from this experience:
Could we add a comparison feature directly from the search results (Dashboard view -> search -> highlight some experiments for comparison)? Could we add a filter on the project name in the "All Experiments" project? Could we add the project for each of the search results? (see above pictur...

I have no idea what’s the difference, but it does not log the internal repository 😞 If I knew why, I would be able to solve it myself… hehe

There's no decorator, just e.g.

def helper(foo: Optional[Any] = None):
    return foo

def step_one(...):
    # stuff

Then the type hints are not removed from helper and the code immediately crashes when being run

Alternatively, it would be good to specify both some requirements and auto-detect 🤔

The only thing I could think of is that the output of pip freeze would be a URL?

Well the individual tasks do not seem to have the expected environment.

Using the PipelineController with add_function_step

Yes. Though again, just highlighting the naming of foo-mod is arbitrary. The actual module simply has a folder structured with an implicit namespace:

foo/
  mod/
    __init__.py
    # stuff

FWIW, for the time being I’m just setting the packages to all the packages the pipeline tasks sees with:

    packages = get_installed_pkgs_detail()
    packages = [f"{name}=={version}" if version else name for name, version in packages.values()]
    packages = task.data.script.require...

Hey @<1523701205467926528:profile|AgitatedDove14> , thanks for the reply!

We would like to avoid dockerizing all our repositories. And for the time being we have not used the decorators, but we can do that too.
The pipeline is instead built dynamically at the moment.

The issue is that the components do not have their dependency. For example:

def step_one(...):
    from internal.repo import private
    # do stuff

When step_one is added as a component to the pipeline, it does ...

It is. In what format should I specify it? Would this enforce that package on various components? Would it then no longer capture import statements?