I've been trying to do things like "color these five experiments one color, color these other five a different color", but then once I maximize the thing the colors all change

Hello! integration in what sense? Training a model? Uploading a model to the hub? Something else?

Can you share the code?

Well, I can just work around it now that I know, by creating a folder with no subfolders and uploading that. But... 🤔 perhaps allow the interface to take in a list or generator? As in,
files_to_upload = [f for f in output_dir.glob("*") if f.is_file()] Task.current_task().upload_artifact( "best_checkpoint", artifact_object=files_to_upload)And then it could zip up the list and name it "best_checkpoint"?

Martin I found a different solution (hardcoding the parent tasks by hand), but I'm curious to hear what you discover!

This sort of behavior is what I was thinking about when I saw "wildcard or pathlib Path" listed as options

Yeah that should work. Basically in --train_file it needs the path to train.txt, --validation_file needs the path to validation.txt, etc. I just put them all in the same folder for convenience

So for example, I'm able to view in the UI that my finetuning task 7725f5bed94848039c68f2a3a573ded6 has an input model, and I can find the creating experiment for that. But how would I do this in code?

This should work. It has the tokenizer files, the train.txt, the validation.txt and a config.json

IrritableOwl63 pm'd you a task ID

Sure, if you want to give up that first-place spot! 😉

I did it the hard way

OK, so with the RC, the issue has gone away. I can now import torch without issue.

I will test both! Thanks for the ideas!

Oh, and good job starting your reference with an author that goes early in the alphabetical ordering, lol:

Or do you just want:
@misc{clearml, title = {ClearML - Your entire MLOps stack in one open-source tool}, year = {2019}, note = {Software available from }, url={ }, author = {ClearML}, }

¯_(ツ)_/¯

As an alternate solution, if I could group runs and get stats across the group, that would be cool

This seems similar but not quite the thing I'm looking for: https://allegro.ai/clearml/docs/docs/tutorials/tutorial_explicit_reporting.html#step-1-setting-an-output-destination-for-model-checkpoints

Yup! That works.
from joeynmt.training import train train("transformer_epo_eng_bpe4000.yaml")And it's tracking stuff successfully. Nice

https://github.com/allegroai/clearml/releases if anyone is curious!

awesome! Thanks!

BTW, http://clear.ml has this at the bottom:

It would certainly be nice to have. Lately I've heard of groups that do slices of datasets for distributed training, or who "stream" data.

Sure, I can give that a try!

So for example:
` {'output_dir': 'shiba_ner_trainer', 'overwrite_output_dir': False, 'do_train': True, 'do_eval': True, 'do_predict': True, 'evaluation_strategy': 'epoch', 'prediction_loss_only': False, 'per_device_train_batch_size': 16, 'per_device_eval_batch_size': 16, 'per_gpu_train_batch_size': None, 'per_gpu_eval_batch_size': None, 'gradient_accumulation_steps': 1, 'eval_accumulation_steps': None, 'learning_rate': 0.0004, 'weight_decay': 0.0, 'adam_beta1': 0.9, 'adam_beta2': 0.999, 'adam...

Or we could do
@misc{clearml, title = {ClearML - Your entire MLOps stack in one open-source tool}, year = {2019}, note = {Software available from }, url={ }, author = {Allegro AI}, }

Ah, should've specified we've got a ClearML server running