Unanswered
Hey Has Anyone Managed To Capture Darts Logging With Clearml When Using The Temporal Fusion Transformers ? Even When Overriding Their Trainer With A Custom Pytorch Lightning Trainer It Seems That Clearml Cannot Retrieve The Iteration Log...
The expected behavior is that the task would capture the iteration scalar of the PL trainer but nothing is recorded
import clearml
from darts.models import TFTModel
model = TFTModel(
input_chunk_length=28,
output_chunk_length=14,
n_epochs=300,
batch_size=4096,
add_relative_index=True,
num_attention_heads=4,
dropout=0.3,
full_attention=True,
save_checkpoints=True,
)
task = Task.init(
project_name='sales-prediction',
task_name='TFT Training 2',
task_type=Task.TaskTypes.training
)
trainer = pl.Trainer(
max_epochs=300,
enable_progress_bar=True,
callbacks=[EarlyStopping(monitor='val_loss', patience=5, verbose=True), ModelCheckpoint(monitor='val_loss', verbose=True)],
precision=64,
accelerator='gpu',
devices=[0]
)
model.fit(series=train_series, val_series=val_series, trainer=trainer)
task.flush()
task.close()
167 Views
0
Answers
one year ago
one year ago