New to clearml, I primarily use rye to handle my package management. For some reason, our training task is capturing all of our depenendencies as declared in our pyproject.toml , requirements.lock , and requirements.txt , but each task we keep launching produces this error:

ClearML results page: 

INFO:root:WaveFakeDataset - loading samples from 

...
Traceback (most recent call last):
  File "/Users/amanmibra/.rye/py/cpython@3.10.13/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/Users/amanmibra/.rye/py/cpython@3.10.13/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/Users/amanmibra/.clearml/venvs-builds/3.10/task_repository/pipelines.git/tasks/rawnet3/train_with_trainer.py", line 37, in <module>
    main()
  File "/Users/amanmibra/.clearml/venvs-builds/3.10/task_repository/pipelines.git/tasks/rawnet3/train_with_trainer.py", line 17, in main
    train_dataset = WaveFakeDataset(subset="train")
  File "/Users/amanmibra/.clearml/venvs-builds/3.10/task_repository/pipelines.git/datasets/wavefake.py", line 59, in __init__
    self.samples = self._load_samples()
  File "/Users/amanmibra/.clearml/venvs-builds/3.10/task_repository/pipelines.git/datasets/wavefake.py", line 63, in _load_samples
    return pd.read_parquet(self.path)
  File "/Users/amanmibra/.clearml/venvs-builds/3.10/lib/python3.10/site-packages/pandas/io/parquet.py", line 651, in read_parquet
    impl = get_engine(engine)
  File "/Users/amanmibra/.clearml/venvs-builds/3.10/lib/python3.10/site-packages/pandas/io/parquet.py", line 67, in get_engine
    raise ImportError(
ImportError: Unable to find a usable engine; tried using: 'pyarrow', 'fastparquet'.
A suitable version of pyarrow or fastparquet is required for parquet support.
Trying to import the above resulted in these errors:
 - Missing optional dependency 'pyarrow'. pyarrow is required for parquet support. Use pip or conda to install pyarrow.
 - Missing optional dependency 'fastparquet'. fastparquet is required for parquet support. Use pip or conda to install fastparquet.

I am noticing in the logs before when its spinning up the container that it's listing all of the deps except these two. I have tried changing the project manage in my conf file between pip and poetry since the poetry setting referes to pyproject.toml , but right now, I am stumped