Final answer was

    docker="ultralytics/ultralytics:latest",
    docker_args=["--network=host", "--ipc=host"],

pip install ultralytics --no-deps would also work. Is there a way to pass this to clearML?

@<1717350332247314432:profile|WittySeal70> what's strange is I can import the package in the docker container when I run it outside of clearML

How to replicate on ClearML:

task = Task.create(
    script="myscript.py",
    packages=["opencv-python==4.6.*", "ultralytics"],
    docker="nvcr.io/nvidia/pytorch:22.12-py3",
)

Contents of myscript.py:
from ultralytics import YOLO

What does ClearML do differently that leads to a failure here?

probably you have to have ffmpeg libsm6 libxext6
packages installed in the docker container where you use opencv.

On local I am able to import ultralytics in this docker image
docker run --gpus 1 -it nvcr.io/nvidia/pytorch:22.12-py3
# pip install opencv-python==4.6.* ultralytics
# python

>>> from ultralytics import YOLO
>>> 

Using docker="ultralytics/ultralytics:latest" and docker_args=["--privileged"] seems to work!