Apple's Depth Pro 2: Four open models measure depth in meters from photos and video
Apple has released Depth Pro 2 on Hugging Face and GitHub. It's a set of four models that take a single photo or video and estimate how far away every pixel is, in actual meters. They run on Nvidia GPUs, on Apple Silicon Macs and even on the CPU. The catch is the license, which only allows non-commercial research.

What's going on
Apple has released Depth Pro 2, the successor to its Depth Pro depth estimator. The package includes four depth models in different sizes, a small focal length estimator and a command-line tool. It turns photos, videos and live camera feeds into metric depth maps: images in which every pixel stores its distance from the camera in meters. The weights are in the apple/DepthPro2 collection on Hugging Face, and the code is in the GitHub repository github.com/apple-aiml-research/ml-depth-pro-2.
There's one catch. The weights are released under the Apple Machine Learning Research Model License. According to the README, it permits use "exclusively for research purposes", meaning non-commercial research and academic work. The details here come from a report by explainx.ai, which went through the Model Card, the README and the project page.
What "monocular metric depth" means
Monocular means one camera and one image. There's no stereo camera pair and no LiDAR sensor like the laser scanner in some iPhones. The model has to infer distance from cues in the image itself, such as perspective, texture and the known size of everyday objects.
Metric means the output comes in real units. A relative depth map only tells you the lamp is closer than the wall. A metric one tells you the lamp is 1.8 meters away. You need that before you can measure a room, place virtual objects at the right size or steer a robot.
This is hard because scale is ambiguous. A model house and a real house can look almost identical in a photo. That's why Apple includes a focal length estimator. It has 23.5 million parameters (about 94 MB) and predicts the camera's focal length in pixels. You can only convert pixel measurements into meters if you know the focal length, and many images and videos lack reliable metadata. You load it with FOVEstimator.from_pretrained("apple/DepthPro2-focal").
What's new: sharper edges and steadier video
According to the Model Card, the technical report is titled "Depth Pro 2: Sharp and Temporally Consistent Metric Monocular Depth". So the focus is on two properties.
Sharp refers to object edges. Many depth models blur the transitions between objects. With effects like portrait-mode background blur or 3D parallax, that shows up straight away as a halo around a person. The project page says Depth Pro 2 beats earlier work on Boundary F1, a metric for how closely the predicted depth edges match the real ones.
Temporally consistent refers to video. If you run a single-image model on every frame, the depth flickers, because even small changes in the image lead to different estimates. Depth Pro 2 is also supposed to lead on jitter, the metric that measures this frame-to-frame wobble. Apple says both figures are averaged over the Sintel and Spring datasets. The page doesn't say which models Apple compared against or what the actual scores were.
Apple says little about the architecture. A multi-view transformer processes one or more frames together. After the first third of its blocks, it starts alternating: some blocks look at relationships within a single image, others at relationships across images. A DPT decoder then produces an inverse depth map for each frame. Put simply, the model sees several frames at once, which lets it stay consistent with itself over time. Video is processed either in buffered clips of 16 frames with a 4-frame overlap, or frame by frame as a live stream.
The four sizes
| Model | Parameters | Download | Intended for |
|---|---|---|---|
| S | 24M | 97 MB | Laptops, quick tests, live use |
| B | 97M | 386 MB | Middle ground if S looks too soft |
| L | 334M | 1.3 GB | Better quality on a desktop GPU |
| H+ | 872M | 3.5 GB | Default, best quality, full-resolution video |
On Nvidia and Apple GPUs, the code runs in bfloat16, a compact number format. On the CPU it uses float32. Hugging Face also hosts Core ML exports, versions built for Apple's own machine learning framework. The repository also includes a live depth demo for the Mac. It requires Apple Silicon, macOS 15 or later and Xcode 16 or later.
Try it
The README quickstart needs the Python tool uv (docs.astral.sh/uv):
git clone https://github.com/apple-aiml-research/ml-depth-pro-2
cd ml-depth-pro-2
uv run depth-pro-2 predict -i assets/example.jpg -o out/
You get a color depth PNG, or an MP4 file for videos. Options let you also save 16-bit PNGs with inverse depth and NPZ files containing depth in meters and focal length in pixels. For your own code, the package offers a Python API with the functions create_model_and_transforms, infer, infer_video and stream. The large model is called apple/DepthPro2-H on Hugging Face.
One important detail: a video clip gets a single focal length for all frames. If the camera zooms during the clip, that assumption no longer holds and the metric scale drifts.
What you need to try it
The barrier to entry is low. You don't strictly need a graphics card, because the models also run on the CPU, though the large variants are slow there. According to the README, the small S model at 97 MB is suitable for laptops. The largest model, H+, is 3.5 GB. An Nvidia card with about 4 GB of VRAM or an Apple Silicon Mac with 16 GB of memory should do fine (that's our estimate, since Apple doesn't list minimum requirements). High-resolution video needs more. About 5 to 6 GB of disk space covers all four models. On Macs it runs on the GPU (MPS), and the live demo needs macOS 15 and Xcode 16. There are no speed figures, so you'll have to benchmark it yourself.
What's still missing
A lot is still open. There's no exact release date, no benchmark table, no frame rates for the individual models and devices, and no direct comparison with Depth Pro 1. The arXiv link to the technical report is still a placeholder. The training and evaluation code is missing, so you can neither verify the results nor fine-tune on your own data. When explainx.ai checked, the GitHub project had no stars and no forks, so there are no community tests yet. The video comparisons on the project page use 2K to 4K footage. H+ runs at native resolution there, while the competing models run at their default settings. That shows off the sharpness nicely, but it isn't a controlled comparison.
The bigger picture
Depth estimation is a building block for lots of applications: portrait blur in video, augmented reality, robotics, 3D reconstruction and camera tracking. A model that outputs real meters, doesn't flicker on video and runs on a laptop in a 97 MB version is very interesting for researchers and hobbyists. The Core ML exports make it especially appealing for Mac users.
So far, though, Apple's claims are only qualitative, and the gallery on the project page comes from Apple itself. You'll only have something solid once the paper and real numbers arrive. The license rules out any commercial use, including internal tools that support revenue. That's Apple's familiar pattern: researchers get the weights, and Apple keeps commercial use for its own products. It's worth a look for prototypes and experiments, but not for products for now.
Sources
- Google News (IN) explainx.ai
Try it yourself: Depth Pro 2 H+
Explained step by step, no prior knowledge needed – on your own computer or free in your browser.
What can I do with it?
Depth Pro 2 turns a regular photo, video or live camera feed into a depth map that records, for every pixel, how many meters it is from the camera. You get a color depth image (PNG) or a depth video (MP4). If you want, you can also get raw data with meters and focal length as an NPZ file, which you can process further in Python or 3D software. Apple says the edges are especially sharp and the depth barely flickers in video, because the model looks at several frames together. A bundled focal length estimator means even images without camera metadata can be converted to meters. Limitations: if the camera zooms during a video, the scale is off. There are no independent benchmarks yet, and the license forbids any commercial use.
Who is it worth it for?
It's interesting for students, researchers and hobbyists who want to experiment with 3D effects, augmented reality, robotics or photography. Mac users benefit from the Core ML exports and the live demo. If you want to build something for a product, a paid app or your company, the research license means you can't use the weights.
Ideas: What you can do with it
Run a phone video through the model and use the depth map as a mask in a video editor. That lets you blur the background like a cinematic portrait mode. Since the edges are supposed to be sharp, you get fewer halos around people.
Take a photo of a room and have the tool output the NPZ file with meters. In Python, you can read off how far away the wall, sofa or door is. It's not good enough for exact measurements, but it works for a first estimate.
With a photo and its depth map, you can build a 2.5D effect in tools like Blender, where foreground and background move differently as the camera pans. That turns a still image into a small animation.
Hobby robots with a simple webcam can get metric depth estimates through the stream function. You can use that to detect obstacles or build simple maps of a room.
Students can compare the four model sizes on the same images and measure quality, speed and memory use themselves. It's a good way into depth estimation and video consistency.
What your computer needs
The README suggests starting with the S model (24M parameters, 97 MB). It's enough for laptops, quick tests and live use. If edges look too soft, switch to B (386 MB) or L (1.3 GB, best with a desktop GPU). H+ (872M parameters, 3.5 GB) is the default and delivers the best quality, including full-resolution video. For that one, a GPU or an Apple Silicon Mac is recommended.
- Runs on Nvidia GPUs (CUDA), Apple Silicon GPUs (MPS) or purely on the CPU.
- GPUs compute in bfloat16, the CPU in float32. Large models are correspondingly slow on the CPU.
- Download sizes: S 97 MB, B 386 MB, L 1.3 GB, H+ 3.5 GB, plus about 94 MB for the focal length estimator.
- The live depth demo for the Mac requires Apple Silicon, macOS 15 or later and Xcode 16 or later.
- Apple doesn't give any speed figures. For live use, measure the frame rate on your own hardware.
- High-resolution video (2K to 4K) with H+ needs noticeably more VRAM than single photos.
How do I find out what my computer has?
- Windows: Press Ctrl + Shift + Esc → Task Manager → Performance tab → click GPU on the left. At the bottom you'll see Dedicated GPU memory (that's your VRAM). You'll find system memory under Memory.
- Mac: Apple menu at the top left → About This Mac. You'll see your chip (e.g. M2) and memory (e.g. 16 GB). On Apple chips, graphics and processor share this memory.
- Linux: Type
nvidia-smiin the terminal (shows your graphics card and its memory) andfree -hfor system memory.
Guide
This model runs with its developers' own program (apple-aiml-research/ml-depth-pro-2 on GitHub). The commands below come from the Model Card and the developers' instructions.
In the browser (no installation)
The developers have an online demo on Hugging Face. You can try the model right away there without installing anything. Sometimes you have to wait a moment for the demo to wake up.
Windows 10 / 11
Windows 10 / 11: not officially tested. According to the README, only Linux and Macs with Apple chips are officially supported. Windows isn't mentioned, but it should work with an NVIDIA graphics card. WSL2 (Linux running inside Windows) is the safer bet. There, you follow the Linux instructions.
- Install the prerequisites. You need a recent Python 3 (check "Add Python to PATH" during installation) and Git for Windows. For decent speed, you need an NVIDIA graphics card with an up-to-date driver. Then open PowerShell, the Windows command line. These two commands check that everything is in place: both should print a version number.
python --version git --version - Download the code. This fetches Apple's code, which also includes a sample photo and a sample video. Then switch into the new folder.
git clone https://github.com/apple-aiml-research/ml-depth-pro-2 cd ml-depth-pro-2 - Create a separate environment. A virtual environment (venv) is an isolated folder for Python packages, so the rest of your system stays untouched. If activation works, you'll see (.venv) at the start of the line. If Windows says scripts are blocked, run "Set-ExecutionPolicy -Scope CurrentUser RemoteSigned" once first.
python -m venv .venv .venv\Scripts\Activate.ps1 - Install Depth Pro 2. This command comes from the Model Card. It installs Depth Pro 2 along with PyTorch, the compute library behind it. This can take a few minutes.
pip install "git+https://github.com/apple-aiml-research/ml-depth-pro-2" - Check your graphics card. This checks whether PyTorch can find your NVIDIA card. If it prints "True", you're all set. If it prints "False", pip installed a CPU-only version. In that case, reinstall PyTorch using the CUDA command shown on the PyTorch website.
python -c "import torch; print(torch.cuda.is_available())" - Create your first depth map. This estimates depth for the bundled sample photo. On the first run, the tool downloads the H+ model (3.5 GB). When it's done, the out folder contains example-depth.png. Red means near, blue means far.
depth-pro-2 predict -i assets\example.jpg -o out\ - Without a powerful GPU. If you don't have an NVIDIA card or don't have enough VRAM, use the small S model (97 MB). It runs much faster, even on the CPU.
depth-pro-2 predict -i assets\example.jpg -o out\ -m depthpro2-s - Result. The out folder contains the file example-depth.png. It shows depth as a color image: red is near, blue is far.
macOS
macOS: supported. According to the README, Macs with Apple chips (M1 or later) are supported. The tool runs on the GPU there, via Apple's MPS interface. There's no information about Intel Macs. On a MacBook, Apple recommends the small S model.
- Install the prerequisites. You need a Mac with an Apple chip. Open the Terminal app and install Apple's Xcode Command Line Tools, which include Git. Confirm the window that pops up. Run "git --version" to check it worked.
xcode-select --install git --version - Install uv. uv is a tool that sets up Python and all the packages automatically. The Depth Pro 2 developers recommend it. You can install it with pip. Afterwards, "uv --version" should print a version number.
pip3 install uv uv --version - Download the code. This fetches Apple's code, which also includes a sample photo and a sample video. Then switch into the new folder.
git clone https://github.com/apple-aiml-research/ml-depth-pro-2 cd ml-depth-pro-2 - Create your first depth map. On the first run, uv automatically creates an environment and installs everything. The tool then downloads the H+ model (3.5 GB), which takes a while. When it's done, the out folder contains example-depth.png.
uv run depth-pro-2 predict -i assets/example.jpg -o out/ - On a laptop: use the small model. H+ is fairly slow on a MacBook. Adding "-m depthpro2-s" switches to the small S model (97 MB), which is what Apple itself recommends.
uv run depth-pro-2 predict -i assets/example.jpg -o out/ -m depthpro2-s - View the result. Use the command below to open the folder in Finder. In the depth image, red means near and blue means far.
open out - Result. The out folder contains the file example-depth.png. It shows depth as a color image: red is near, blue is far.
Linux
Linux: supported. According to the README, Linux is officially supported. The tool automatically uses an NVIDIA graphics card (CUDA) if one is present, and otherwise falls back to the CPU. Without a GPU, H+ is very slow, so use the S model.
- Install the prerequisites. You need Git and Python 3. Most Linux systems already have both, or you can install them quickly through your package manager. For decent speed, you need an NVIDIA graphics card with the driver installed. "nvidia-smi" shows your card and its VRAM.
git --version python3 --version nvidia-smi - Install uv. uv is a tool that sets up Python and all the packages automatically. The Depth Pro 2 developers recommend it. If your system refuses "pip install", install uv through your package manager or use the installer from the uv website. Afterwards, "uv --version" should print a version number.
pip install uv uv --version - Download the code. This fetches Apple's code, which also includes a sample photo and a sample video. Then switch into the new folder.
git clone https://github.com/apple-aiml-research/ml-depth-pro-2 cd ml-depth-pro-2 - Create your first depth map. On the first run, uv automatically creates an environment and installs PyTorch and everything else. The tool then downloads the H+ model (3.5 GB). When it's done, the out folder contains example-depth.png. Red means near, blue means far.
uv run depth-pro-2 predict -i assets/example.jpg -o out/ - Try a video. The sample video works the same way. The result is the file out/example-depth.mp4. A video needs more VRAM than a photo.
uv run depth-pro-2 predict -i assets/example.mp4 -o out/ - Low on memory: use the small model. If you run out of VRAM or don't have an NVIDIA card, use the small S model (97 MB). For videos, you can try lowering "--window" first instead. That's the number of frames the model processes at once.
uv run depth-pro-2 predict -i assets/example.jpg -o out/ -m depthpro2-s - Result. The out folder contains the file example-depth.png. It shows depth as a color image: red is near, blue is far.
What to do after your first result
- Your own photos: pass your image or a whole folder with -i, e.g. "-i myphoto.jpg". The tool reads the focal length (the camera's zoom level) from the photo metadata or estimates it itself.
- With "--save-npz", the tool also saves the real depth in meters. With "--save-png16", you get 16-bit grayscale images for Blender or Nuke.
- For videos, there are "--window" and "--overlap", which set how many frames the model processes together. For live camera feeds, use "--mode unbuffered".
- On the Mac, the demo/ folder contains a small app for live depth from the camera (requires macOS 15 and Xcode 16). Note: you may only use the model weights for non-commercial research.
Try it
uv run depth-pro-2 predict -i assets/example.jpg -o out/ (bundled sample image as a smoke test)uv run depth-pro-2 predict -i livingroom.jpg -o out/ (your own photo of a room, to check whether the distances in meters look plausible)uv run depth-pro-2 predict -i portrait.jpg -o out/ (photo of a person, to inspect edge sharpness around hair and shoulders)uv run depth-pro-2 predict -i walk.mp4 -o out/ (phone video without zoom, to check whether the depth flickers)In Python: load the model with create_model_and_transforms and run infer_video on a clip passed as frames in the format (T, 3, H, W)
Tips for better results
- Start with S and only move to B, L or H+ if the edges look too soft on your own footage.
- Judge quality on your own images and videos, not just Apple's gallery. Pay special attention to edges and flicker.
- For measurements and 3D work, have the tool write the NPZ output. It contains depth in meters and focal length in pixels.
- If you know your camera's real focal length, the metric values are more reliable than with an estimated one. For photos without metadata, the focal length estimator steps in.
- For live applications on the Mac, use the Core ML exports from Hugging Face.
If something goes wrong
The quickstart needs the uv tool (docs.astral.sh/uv). Install it first, or the "uv run" command won't work.
The license only allows non-commercial research, so the weights can't go into apps, paid APIs or company tools.
Each video clip gets only one focal length. With zoom shots, the metric scale drifts, so use videos with a fixed focal length.
The training and evaluation code is missing. You can't fine-tune on your own data or reproduce Apple's results.
The Mac live demo only runs on macOS 15 and Xcode 16 or later.
The large models are very slow on the CPU. Without a GPU, stick with the S model.
“out of memory,” a crash, or your computer freezes
The model is too big for your VRAM (video memory). Pick a smaller variant from the table above and close other programs, especially games and browsers with lots of tabs.
It runs, but extremely slowly
Your CPU is probably doing the work instead of your graphics card. In Ollama, the PROCESSOR column of ollama ps shows how much runs on the GPU. If it says "100% CPU", update your graphics driver or pick a smaller variant that fits entirely into VRAM (video memory).
“ollama” or “python” is not found
Close the terminal window and open it again so it recognizes the newly installed program. For Python on Windows: reinstall it and check Add python.exe to PATH.
Windows: "Running scripts is disabled on this system"
Run the script with exactly the command above: powershell -ExecutionPolicy Bypass -File try_windows.ps1. This allows execution for this one time only.
Error message when loading the model (version, "trust_remote_code", unknown architecture)
Brand-new models often need the latest software version. With Ollama: update Ollama. With Python: pip install -U transformers in the activated environment. The Model Card usually lists the exact version.
Tokenstrom generates the scripts and notebook automatically. Please take a quick look before running them – they're plain, readable text. Mind the model's license: custom license (see Model Card).
Ask the AI assistant
Questions about this article, installation or background? Our assistant knows this page and runs on our own machine, not in someone else's cloud.
The Assistant is an AI (Gemma 4) and can make mistakes – please check important details against the source. Questions are used only to generate an answer and are deleted after one hour. Please don't enter any personal data.
Comments
What do you think? Already tried it? Write it here, no sign-up needed.
Comments are public. Links aren't allowed, and we delete abusive posts. You can delete your own comment at any time in this browser. Learn more in the Privacy Policy.