Summary: Meta described RAMMP on July 27, 2026 as an ARPA-H-supported assistive robotics project with up to $41.5 million in funding, led by the University of Pittsburgh’s HERL. The system uses DINO, SAM, and RF-DETR for on-device perception in battery-powered mobility aids, optimized for speed and stability. The article says no exact latency or power budget was provided.
One constraint governs everything else
Most AI deployments can dodge a problem that assistive mobility platforms cannot: the model has to sit on the device itself, running in real time off a battery. A child who steps into the sidewalk’s path, or a curb that shows up with no warning, will not wait while a request travels out to a cloud endpoint and back.
Meta’s July 27, 2026 post on the RAMMP project describes an ARPA-H-supported effort with up to $41.5 million in funding, led by the Human Engineering Research Laboratories (HERL) at the University of Pittsburgh. RAMMP stands for the Robotic Assistive Mobility and Manipulation Platform; it brings robotics, AI, and user-centered design into one system, with ATDev on board as engineering partner. Two of Meta’s open source vision models, DINO and Segment Anything Model (SAM), sit inside its perception stack.
Two numbers explain the urgency: roughly 5.5 million wheelchair users live in the United States, and each year US emergency departments treat over 100,000 injuries tied to wheelchairs, frequently from trips and falls.
The stack that runs on the machine
A compact visual brain is the role DINOv3 fills — a general-purpose foundation that lightweight, task-specific modules can stack onto for detection or movement tracking. Segmentation falls to SAM, which needs only a handful of prompts to single out and trace objects. Underneath sits RF-DETR, the lightweight detection model that RAMMP’s perception system is built on, fine-tuned with DINOv2 embeddings. SAM auto-labels the training data, and that is what lets the team produce annotations spanning the lighting, backgrounds, heights, and angles assistive devices really meet.
The tradeoffs here are engineering, and they bite. Cutting the memory footprint, stepping down to lower precision where that makes sense, and shipping in a deployment format that survives real conditions are the standard moves on edge hardware. Add practical resolutions and efficient batching, and both models stay quick on a battery. As the post says outright, doing so sometimes costs a little boundary precision or feature detail, paid in exchange for the speed and stability a user needs.
Sivashankar Sivakanthan, Chief of Staff to the RAMMP project, names the standard plainly: a benchmark accuracy figure by itself proves nothing — what matters is whether the system survives the unpredictability of everyday life.
The interface is the user’s own words
Treating the user’s immediate surroundings as context is among the sharper design choices here. Instead of elaborate interfaces, users pose questions to the robot’s image sensors in ordinary language, paired with image data. For a task as ordinary as picking up a cup, that trims both cognitive load and context switching.
Already the opening prototype leans on DINO-based tools to pick out automatic door buttons, cups, and curbs as navigation aids. Voice and touch input is what engineers are adding now, so a user can choose and act on particular objects. That raises a fresh robustness demand: beyond accuracy and temporal coherence, behavior has to stay predictable no matter how varied the prompts and inputs get.
Anyone building agentic tools that fuse language with perception faces the same class of problem described in what a hosted sandboxed shell changes for agentic tools: the model’s output is half the story, and whether the thing is usable comes down to the interface and validation layer.
What comes next
The consortium behind RAMMP takes in Kinova Robotics, LUCI Mobility, ATDev, plus academic partners at Carnegie Mellon, Cornell, Northeastern, and Purdue. HERL carries the lead, contributing biomedical engineering and user-centered research, while ATDev makes sure the research holds up inside real devices. Wheelchair users, clinicians, and advocacy groups all take part in the design work as it unfolds.
Three priorities shape the work ahead: making temporal consistency firmer, extending robustness to diverse real-world conditions, and wiring the system more closely into decision-making and control. Next-generation perception models — SAM 3.1 and DINOv3 among them — are also slated for further integration.
For builders, the model names are not the lesson — the pattern is. Take a general-purpose vision foundation, train a lightweight detector on top of it with fine-tuning, let a segmentation model do the auto-labeling, then optimize hard for the hardware actually in hand. Because the post gives no exact latency figures or power budgets, the precision-versus-stability tradeoff reads better as a design principle than a measured threshold. The harder question is where the line sits: how much accuracy can you surrender before the person who depends on the system no longer has grounds to trust it?