Touch as tokens: Why it's so hard to teach robots to feel

What this is about

Twisting off a bottle cap is easy for a human. For a robot that only sees through cameras, it's hard. Whether the cap is loosening or the fingers are just slipping over it looks almost identical on camera. Ant Group uses exactly this example to argue that robots need a sense of touch. The gap shows up everywhere: Apple's research team writes that its humanoid HALO can "barely directly sense" whether it is touching something. Chinese tactile sensor maker Qianjue, in an interview with 163.com, calls the moment of contact the "last centimeter" hurdle.

Hands with tactile sensors in their fingertips now exist. The real problem is the software. If you simply feed the raw signals as an extra input into an existing vision-language-action model (VLA), a model that sees images, understands spoken or written commands and outputs movements, you often gain little or even lose performance. On November 10, 2026, Robbyant, Ant Group's robotics unit, released LingBot-Touch, an open tactile model designed to solve this problem. We explain why touch is so hard for AI, how the approach works and how solid the numbers are.

Three reasons an extra sense doesn't automatically help

1. There isn't enough data. Videos of humans exist in huge quantities. Nvidia used more than 400,000 hours of them for GR00T 2, and Nature Will says it used more than a million hours for Natura-1. Tactile data, by contrast, only gets created when someone wears a sensor glove or teleoperates a sensor-equipped hand. Even the largest open dataset, the one LingBot-Touch was trained on, covers just under 5,000 hours.

2. Every sensor measures something different. According to an overview by 36Kr, there are five main designs: - Piezoresistive sensors change their resistance under pressure. They're cheap, but only measure normal (perpendicular) force and are sensitive to temperature and humidity. - Magnetic sensors register how a magnetic field shifts when the surface deforms. They're prone to interference and have limited resolution. - Camera-based (visuotactile) sensors film a gel layer from the inside as it gets pressed in on contact. They're very precise, but expensive. - On top of that, there are piezoelectric and capacitive sensors.

Researchers at the National University of Singapore sum up the result: a model trained on one sensor can't be used directly with another. In other words, data from different manufacturers doesn't fit together.

3. A pretrained model is bad at picking up a new sense. A pretrained VLA has learned to rely on images. The authors of ResTacVLA call this "modality collapse": the broad, continuous image stream drowns out the rare, sudden touch events, and the model learns to ignore them. Measurements back this up. In the ReTouch paper from August 2026, π0's success rate dropped from 45.6 to 43.8 percent with added tactile data, while π0.5 only rose from 51.5 to 53.1 percent. The authors speak of "inconsistent effects." An even more dramatic figure was cited in July by the Chinese portal 机器人前瞻: with one classic model, simply adding tactile signals reportedly dropped the success rate from 17 to 6 percent. The report attributes this to a study involving Fei-Fei Li and Jim Fan but gives no title. We couldn't find the original paper. So the number is unverified, but it fits the pattern.

The new approach: a foundation model just for touch

Instead of feeding in raw signals directly, you put a dedicated tactile foundation model in front: a base model that first learns from lots of tactile data what touch means. It translates sensor readings into tokens, compact bundles of numbers an action model can work with. An early pioneer was Meta's Sparsh from 2024. It was trained self-supervised on more than 460,000 images, meaning it learned without human labels, and only worked with camera-based sensors like DIGIT and GelSight Mini. The weights are on Hugging Face; the GitHub repo has been archived since April 2026.

LingBot-Touch goes three steps further:

• Shared finger coordinates. Each measuring point (taxel) is mapped into a unified fingertip coordinate system using fixed per-finger calibration values. After that, it no longer matters how a manufacturer arranged its points. The signals are normalized too: shear forces, i.e. sideways forces, range from −1 to 1, and pressure perpendicular to the surface from 0 to 1.
• Sensor-specific tokenizers. Each sensor type gets its own input stage that preprocesses the raw signals. After that, a shared "Haptic Transformer" processes all the data. Every output token carries a timestamp, finger and position. The "5D RoPE" mentioned in the report, a positional encoding across five dimensions, maps exactly this information as we read it: time, finger and three spatial coordinates.
• JEPA pretraining. JEPA (Joint-Embedding Predictive Architecture) means the model predicts masked parts of the data, not as raw measurements but as an abstract representation. The idea is that it learns the rules of contact rather than sensor noise. The target is a slowly updated copy of the encoder (EMA encoder). That frozen copy is what's being released.

The training data, according to the project page:

That's roughly 1.33 billion individual frames from tactile gloves and robot hands. The smallest variant, Tiny, has 5.46 million parameters according to the project page; Chinese media put it at around 5.3 million. Attached to LingBot-VLA, π0.5 or T-Rex, it grows the overall system by only 0.13 to 0.16 percent. Working backwards, that means these action models have roughly 3.4 to 4.2 billion parameters. There's also a larger Small variant.

What the numbers show, and what they don't

Robbyant tested 26 tasks on real robots, 20 of them on the Sharpa Wave and 6 on the XHand, with 20 trials each. They include turning book pages, twisting off caps and sorting mahjong tiles. The headline claim: the success rate of four action models rises by 14 to 26 percentage points on average. But the baseline here is a model with no sense of touch at all.

The more telling comparison is the company's own baseline with raw sensor data. For that, the reports give only one example, T-Rex twisting off a cap on the Sharpa Wave:

In this case, raw data adds 6 points and the pretrained model adds 34. That supports the thesis. For the remaining tasks, the project page shows the raw-data comparison only as a chart without numbers. In simulation, the project page says five input variants were compared, but only three are named: no touch, "Tactile CNN" (a classic small image network for tactile data) and LingBot-Touch. We couldn't find a baseline using plain binary contact ("touching yes/no") in the publicly readable materials. We weren't able to fully analyze the 18 MB tech report. On statistics: with 20 trials per task, every single trial shifts the result by 5 percentage points.

What convinces us most is the hardware transfer. The XHand's magnetic sensor model never appeared in pretraining; the knowledge about magnetic tactile data came only from glove recordings. The encoder stayed frozen, and only the action model was trained.

The limitations are spelled out in the project's own documentation: - Camera-based sensors provide only one value per point for indentation or brightness, with no raw images and no shear force. So the common denominator is smaller than what the hardware can deliver. - According to the data guide, the piezoresistive path in the released package has "only been tested synthetically." - Only the encoder and inference code are released. Training code, predictor and dataset are missing. - At 16 frames, Tiny produces 8,000 tokens for one Sharpa hand. The developers therefore recommend reading out the tokens via attention, i.e. picking out the relevant ones, rather than averaging them.

How the other approaches compare

TouchWorld uses touch for prediction. A world model based on Wan2.2 predicts how contact will evolve. According to the paper, it reached a 65.0 percent success rate versus 40.7 percent for π0.5 and 39.3 percent for GR00T N1.7. DexTact shows where touch helps most. Without touch, the grasp rate under unfamiliar lighting drops by 16.6 points. With touch, the paper's abstract says 4.2 points, but the detailed table says 1.6 points. And everything ran in simulation only. HTT from Singapore pursues the same idea as LingBot-Touch: on a Sharpa hand with an unseen sensor, the screw toy task succeeded 90 percent of the time, versus 45 percent with raw force values.

The opposing view comes from Natura-1. According to Nature Will, the 50-billion-parameter MoE model takes in only language and images and still controls hands with 20 degrees of freedom. A mixture-of-experts model uses only part of its network at each step. It scored 53.4 and 59.1 points respectively, while π0.5 scored 13.5. However, π0.5 ran on a different platform with a two-finger gripper, so the comparison is shaky. Nature Will itself lists better sensor systems as an open problem. Nvidia's GR00T 2 also works with camera images, joint positions and text, without tactile input, according to the reports available to us.

What the hardware costs

The two test hands sit in very different price brackets: - Sharpa Wave: 22 degrees of freedom, with a mini camera in each fingertip offering more than 1,000 tactile pixels. One retailer lists it at $50,000; a stripped-down SE version costs $20,000 according to Humanoid.Guide. - XHand1 from Robotera: 12 degrees of freedom, fingertips with more than 100 measuring points wrapping around 270 degrees. RobotShop sells it for about $16,750.

So this isn't hobbyist territory yet. LingBot-Touch itself runs on a CPU, but it's only useful if you have one of these hands.

Our take

Chinese media and the WAIC industry show are calling 2026 the "year zero of touch." LingBot-Touch is the first major open model to unify three sensor types in a shared representation. The idea is compelling: the approach tackles the problem of many sensor formats by mapping them all onto shared finger coordinates instead of retraining for every hand.

Our assessment: the company's 14-to-26-point claim is measured against the weakest baseline. The important comparison with raw tactile data is backed by hard numbers in only one example. There are no independent replications yet, and without training code and dataset, nobody can reproduce the pretraining. It also remains open whether touch will need its own model as data grows, or whether video models like Natura-1 will close the gap from the other side. So far, this year's research suggests that a separately pretrained sense of touch delivers more than simply feeding in the raw data.