Every few years, autonomous driving feels close. Almost every year Tesla comes up with a much improved version of FSD, Waymo expands it’s service to another (geofenced) city and another autonomous startup raises a few hundred million dollars on the premise of achieving perfect autonomy..

The demos improve bit by bit but eventually we get to hear that full autonomy is still five years out they say. As a result, confidence drops and the cycle starts all over again.

At this point, the interesting question is no longer whether autonomy is possible. It’s why it keeps taking longer than almost everyone expects, including the companies building EVs around it.

The usual explanations miss the mark. It’s not regulation. It’s not public trust. And with the amount of data that car sensors tend to collect these days its certainly not due to a lack of data.

The real reason autonomy is slow is simpler and more uncomfortable. It’s a real-world AI problem. And real-world AI behaves very differently from the AI most people have gotten used to over the last few years.

The bottleneck isn’t intelligence. It’s inference.

Tesla, Rivian, and Waymo technologies all agree on something fundamental, even if they don’t say it explicitly. The hardest part of autonomy is not training models. It’s running them reliably inside a moving vehicle, every second, in unpredictable conditions.

Training happens offline. It’s flexible and forgiving. If something breaks, you rerun the job.

Inference happens on the edge, inside the vehicle. It’s real time, power constrained, and unforgiving. There are no retries. No time to hesitate. Errors are costly and so is latency.

This matters especially for EVs. Compute, sensors, and cooling all draw energy. Power used by autonomy hardware is power not moving the wheels.

A language model can hedge or hallucinate and still feel useful. A driving system inside a two or three ton EV has to commit and has no room for error.

Tesla’s approach: simplify the world so inference fits inside an EV

Tesla’s autonomy strategy has always been shaped by inference constraints.

Tesla builds its own hardware because autonomy depends on fast, predictable decisions inside the car, not brute-force compute. HW3, Tesla’s third-generation onboard computer allowed Tesla to run real-time neural networks instead of falling back on hand-written code. HW4 was an improvement but the point is that the inference computer has to run inside the vehicle, reliably and efficiently, without hurting range.

That helps explain Tesla’s move to vision-only autonomy. Early stacks had to reconcile disagreements between radar and vision using hand-written logic. That proved brittle and hard to scale. When sensors disagreed, the system itself became the problem. Removing multiple inputs simplified inference. Fewer conflicting inputs meant more predictable behavior and lower compute overhead, even if it sacrificed redundancy in some conditions.

Tesla FSD is impressive but it still needs work

Tesla’s latest Full Self-Driving releases are genuinely impressive. They handle dense urban driving, navigate construction zones, and manage situations that would have felt unrealistic a few years ago.

They also still require interventions and cause accidents that could have been avoided by a human driver.

Drivers also report hesitation at unprotected left turns, awkward merges, and moments where the system does the right thing late rather than early. These are not catastrophic failures. They are timing and judgment issues.

In other words, there is a reason why Tesla still sells it as FSD “Supervised”.

Rivian’s approach is different but it has a long way to go

Rivian has just started its path to autonomy, but it’s clearly building with Tesla’s lessons in mind.

Like Tesla, Rivian has concluded that autonomy lives or dies at the inference layer. That belief shows up in its decision to design RAP1, its own autonomy processor.

Rivian is betting on end-to-end learning and what it calls a Large Driving Model. Instead of hand-coding logic to glue sensors together, the model ingests raw data from cameras, radar, ultrasonics, and eventually LiDAR, and outputs driving commands directly.

The tradeoff here is that “Large Driving Models” require more energy and complexity. The richer sensors increase inference load. Rivian’s bet is that their customized chip and better models can manage that cost without compromising range. Whether they can achieve that remains to be seen.

Waymo’s approach: geofencing

Waymo solves the same inference problem by shrinking it.

Its vehicles run powerful onboard compute and a rich sensor suite, including LiDAR, but operate within tightly geofenced areas supported by high-definition maps. That dramatically reduces uncertainty.

This works well for commercial mobility, where vehicle cost and energy use are less constrained. The tradeoff is that its harder to scale.

So how long will autonomy actually take?

It’s probably going to take longer than the marketing decks suggest. Longer than most EV buyers expect. And longer than people used to the pace of generative AI progress might assume.

That doesn’t mean autonomy isn’t working. It means it’s a real-world problem that will take longer to solve.

Electric vehicles are slowly shifting from being just vehicles with fast torque to being mobile learning systems deployed at scale. Sure, the battery, range and the driving experience all still matter. But more and more, those from just the outer layer. The real work is underneath, in getting AI to behave reliably and efficiently inside a physical machine, in real time, with real consequences.

Getting to full autonomy isn’t just a function of data and software. It’s also a function of building better, faster, more efficient hardware that can resolve multiple sensor inputs in fractions of a second.

The company that gets that hardware right may be the first to get autonomy right too.