XPILOT VP on Control vs Planning: What End-to-End Will and Won't Replace

A few days ago XPeng XPILOT VP 吴新宙 talked about control modules and end-to-end. Sharing it here:

晚点 Auto: Perception has been replaced by models. Will planning-and-control later get a new model that replaces the hand-written rules?

吴新宙: Control is an optimization problem. I believe it stays rule-based for a long time — solve with math what math can solve; it's a very well-defined mathematical problem.

But for planning, the future is definitely large models, large compute, large data. To raise autonomous driving's ceiling high enough, under the whole "Software 2.0" concept, more and more will be solved by models — ChatGPT already proved the path works. We've actually been at it for three months.

晚点 Auto: You once said you wouldn't jump into end-to-end lightly. What's your thinking now?

吴新宙: For end-to-end, control we definitely won't model. Planning will start using a model — a big model, though not as big as ChatGPT, since the complexity is much lower. Same ideas, same core, for future development.

But you don't throw away the old stuff — that would be very foolish. Our static and dynamic perception models are still converging; we can afford to let them converge because the existing system is already strong enough. It's an input as part of sensor fusion — an increment for us.

Likewise, for planning, once you have the large model you also need time for it to learn, to handle the corner cases well.

Training a large model is a lot like teaching a child — you have to tell it "you didn't do this well." Who's the teacher? Our existing algorithm stack. The two will coexist for a long time.

Because we now have a strong teacher, the student might reach PhD or post-doc level and surpass the teacher someday — but for quite a while, they complement each other.

---

Source: 不是郑小康 (Weibo) — translated & adapted for XPENGTech.