Empiric Earth closes that gap with 4D reconstruction. A single forward-facing camera clip becomes an environment where every dynamic in the original scene can be replayed.
Why it matters
AV developers typically build simulation data from multi-sensor rigs: LiDAR, radar, panoramic camera arrays. Empiric Earth captures a single monocular camera, plus GPS and IMU. The extra sensors and the surround are not there. That difference is the reason 4D reconstruction exists.
The prerequisite for any of this to matter is that the source video contains something worth replaying. Recorded driving, from real people on real roads, holds edge cases that a curated public dataset does not carry. Reconstructing those events lets AV developers fine-tune models on the exact future trajectories of the third parties involved, which is not what a public dataset with expert drivers can teach.
What 4D reconstruction is
The definition, plainly: take a video captured by one of the cameras in the network and turn it into a representation where the dynamics can be replayed. The output covers more than the appearance of the scene. It covers the motion of the ego vehicle and the motion of every relevant third party.
Every reconstructed video ships with the objects the scene contains, in 3D world coordinates, plus orientation around each axis, and each object's dimensions and class. Ego vehicle and third parties, together.
Alongside the trajectories, the pipeline outputs a sparse point-cloud map, the camera's intrinsics, an undistorted (de-fisheyed) video, and an estimate of camera height that tells you whether the ego is a truck or a car.
How it works
The pipeline is geometry-based rather than end-to-end learned. Feature detection and matching, image segmentation, structure from motion, bundle adjustment, and 3D detection and tracking, in that order. The static scene falls out as the map; the moving objects fall out as trajectories.
Every reconstruction is scored automatically before a human ever looks at it. The score is a composite of 20 different metrics, covering the ego trajectory's physical plausibility against the GPS trace, relative and global accuracy, and the smoothness of the reconstructed path.
The automated pipeline itself runs in 15 minutes for a single video, with all outputs generated in parallel.
Where Empiric Earth fits
Every rebuilt scene rests on the same measurement: a real event, recorded from a real road, reconstructed into geometry that a downstream tool can consume. The trajectories themselves are the value for a team building its own simulation stack, and the pre-computed camera intrinsics and defisheyed video are the value for a team that needs the clip clean before doing anything else with it.
Only we know
The quality of a reconstruction is not a subjective judgment. It is measured against 20 metrics before human review, including comparison of the ego trajectory against its own GPS trace. That scoring exists because the pipeline is not always right and the failures need to be caught before they reach a customer. Publishing what the metrics cover is a different act from publishing a score claim, and only a company that runs its own reconstructions can name the metrics.
The pipeline retrains on human-corrected data. After a single round, the 3D object detector's recall doubled. That flywheel is what turns geometry work into a measured process, not a one-shot pipeline.
Limits
The pipeline is geometry-based rather than end-to-end learned, which is a deliberate accuracy trade-off: geometry gives verifiable outputs, at the cost of engineering effort per stage. Reconstructions still pass through a human review layer, which is why the recall-doubled improvement was possible after a single retraining pass.
The 15-minute run time is machine time. IMU is not yet consumed by the pipeline; today the inputs are the video and the GPS trace.
FAQ
Q: What is 4D reconstruction? A: Turning a video into a representation where the dynamics of the ego vehicle and every relevant third party can be replayed in 3D.
Q: What does Empiric Earth Atlas need to reconstruct a scene? A: A monocular video and a GPS trace. IMU is designed for but is not currently an input.
Q: How long does one reconstruction take? A: The automated pipeline runs in 15 minutes. Human review is a separate step.
Q: What does the reconstruction contain? A: Ego vehicle and third-party trajectories in 3D, with each object's orientation, dimensions and class, plus a sparse point-cloud map and the camera's intrinsics.
Q: Why is the pipeline geometry-based rather than end-to-end learned? A: The trade-off is deliberate: geometry produces verifiable outputs.