Geometric Relative Positional Encoding has become the primary mechanism for injecting spatial inductive biases into multi-view transformers. Existing designs condition attention on camera-level frustums or ray-marched depth, associating cross-view tokens through the sensor geometry. Such sensor-centric formulations, however, describe only where a token is observed from, leaving the rich surface geometry of the scene region depicted by the token. In this paper, we advocate a shift toward surface-aware positional encoding that explicitly grounds patch tokens in the geometry of physical surfaces. We introduce HRoPE, which represents the local geometry of each patch as a slanted 3D plane, following the slanted support-window assumption of classical multi-view stereo. We inject token-level, plane-induced homographies into attention as relative transformations. This equips the transformer with a geometric prior for rectifying perspective distortions and aligning patch features across viewpoints. Our encoding requires no ground-truth geometry: the plane parameters can be predicted end-to-end under a view-synthesis objective alone, or derived from foundation models. Extensive experiments on RealEstate10K, CO3Dv2, and DL3DV show that our best variant of HRoPE consistently outperforms the evaluated geometric positional encodings for novel view synthesis. Extensive ablations and analysis show the effectiveness of the proposed encoding. Please refer to our project page at: https://gaohchen.github.io/HRoPE
Sam: A surface’s tilt can help a model connect what different cameras see. This preprint finds that giving image patches estimated surface orientation improves images generated from new viewpoints.
Alex: So the claim is better images from a different geometric description, rather than simply a bigger model?
Sam: Yes, within the models and benchmarks tested. It’s a method paper by Huachen Gao, Yang Yang, and colleagues, called Homography Relative Positional Encoding, or H-rope. Their strongest version beats the geometric encodings they evaluate across several image-generation datasets.
Alex: That sounds relevant to researchers combining photographs or video frames. What should they listen for before deciding whether to use it?
Sam: Whether their model needs to connect the same surface across viewpoints. The work suggests that where something is may not be enough; its orientation also helps. We’ll unpack that distinction, examine the comparisons, and separate useful geometry from claims of physical accuracy.
Alex: Start with the existing approach. What information do these models already have, and what is missing?
Sam: They split images into patches, represented as tokens. Attention is the mechanism that weights information exchanged between those tokens. On its own, it doesn’t know how the cameras or patches relate geometrically.
Alex: So the model needs some spatial information before comparing patches from different views.
Sam: That’s the role of positional encoding: information telling attention how observations relate geometrically. One existing method, P-rope, uses camera projection transforms. But every patch within a view receives the same transform, so it cannot express local surface variation.
Alex: The paper also compares against methods that predict depth. Doesn’t depth already tell you where a patch belongs?
Sam: It locates a patch center along its viewing ray. But matching patches means comparing neighborhoods, not isolated points. Surface orientation helps determine how that neighborhood changes appearance between views.
So locating the center and accounting for the surrounding patch’s deformation are different jobs. What does H-rope add for that second job?
Sam: It assigns each observed patch a local tilted plane. A homography is a projective mapping between images of a plane. The plane’s orientation and distance, combined with camera geometry, determine that mapping.
Alex: Is the new contribution the use of planes, or where those planes enter the model?
Sam: Where they enter. Classical stereo already used tilted comparison windows to handle slanted surfaces. H-rope brings that geometric idea inside attention, rather than using it only to warp images or evaluate stereo matches.
Alex: What does “inside attention” mean operationally? Are the authors moving pixels around before the model sees them?
Sam: No. They transform the feature vectors that attention compares. Each patch gets its own transform, and the comparison uses one patch’s transform composed with the inverse of the other’s. The authors interpret this as an attention-level analogue of comparing neighborhoods after aligning them to a shared plane.
Alex: That sounds physically meaningful when both patches show the same flat surface. Does it still mean that for arbitrary patch pairs?
Sam: Not necessarily, and that distinction matters. For patches on the same physical plane, with consistent camera geometry, the composition gives the actual planar mapping. For patches on different planes, it is not guaranteed to describe a physical image warp.
Alex: Then should we regard the estimated planes as recovered geometry, or as useful internal parameters?
Sam: The fully predicted version uses latent transformation parameters, optimized for image synthesis. It even allows rotation and translation to vary by patch, although camera pose is shared within a view. The structure is geometrically motivated; every prediction need not be a literal scene measurement.
Alex: How do they train that without making surface annotations a new requirement?
Sam: A lightweight prediction head estimates the parameters from patch features, using the image-synthesis objective without ground-truth geometry supervision. The feature source can be trained from scratch or pretrained. Their strongest main version uses VGGT, a backbone pretrained on geometric prediction tasks.
Alex: That creates an obvious alternative explanation: better pretrained features, rather than better attention geometry. How do they separate those?
Sam: They hold the synthesis architecture fixed and vary the encoding. They also compare different transformations using the same feature source. The homography structure beats an unrestricted matrix, despite fewer degrees of freedom, and beats adding features from VGGT rather than using them to transform attention.
Alex: Put the main result in scale for us. What model and comparison should we remember?
Sam: The main benchmark comparisons use an LVSM image-synthesis backbone with about twenty-five million parameters. On RealEstate10K, the best H-rope version improves peak signal-to-noise ratio by about 1.27 decibels over P-rope. That metric measures image reconstruction quality; higher is better.
Alex: A reconstruction score alone could miss changes in visual appearance. Did the other measures point the same way?
Sam: Yes. Structural similarity and a learned perceptual-distance measure also improve. The strongest version beats the evaluated encodings on RealEstate10K, CO3Dv2, and Objaverse. On DL3DV, it also beats P-rope across the tested input-view counts, with the largest reconstruction gains under sparse observations.
Alex: Which comparison most directly supports the claim that local surface geometry matters, rather than just extra flexibility?
Sam: The calibrated-camera comparison. They retain P-rope’s ground-truth camera projections and add predicted local surface frames. That improves every reported image-quality metric on RealEstate10K. Because the camera projection stays unchanged, this is a concrete test of the added local geometry.
Alex: And if the geometry becomes more accurate, does the model actually benefit?
Sam: In a synthetic dataset with ground-truth geometry, replacing predicted parameters with ground-truth camera and plane parameters improves performance further. That supports the geometry interpretation under that test. It does not establish that all learned parameters are physically accurate.
Alex: What happens when the camera settings differ from training? That is where I would want a geometric prior to earn its keep.
Sam: They test stronger target-camera zoom, targets beyond the reference-view interval, and wider spacing between input frames. H-rope retains its advantage over the comparison encodings under those tested shifts. I’d read that as evidence of useful generalization, not robustness to every possible camera change.
Alex: Does the benefit survive outside their main image-synthesis architecture?
Sam: They also replace the camera encoding in EscherNet, a multi-view diffusion model, and improve every reported image metric across the tested view counts. In a video language model, adding H-rope improves the spatial-reasoning average by 1.8 points over the fine-tuned control. But performance on appearance order declines, so the transfer is not uniformly beneficial.
Alex: There is also a cost question. Does getting the best result require expensive features?
Sam: The learned-from-scratch version improves over the original backbone at the same reported training cost of thirty-three GPU-hours. The strongest VGGT version takes seventy-six GPU-hours under that schedule. So pretrained geometry helps, but its additional training cost belongs in the comparison.
Alex: What is your overall confidence, and who should read the full document?
Sam: I find the evidence persuasive for the evaluated settings: several datasets, fixed-backbone comparisons, and targeted ablations. But this is a preprint, and geometrically structured attention is not proof of recovered physical geometry. Researchers designing multi-view attention should start with the method’s per-patch transform subsection, then the transformation and calibrated-camera comparisons.
Alex: And what should everyone else carry away?
Sam: To compare what cameras see, model the surface’s orientation—not just its location.
Alex: Thanks for listening.