Kailai Sun, Zijie Gao, Mingyi He, Can Rong, Alok Prakash, Baoshen Guo, Shenhao Wang, Jinhua Zhao
8 min
Abstract
Cities account for over two-thirds of the world's energy and more than 70% of global carbon dioxide emissions. City energy is essential for achieving the 2050 net-zero target and the UN's Sustainable Development Goals 7 and 11. Although existing studies have made progress, they are limited by data and methodological gaps. Existing studies are largely predictive for existing cities rather than generative for future cities; fine-grained, aligned monthly dynamic energy data are scarce; and existing GenAI models struggle to generate dynamic urban functional monthly energy. We present DySENSE, a joint multimodal GenAI framework that synthesizes static urban satellite imagery together with spatially aligned, dynamic monthly building energy consumption maps. We decouple long-term static urban morphology from short-term dynamic energy consumption. By conditioning on static urban land use, DySENSE can generate realistic, diverse and new static urban imagery with aligned energy maps. By conditioning on dynamic monthly and climate prompts, DySENSE can jointly generate dynamic, monthly energy consumption maps while keeping the urban morphology static. Experiments across four cities (New York, Washington, Norfolk, Busan) demonstrated that DySENSE can generate annotated high-fidelity synthetic energy data using less than 25% labelled data, boosting downstream prediction performance by 8% mIoU, and achieving strong energy dynamic consistency. Compared to existing methods, DySENSE significantly improved prediction performance (8.7% accuracy) and reduced prediction error (9%-18% CVRMSE, ASHRAE). DySENSE offers an effective foundation for energy-efficient cities, peak-load analysis, building performance simulation and dynamic demand response for city, energy, and building science. The data and code: https://huggingface.co/datasets/skl24/MUSE2 and https://github.com/kailaisun/DySENSE
Alex: Energy maps are not photographs, though. Does treating both as images lose what makes energy data different?
Sam: The authors account for that. Each input becomes a latent representation—a compressed internal description—but energy gets its own specialised encoder and decoder. They argue that sparse, unevenly distributed energy values need different treatment from colour imagery.
Alex: Then the monthly trick is to keep the city description fixed while letting the energy description change?
Sam: More specifically, they hold the compressed satellite image fixed and generate energy maps under changing monthly prompts. This does not require retraining the model. Their examples show a fixed urban tile with generated energy patterns that follow recorded seasonal changes.
Alex: Before judging those patterns, I want to know what the model learned from. How broad is the dataset?
Sam: It covers four cities: New York City, Washington, D.C., Norfolk, and Busan. After filtering and balancing, the dataset contains about twenty-four thousand monthly paired samples. These combine imagery, energy labels, land use, and climate information.
Alex: The paper calls these building energy maps. Are listeners picturing individual meters when they should be picturing something more aggregated?
Sam: The records originate from building or address-level sources, but consumption is combined into hundred-metre grid cells. Electricity and natural gas are converted into a common energy unit. The resulting labels represent total operating energy, not a detailed breakdown of building systems.
Alex: And what exactly does the model predict in each cell? That affects how we interpret its accuracy.
Sam: The energy values are transformed to reduce their extreme skew, then grouped into energy-level categories. There is also a background category for cells without positive energy values. So a central test is how well predicted categories overlap the recorded map.
Alex: Plausible maps could still be useless training data. What is the strongest evidence that the generated pairs carry useful information?
Sam: The authors train existing energy prediction models with and without synthetic examples. Both the generator and predictor use the same available real training data. Testing uses held-out real tiles, with locations separated between training and testing to reduce leakage.
Alex: If the synthetic group simply gets more training examples, repetition might explain the gain. Did they check that?
Sam: They compare against equal-sized duplication of real examples and simple augmentation with Gaussian noise. Those controls do not reproduce the gains from generated pairs. They also compare static synthetic pairs against examples that include changing monthly energy maps.
Alex: Give us one comparison we can carry away, rather than a catalogue of scores.
Sam: For the predictor based on Yap and colleagues’ method, this comparison uses a quarter of the real training data. Adding dynamically generated pairs improves mean intersection over union—the average overlap between predicted and recorded energy categories.
Alex: How much does that overlap improve? And is it the sort of change that survives outside a visual impression?
Sam: It rises from about thirty-seven percent to about forty-six percent. That is an increase in percentage points, not merely a relative percentage gain. The test is on real data, rather than the synthetic examples used for training.
Alex: Map overlap is useful, but planners also care about demand totals. Do the authors evaluate energy error, rather than just category placement?
Sam: They do. Their aggregate error measure scales root mean square error by average recorded demand. In that same comparison, it falls from about thirty-four percent to about fifteen percent. This supports usefulness beyond making the maps look more convincing.
Alex: Is that the best-looking example, or does the basic improvement appear with other predictors too?
Sam: All the evaluated prediction methods improve with dynamic synthetic augmentation, though the gains vary. Static synthetic examples also help, and dynamic examples improve further. The advantage narrows as more real training data become available, fitting the claim that this is especially useful under scarcity.
Alex: So the evidence supports synthetic data as a training resource. Does it also establish that an imagined future neighbourhood would actually consume energy as generated?
Sam: No. That is the main boundary I would draw. The experiments support prediction and seasonal consistency within this assembled dataset; they do not validate the energy performance of a city that has not been built.
Alex: What limits confidence beyond these settings? Geography, time coverage, or the quality of the underlying records?
Sam: All matter here. The energy records come from a single year, and geographic coverage remains limited. Specialists filtered out maps with large missing regions, but the authors still call for broader coverage and external validation using utility audits or smart-meter data.
Alex: With that boundary in place, who should read the full document, and where should they begin?
Sam: Researchers in building energy modelling and geospatial generation should read it. Start with dataset construction, then the dynamic monthly generation method. Focus on the comparison between real-only training, static synthetic augmentation, and dynamic synthetic augmentation; that is where the practical case is strongest.
Alex: And for researchers who mainly want the idea, rather than a model to implement?
Sam: Generating an image together with its changing energy labels can make scarce records more useful. But useful synthetic training data are not yet a validated forecast of a future city.
Alex: Thanks for listening.