One of the most popular approaches for solving total variation-regularized optimization problems in the space of measures are Particle Gradient Flows (PGFs). These restrict the problem to linear combinations of Dirac deltas and then perform a Euclidean gradient flow in the weights and positions, significantly reducing the computational cost while still decreasing the energy. In this work, we generalize PGFs to convex optimization problems in arbitrary Banach spaces, which we call Atomic Gradient Flows (AGFs). To this end, the crucial ingredient turns out to be the right notion of particles, chosen here as the extremal points of the unit ball of the regularizer. This choice is motivated by the Krein-Milman theorem, which ensures that minimizers can be approximated by linear combinations of extremal points. We investigate metric gradient flows of the optimization problem when restricted to such sparse representations, for which we define a suitable discretized functional that we show to be to be consistent with the original problem via the means of $Γ$-convergence. We prove that the resulting evolution of the latter is well-defined using a minimizing movement scheme, and we establish conditions ensuring $λ$-convexity and uniqueness of the flow. Then, using Choquet's theorem, we lift the problem into the Wasserstein space on weights and extremal points, and consider Wasserstein gradient flows in this lifted setting. Our main result is that the lifting of the AGF evolution is again a metric gradient flow in the Wasserstein space, verifying the consistency of the approach with respect to a Wasserstein-type dynamic. Finally, we illustrate the applicability of AGFs to several relevant infinite-dimensional problems, including optimization of functions of bounded variation and curves of measures regularized by Optimal Transport-type penalties.
Alex: Welcome to another episode of ResearchPod. Sam, what paper are we looking at today?
Sam: This is a paper called "Atomic Gradient Flows: Gradient Flows on Sparse Representations" by Christian Amend, Marcello Carioni, and Konstantinos Zemas. It tackles a key challenge in math: how to solve optimization problems—finding the input that gives the lowest value for a given function—in spaces that are infinite-dimensional. These are like spaces with endlessly many directions or coordinates, far beyond the three we see in everyday life, such as the space of all possible probability measures or functions.
Alex: So the core puzzle here is figuring out efficient ways to optimize in those vast, infinite spaces without drowning in endless possibilities?
Sam: Yes, exactly. Traditional methods struggle because directly handling infinite dimensions is computationally impossible—you can't check every point. The paper generalizes a method called particle gradient flows, which works well for certain image-processing tasks like denoising with total variation regularization, where you smooth noisy pictures while keeping sharp edges by favoring sparse solutions.
Alex: Right, so particle gradient flows simplify by using a finite bunch of particles, like points with weights, to approximate the full measure space solution?
Sam: That's correct. In those flows, you restrict to sparse combinations of Dirac deltas—think of them as point masses at specific locations—and evolve their positions and weights via a gradient flow, which is like steadily rolling downhill on an energy landscape defined by the problem. But this was limited to measure spaces; the paper extends it to broader Banach spaces, which are complete normed vector spaces where convergence is well-behaved, using extremal points of the regularizer's unit ball as the new "atoms" or building blocks.
Alex: And why extremal points specifically—what makes them the right sparse basis?
Sam: The Krein-Milman theorem guarantees that in convex sets, the extreme points—those that can't be averaged from others inside the set—can approximate any point via combinations, much like using corner Lego bricks to build complex shapes efficiently. By optimizing sparse linear combinations of these, weighted by squares of coefficients, the approach stays tractable while Γ-converging to the true infinite-dimensional minimizer, meaning minimizers of the sparse version approach the real ones as you add more atoms.
Alex: So these atomic gradient flows let you evolve sparse combos of extremal points... but how do they connect back to a true gradient flow in the full space?
Sam: To bridge that, the authors lift the sparse problem to a space of probability distributions over pairs of weights and extremal points—like treating your collection of particles not as fixed list, but as a probability cloud describing all possible such pairs. This lifted functional matches the original sparse energy exactly, thanks to something called Choquet representation, which says any point in the convex set can be written as an average over the extremal points, similar to expressing any color as a mix of pure pigments.
Alex: A probability cloud over the building blocks... okay, so you can run a metric gradient flow there. But what's special about how they solve it?
Sam: The space of weights times extremal points is just a metric space without smooth structure—no easy derivatives like in flat space—so they use a fully metric toolkit. They prove the lifted functional is λ-convex, which ensures the minimizing movement scheme works well. Every such movement forms a curve of maximal slope for the functional, meaning it descends the energy as fast as possible given the local steepness, like rolling downhill at top speed without shortcuts.
Alex: And that ties the finite-n atomic flows to the infinite limit?
Sam: Precisely: if you take the atomic gradient flow for the n-particle discretized version, and form the empirical measure—just the average of Dirac deltas at each particle's position—that sequence of distributions is itself a curve of maximal slope for the lifted functional. The proof uses optimal transport tools on metric spaces to match the slopes particle-by-particle. It's a notable link showing sparse dynamics recover the true flow behavior, though full limits as n grows await future work.
Alex: So the empirical measure from the finite atomic flow acts like a maximal slope curve for the lifted functional. But walk me through how they build those recovery sequences to match the true energy—why do the sparse combos actually hit the optimum in the limit?
Sam: For the upper bound in the convergence, they start with any candidate solution where the regularizer isn't zero. By the Krein-Milman theorem, they find convex combinations of extremal points that weakly converge to a scaled version of it. Then they tweak the weights by squaring and rescaling so the total matches exactly, and pad with zero-weight points to fit n particles—like filling a fixed tray with the right mix of ingredients, adding empties if needed. That converges weakly while the energy approaches the true value, thanks to continuity properties.
Alex: Padding with zeros keeps it sparse but fills the slots. And every sequence of minimizers for the bounded version then has a weak*-convergent subsequence to a true minimizer?
Sam: Yes—by coercivity and picking bounds above the true minimum, like knowing good approximations stay in a compact set and can't do worse than the best.
Alex: Okay, so minimizers converge. Now these atomic gradient flows—how do they actually define and compute the flow itself?
Sam: They use an approximation scheme called minimizing movements, or JKO scheme: from a starting point, repeatedly solve for the next point that minimizes the current energy plus half over time step τ times the squared distance to the previous—like plotting a path by always picking the lowest spot a small hop away, biased toward staying close. As τ shrinks to zero, the piecewise constant curve limits to a smooth absolutely continuous path in the space.
Alex: That stepwise minimization enforces the flow. But to get maximal slope behavior, they need the slope to control energy drop along curves?
Sam: Right—they prove the energy is λ-convex along geodesics in the bounded-weight space, assuming the forward operator has controlled first and second derivatives along geodesics, like smooth motion without jerks. This makes the metric slope a strong upper gradient: energy change along any curve is bounded by integral of slope times speed. Thus minimizing movements are curves of maximal slope, descending as fast as locally possible.
Alex: And if the extremal space has non-positive curvature—like a space where triangles don't bulge out—uniqueness follows with contraction?
Sam: Exactly: the product space inherits that property, so flows from different starts contract exponentially at a rate tied to λ and initials. This rigorizes the sparse dynamics as true gradient flows approximating the infinite case.
Alex: So the sparse atomic flows behave like true gradient flows in their own space, contracting nicely under those curvature conditions. But to tie back to the full infinite-dimensional problem, how do they show the finite-particle paths actually induce a gradient flow in this lifted probability space?
Sam: They define a distance on pairs of weights c and extremal points u, combining differences in c and the metric on u—like measuring how far two labeled building blocks are by both label and shape mismatch. Collections of such pairs become probability measures in a Wasserstein-2 space, where distance between collections averages squared pairwise distances over optimal matchings, similar to earth-mover's distance for dirt piles. The key functional on these takes the projected measure—which sums weighted bricks ignoring the sqrt weights—and plugs into the original energy.
Alex: Okay, so the projection recovers the sparse combo from the probability cloud. And they prove the minimum energies match exactly?
Sam: Yes—for large enough L bounding weights, any target sparse measure gets lifted by concentrating mass appropriately. Thus minimizers correspond perfectly. They run minimizing movements on this Wasserstein space, getting a piecewise curve limiting to an absolutely continuous path as tau shrinks.
Alex: That gives a flow for the lifted functional. But does it recover the atomic ones?
Sam: Precisely: take an n-particle maximal slope curve for the finite version on fixed particles. Form the empirical measure as average of Diracs at each particle position. Then the lifted energy matches exactly, speeds bound appropriately, and crucially the metric slopes match: the lifted slope at the empirical measure equals the finite one at particles, proved via localization for n=1 and dynamic plans generally. Thus the empirical path is maximal slope for the lifted functional—sparse finite dynamics lift directly to Wasserstein flow on atom distributions.
Alex: So the particle paths push a probability measure that's descending the lifted energy as fast as possible locally. That's a clean bridge from finite to the lifted infinite.
Sam: It is. This enables efficient computation of infinite-dimensional optimizations, like deblurring medical images where traditional total variation on measures stalls, by evolving just particles toward sparse extremal combos. The paper recovers established particle schemes for measure spaces and extends to one-dimensional functions with jumps or optimal transport paths. Full convergence to original functionals as n grows awaits future work.
Alex: There are limits—like needing the extremal space to have non-positive curvature for unique flows.
Sam: Yes, the approach assumes one-homogeneous convex regularizers, and while Γ-convergence ensures minimizers approximate well, proving gradient flow limits as n goes to infinity is future work. Still, the finite-particle dynamics offer a tractable way forward with contraction guarantees under those conditions.
Alex: A meaningful step for scalable optimization in imaging and machine learning inverse problems, where sparsity emerges automatically.
Sam: It generalizes sparse flows across Banach spaces, enabling real-time solutions without discretizing the continuum blindly. That's the core contribution here.
Alex: Thanks, Sam—this has been a clear dive into handling infinite dimensions practically. Thanks for listening to ResearchPod.