Christian Amend, Marcello Carioni, Konstantinos Zemas
10 min
Abstract
One of the most popular approaches for solving total variation-regularized optimization problems in the space of measures are Particle Gradient Flows (PGFs). These restrict the problem to linear combinations of Dirac deltas and then perform a Euclidean gradient flow in the weights and positions, significantly reducing the computational cost while still decreasing the energy. In this work, we generalize PGFs to convex optimization problems in arbitrary Banach spaces, which we call Atomic Gradient Flows (AGFs). To this end, the crucial ingredient turns out to be the right notion of particles, chosen here as the extremal points of the unit ball of the regularizer. This choice is motivated by the Krein-Milman theorem, which ensures that minimizers can be approximated by linear combinations of extremal points. We investigate metric gradient flows of the optimization problem when restricted to such sparse representations, for which we define a suitable discretized functional that we show to be to be consistent with the original problem via the means of $Γ$-convergence. We prove that the resulting evolution of the latter is well-defined using a minimizing movement scheme, and we establish conditions ensuring $λ$-convexity and uniqueness of the flow. Then, using Choquet's theorem, we lift the problem into the Wasserstein space on weights and extremal points, and consider Wasserstein gradient flows in this lifted setting. Our main result is that the lifting of the AGF evolution is again a metric gradient flow in the Wasserstein space, verifying the consistency of the approach with respect to a Wasserstein-type dynamic. Finally, we illustrate the applicability of AGFs to several relevant infinite-dimensional problems, including optimization of functions of bounded variation and curves of measures regularized by Optimal Transport-type penalties.
Alex: A probability cloud over the building blocks... okay, so you can run a metric gradient flow there. But what's special about how they solve it?
Sam: The space of weights times extremal points is just a metric space without smooth structure—no easy derivatives like in flat space—so they use a fully metric toolkit. They prove the lifted functional is λ-convex, which ensures the minimizing movement scheme works well. Every such movement forms a curve of maximal slope for the functional, meaning it descends the energy as fast as possible given the local steepness, like rolling downhill at top speed without shortcuts.
Alex: And that ties the finite-n atomic flows to the infinite limit?
Sam: Precisely: if you take the atomic gradient flow for the n-particle discretized version, and form the empirical measure—just the average of Dirac deltas at each particle's position—that sequence of distributions is itself a curve of maximal slope for the lifted functional. The proof uses optimal transport tools on metric spaces to match the slopes particle-by-particle. It's a notable link showing sparse dynamics recover the true flow behavior, though full limits as n grows await future work.
Alex: So the empirical measure from the finite atomic flow acts like a maximal slope curve for the lifted functional. But walk me through how they build those recovery sequences to match the true energy—why do the sparse combos actually hit the optimum in the limit?
Sam: For the upper bound in the convergence, they start with any candidate solution where the regularizer isn't zero. By the Krein-Milman theorem, they find convex combinations of extremal points that weakly converge to a scaled version of it. Then they tweak the weights by squaring and rescaling so the total matches exactly, and pad with zero-weight points to fit n particles—like filling a fixed tray with the right mix of ingredients, adding empties if needed. That converges weakly while the energy approaches the true value, thanks to continuity properties.
Alex: Padding with zeros keeps it sparse but fills the slots. And every sequence of minimizers for the bounded version then has a weak*-convergent subsequence to a true minimizer?
Sam: Yes—by coercivity and picking bounds above the true minimum, like knowing good approximations stay in a compact set and can't do worse than the best.
Alex: Okay, so minimizers converge. Now these atomic gradient flows—how do they actually define and compute the flow itself?
Sam: They use an approximation scheme called minimizing movements, or JKO scheme: from a starting point, repeatedly solve for the next point that minimizes the current energy plus half over time step τ times the squared distance to the previous—like plotting a path by always picking the lowest spot a small hop away, biased toward staying close. As τ shrinks to zero, the piecewise constant curve limits to a smooth absolutely continuous path in the space.
Alex: That stepwise minimization enforces the flow. But to get maximal slope behavior, they need the slope to control energy drop along curves?
Sam: Right—they prove the energy is λ-convex along geodesics in the bounded-weight space, assuming the forward operator has controlled first and second derivatives along geodesics, like smooth motion without jerks. This makes the metric slope a strong upper gradient: energy change along any curve is bounded by integral of slope times speed. Thus minimizing movements are curves of maximal slope, descending as fast as locally possible.
Alex: And if the extremal space has non-positive curvature—like a space where triangles don't bulge out—uniqueness follows with contraction?
Sam: Exactly: the product space inherits that property, so flows from different starts contract exponentially at a rate tied to λ and initials. This rigorizes the sparse dynamics as true gradient flows approximating the infinite case.
Alex: So the sparse atomic flows behave like true gradient flows in their own space, contracting nicely under those curvature conditions. But to tie back to the full infinite-dimensional problem, how do they show the finite-particle paths actually induce a gradient flow in this lifted probability space?
Sam: They define a distance on pairs of weights c and extremal points u, combining differences in c and the metric on u—like measuring how far two labeled building blocks are by both label and shape mismatch. Collections of such pairs become probability measures in a Wasserstein-2 space, where distance between collections averages squared pairwise distances over optimal matchings, similar to earth-mover's distance for dirt piles. The key functional on these takes the projected measure—which sums weighted bricks ignoring the sqrt weights—and plugs into the original energy.
Alex: Okay, so the projection recovers the sparse combo from the probability cloud. And they prove the minimum energies match exactly?
Sam: Yes—for large enough L bounding weights, any target sparse measure gets lifted by concentrating mass appropriately. Thus minimizers correspond perfectly. They run minimizing movements on this Wasserstein space, getting a piecewise curve limiting to an absolutely continuous path as tau shrinks.
Alex: That gives a flow for the lifted functional. But does it recover the atomic ones?
Sam: Precisely: take an n-particle maximal slope curve for the finite version on fixed particles. Form the empirical measure as average of Diracs at each particle position. Then the lifted energy matches exactly, speeds bound appropriately, and crucially the metric slopes match: the lifted slope at the empirical measure equals the finite one at particles, proved via localization for n=1 and dynamic plans generally. Thus the empirical path is maximal slope for the lifted functional—sparse finite dynamics lift directly to Wasserstein flow on atom distributions.
Alex: So the particle paths push a probability measure that's descending the lifted energy as fast as possible locally. That's a clean bridge from finite to the lifted infinite.
Sam: It is. This enables efficient computation of infinite-dimensional optimizations, like deblurring medical images where traditional total variation on measures stalls, by evolving just particles toward sparse extremal combos. The paper recovers established particle schemes for measure spaces and extends to one-dimensional functions with jumps or optimal transport paths. Full convergence to original functionals as n grows awaits future work.
Alex: There are limits—like needing the extremal space to have non-positive curvature for unique flows.
Sam: Yes, the approach assumes one-homogeneous convex regularizers, and while Γ-convergence ensures minimizers approximate well, proving gradient flow limits as n goes to infinity is future work. Still, the finite-particle dynamics offer a tractable way forward with contraction guarantees under those conditions.
Alex: A meaningful step for scalable optimization in imaging and machine learning inverse problems, where sparsity emerges automatically.
Sam: It generalizes sparse flows across Banach spaces, enabling real-time solutions without discretizing the continuum blindly. That's the core contribution here.
Alex: Thanks, Sam—this has been a clear dive into handling infinite dimensions practically. Thanks for listening to ResearchPod.