A Multi-stage Constrained Optimization Framework for Data-driven Problems

How can we effectively perform constrained optimization in high-dimensional, black-box settings where the objective and constraints are implicit and the search space is complex? The author addresses three persistent challenges in VAE-based optimization: sampling within latent spaces, identifying active decision variables, and enforcing hard constraints without destabilizing the training process.

The author introduces the Multi-stage Constrained Optimization Framework (MCOF), which operates in two distinct phases: a training phase to build a surrogate model and an optimizing phase to find solutions.

1. Training Phase: This phase uses an entropy-constrained VAE (EC-VAE) combined with a feature selector to embed critical objective and constraint information into a specific subset of latent variables. A Uniform Transformation (UT) module then maps the latent representation to a uniform distribution, which helps mitigate issues like posterior collapse and Gaussian mixture bias, effectively recovering convexity in the latent space.
2. Optimizing Phase: The framework employs a Constraint-Priority Filter Method (CPFM). Instead of using traditional penalty-based approaches, CPFM treats constraint satisfaction as a high-priority objective and uses a filter acceptance test to navigate the Pareto frontier between the objective function and constraint violations. This allows the framework to return feasible solutions to a specified tolerance.

Traditional optimization methods often struggle with black-box problems where the underlying functions are unknown or noisy. By leveraging the generative power of VAEs while imposing structural constraints on the latent space, MCOF provides a robust, general-purpose methodology. The framework's ability to handle hard constraints without the instability associated with Lagrangian multipliers makes it particularly promising for complex, real-world applications such as drug discovery and engineering design, where maintaining strict feasibility is essential.