Generating High-Diversity Synthetic Tabular Data via Less-Constrained Prior

1 Introduction

Tabular data is one of the most commonly encountered data types across diverse domains [Jordon et al., 2018], and tabular data synthesis is useful for a wide range of purposes [Hernandez et al., 2022; Miletic and Sariyar, 2024]. This has led to extensive research on tabular data synthesis, which requires estimating the multivariate joint distribution while handling complex dependency structures. To address this, the Variational AutoEncoder (VAE) has been widely adopted, which combines the principle of maximum log-likelihood estimation with latent spaces that capture dependency structures in the data space [Xu et al., 2019; An and Jeon, 2023; Liu et al., 2023; Zhang et al., 2024].

In VAE-based approaches, reducing the discrepancy between the prior and the aggregated (i.e., marginalized) posterior distribution is critical for generating high-quality synthetic samples [Li et al., 2020; Aneja et al., 2021]. To achieve this alignment, a promising approach is the two-stage generative modeling framework, which reduces the discrepancy by employing a trainable prior while maintaining a stable training process [Li et al., 2015; Dai and Wipf, 2019; Razavi et al., 2019; Rombach et al., 2022]. In this context, [Zhang et al., 2024] proposed TabSyn, a two-stage VAEbased model that demonstrates strong synthetic data generation performance.

∗Current address: Gwanak Lab Co., Ltd., South Korea.

†Corresponding author.

On the other hand, the quality of synthetic data is commonly evaluated along three dimensions: (1) fidelity, (2) diversity, and (3) utility [Kim et al., 2023; Liu et al., 2023]. It is well known that fidelity and diversity are regarded as tradeoffs, and that most existing methods achieve remarkable statistical fidelity [Hansen et al., 2023]. However, our empirical results indicate that there remains substantial room for improving diversity while maintaining fidelity. Diversity is determined by the extent to which the generative model covers the support of the true data distribution [Naeem et al., 2020; Alaa et al., 2022], and that, in latent-variable models, the support of the generative model is also determined by the effective latent support.

In this context, our main contributions are:

1. We show that existing two-stage modeling approaches

impose an additional constraint on the latent support.

1. We propose a new method in which this redundant constraint is effectively eliminated.
2. We demonstrate that our proposed approach significantly improves the quality of synthetic data, particularly in terms of diversity.

Our proposed objective function is directly derived from the KL-divergence between the ground-truth density and the generative model used for synthetic sample generation, relying on a single prior distribution. Our proposed method is termed SPT (Single Prior Tabular data synthesis). Note that previous studies have shown that over-regularization by a simple prior can lead to posterior collapse, distort latent structures (e.g., in clustering), and under-represent low-density regions, which degrades sample quality [Bowman et al., 2016; Dilokthanakul et al., 2016; Kim et al., 2024].

As illustrated in Figure 1, which summarizes fidelity, diversity, and overall performance, our proposed model achieves superior fidelity and diversity scores while maintaining high overall performance.

2 Preliminary andMotivation

Notations. A random variable x denotes an observation with p attributes comprising both continuous and discrete variables (i.e., mixed-type), and its jth coordinate is denoted as xj. The ground-truth probability density function of x is represented by p∗ . A lower-dimensional latent representation is denoted by z ∈ Rd .

Preliminary. The objective function of the conventional two-stage generative model can be written as:

where p(x | z; θ), p(z), and p(z; η) denote the decoder, fixed prior, and trainable prior distributions, respectively. q(z; ϕ) is the aggregated posterior, defined as q(z; ϕ) := ∫ q(z | x; ϕ) p∗ (x) dx where q(z | x; ϕ) is the posterior (or encoder), and it serves as the optimal prior that maximizes the ELBO [Tomczak and Welling, 2018].

The two problems in (1) are optimized sequentially, and for the first problem (i.e., stage 1), its upper bound

is minimized and is equivalent to the VAE framework, where the expectations of Ex,z and Ex are taken over p∗ (x)q(z | x; ϕ) and p∗ (x), respectively, and H(·) is the entropy.

The second optimization problem of (1) (i.e., stage 2) is additionally integrated into the conventional VAE framework to align the aggregated posterior with the trainable prior distribution. The trainable prior distribution can be parameterized with various generative models, such as GANs, VAEs, normalizing flows, or diffusion models [Choi et al., 2017; Dai and Wipf, 2019; Xie et al., 2023; Zhang et al., 2024].

Motivation. In the formulation of (1), there are two distinct prior distributions: p(z) and p(z; η). And the generative model p(x; θ, η), which is used for synthetic data generation, is defined with p(z; η):

where this approximation is carried out using the KLdivergence in stage 2, which means that q(z; ϕ) determines the distributional characteristics of p(z; η), including its effective latent support. Moreover, the non-informative prior in stage 1, p(z), serves solely to regularize the representation mapping and is not directly used in the generative model. Based on these observations, our research is guided by the followingmotivation: Is the non-trainable, fixed prior distribution p(z) essential for efficient two-stage generative modeling?

3 Proposal

In this section, we introduce our proposed method, SPT, to address the above research question.

3.1 Two-stage Modeling with Single Prior

To derive a two-stage generative model without a redundant prior distribution, we directly begin with the single KLdivergence between the ground-truth density p∗ (x) and the generative model p(x; θ, η), which is the ultimate goal of the generative modeling:

by Jensen's inequality (see Appendix for the detailed derivation). Note that the decoder variance parameter β, which also serves as a scheduling parameter, is included in θ.

Since the first term of the upper bound in (3) does not involve the parameter η, our approach naturally leads to the following two-stage generative modeling framework:

Stage 1: Without redundant constraint

In (4), the first term is the reconstruction loss, the second term is constant, and the last term serves as a regularizer for the posterior distribution.

Stage 2: Diffusion models

On the other hand, the objective function of stage 2 in (5) remains the same as that of the existing method described in Section 2. After training our two-stage framework, synthetic data are generated via an ancestral sampling procedure.

3.2 Less-Constrained Prior

From (2) and (4), the KL-divergence regularization imposed on the latent space by existing methods and our proposed method can be compared as follows:

where the expectation is taken over p∗ (x).

This implies that the additional latent regularization term KL(q(z; ϕ) ∥p(z)) is effectively canceled under our proposed method, unlike in existing approaches. Since p(z) is a non-informative prior, such regularization forces the latent distribution q(z; ϕ) to capture less information from the input data, which in turn reduces the diversity of latent variables generated by p(z; η) [Zhao et al., 2019; Rezaabad and Vishwanath, 2020].

Theoretical Analysis. (6) further indicates that the stage 1 objective in the existing two-stage generative modeling can be interpreted as a constrained optimization problem based on our proposed stage 1 objective (4), with an additional constraint

Definition 1. Suppose that the decoder is Gaussian, p(x | z; θ) = N (µ(z; θ), Σ) where Σ ≻ 0, and let the corresponding generative model be ˆ p(x; θ, ϕ) := ∫ p(x | z; θ)q(z; ϕ)dz. Let a measurable set Z0 ⊂ Rd denote the effective latent support of q(z; ϕ), defined as the subset of the latent space with high probability mass. For any α ∈ (0, 1), we define the reachable region induced by the generative model as

where cα := F -1 χ2p (1-α) and the (1-α) high-density region of each decoder is

Proposition 1. Suppose that q(z; ϕ) is a distribution absolutely continuous with respect to the prior p(z) and assume KL(q(z; ϕ) ∥ p(z)) ≤ ϵ. Then, for any t > 0, it holds that

where the t-region of q relative to p is defined as

Proposition 2. Under Definition 1 and the conditions of Proposition 1, it holds that

Proposition 1 implies that, as ϵ → 0, the aggregated posterior density concentrates on a subset of the support where the log-likelihood ratio with respect to a fixed prior (e.g., the standard Gaussian) is upper bounded, thereby restricting the effective latent support to Rt. Consequently, by Proposition 2, the generated samples fall into the reachable region with very high probability, whose size is determined by the effective latent support and is therefore constrained by the additional regularization (7). The proofs of Propositions 1 and 2 are provided in Appendix A.1 and A.2, respectively.

Significance. In existing two-stage modeling approaches, the effective support of the latent distribution is constrained by the additional regularization (7). As a result, the image of the decoder outputs in the data space is constrained, which consequently reduces the diversity of the generated synthetic samples. However, we derive a new objective without these constraints on the support by using only a single prior, which leads to significantly higher diversity in synthetic data (see Section 4.3).

3.3 Implementation

Closed-formregularization. We derive an upper bound for the posterior regularization term (the last term in (4)), which can be expressed in closed form under a diagonal Gaussian posterior, q(z | x; ϕ) := N (z | µ(x; ϕ), diag(σ2 (x; ϕ))). Here, diag(a) for a ∈ Rd denotes a diagonal matrix with diagonal entries given by a. µ(·; ϕ) : Rp ↦→ Rd and σ2 (·; ϕ) : Rp ↦→ Rd++ indicate the mean and variance vectors of the Gaussian encoder, respectively. Under this assumption, the posterior regularization term is bounded as

(see Appendix A.5 for details).

This regularization prevents latent variables from capturing information specific to individual observations. Intuitively, the upper bound in (8) can be viewed as the negative coefficient of determination (R2 ), which quantifies how much of the latent variability is explained by a single data point. Minimizing this term, therefore, encourages the latent representation to remain uninformative at the instance level.

Incorporating discrete columns. To handle heterogeneous tabular data, we define the decoder as a Gaussian distribution for continuous columns and a categorical distribution for discrete columns, both conditioned on the latent variable. The final objective function for stage 1, combining this decoder with our closed-form regularization of the posterior distribution in (8), is provided in the Appendix.

Stage 2. As in TabSyn [Zhang et al., 2024], we adopt scorebased diffusion models [Song et al., 2021; Karras et al., 2022] to minimize the KL-divergence term in (5), with ϕ fixed as the parameter fitted during stage 1. Due to space constraints, we refer the reader to the Appendix for the detailed optimization and training procedure of stage 2.

Controllable privacy. Since stage 2 of our framework generates latent representations from noise using a diffusionbased generative model, privacy can be enhanced at inference time by adjusting the noise level during latent sampling (i.e., by controlling the noise scaling factor c), without requiring any model re-training. Specifically, sampling the latent variable z at the initial time tN from N(0, c2 · σ2 (tN)I), where c = 1 serves as the baseline. This mechanism enables a direct and interpretable trade-off between privacy and sample quality. Moreover, perturbing latent variables is more effective than injecting noise into the generated data, as it jointly affects both continuous and categorical features through the compact latent representation of the model.

4 Experiments

We empirically evaluate SPT across a range of benchmark datasets. Our experiments 1 are designed to address the following key research questions 2 :

• RQ1. Does SPT achieve state-of-the-art performance in synthetic tabular data generation?
• RQ2. Towhat extent does a less-constrained prior contribute to the synthetic data generation performance of SPT?
• RQ3-1. Ablation study: How does the encoder variance affect the performance of SPT in synthetic data generation?
• RQ3-2. Ablation study: How does the adaptive regularization influence the synthetic data quality generated by SPT?

Datasets. To evaluate the performance of our method, we conduct experiments on 12 real-world tabular datasets from Kaggle and the UCI Machine Learning Repository 3 . These datasets include variation in the number of continuous and categorical variables and in the total sample sizes. A detailed description of each dataset is provided in the Appendix.

Baseline models. We compare our proposed method with the following 6 tabular data synthesizers. Details of the hyperparameter configuration for each baseline model are provided in the Appendix.

• GAN-based: CTGAN [Xu et al., 2019]
• VAE-based: TVAE [Xu et al., 2019]
• Transformer-based: TabMT [Gulati and Roysdon, 2023]
• Diffusion-based: TabDDPM [Kotelnikov et al., 2023], STaSy [Kim et al., 2023], TabSyn [Zhang et al., 2024]

4.1 EvaluationMetrics

To comprehensively evaluate the quality of the generated synthetic data, we consider three dimensions: (1) statistical fidelity, (2) statistical diversity, and (3) synthetic data utility.

1 All experiments are conducted on an NVIDIA RTX A6000 GPU using implementations in PyTorch.

2 We conduct an additional experiment to evaluate the privacy preservability of SPT in Subsection 4.5.

3 https://archive.ics.uci.edu/, https://www.kaggle.com/datasets/

Statistical fidelity. Statistical fidelity quantifies the extent to which the synthetic data replicate the distributional properties of the original dataset. We assess this aspect using four metrics: Goodness-of-Fit (GoF), Maximum Mean Discrepancy (MMD), Pairwise Correlation Difference (PCD), and α-precision [Alaa et al., 2022].

Statistical diversity. Statistical diversity evaluates how well the synthetic data captures the variability and support of the real data distribution. We assess this aspect using three metrics: β-recall [Alaa et al., 2022], Coverage [Naeem et al., 2020], and Clipped Coverage (CC) [Salvy et al., 2025].

Synthetic data utility. Synthetic data utility evaluates how effectively the synthetic data can be used for the downstream task. Following [Hansen et al., 2023], we adopt the three metrics: classification accuracy (Acc), model selection consistency (Model), and feature selection consistency (Feature).

Overall performance. To ensure fair evaluation across diverse metrics and to identify the best hyperparameter configurations, we adopt the TOPSIS method [Tzeng and Huang, 2011], a widely used multi-criteria decision-making approach across various domains [Behzadian et al., 2012]. Detailed descriptions of the evaluation procedure are in the Appendix.

4.2 RQ1: Synthetic Data Generation.

Overall results. As shown in Table 1, our proposed model SPT outperforms all baseline methods by achieving the best performance on 8 out of 10 evaluation metrics. Across statistical fidelity, statistical diversity, and synthetic data utility, SPT consistently demonstrates strong and well-balanced performance, whereas baseline models tend to favor specific aspects of the trade-off. Based on the TOPSIS results reported in Appendix D.1, which summarizes overall performance across all metrics, SPT achieves the highest TOPSIS score on 9 out of 12 datasets and ranks second on the remaining 3 datasets, indicating consistent performance across diverse real-world tabular datasets. According to the detailed experimental results presented in Appendix D, SPT achieves the best metric values in 53 out of 120 settings (12 datasets × 10 metrics), compared to 32 for TabDDPM and 30 for TabSyn.

Statistical fidelity. As shown in Table 1, SPT attains the best values for GoF (0.020), MMD (0.008), and PCD (0.441) outperforming the second best baseline, TabSyn, with relative improvements of 18.9%, 18.7%, and 5.9%, respectively. Although SPT ranks second in α-precision (0.962), its performance remains comparable to the best-performing baselines. Overall, these results demonstrate that SPT preserves the statistical properties of real data more faithfully than other synthesizers.

Statistical diversity. As shown in Table 1, SPT achieves the best performance on all diversity-related metrics, including β-recall, Coverage, and CC. Compared to second best baseline, SPT improves β-recall by 8.6%, indicating substantially enhanced coverage of rare or underrepresented modes in the data. Moreover, SPT achieves the highest Coverage (0.953) and CC (0.923), reflecting its ability to generate diverse samples while maintaining coherent joint distributions. These results suggest that the proposed approach effectively addresses the fidelity-diversity trade-off by allowing the model to have a less-constrained prior and a less-constrained effective latent support.

Synthetic data utility. In terms of synthetic data utility, SPT consistently matches or outperforms existing baselines in Table 1. SPT achieves the highest downstream classification accuracy (Acc: 0.741), improving upon the second best method by 0.3%. For feature-based and model-based utility metrics, SPT ranks first in Feature and second in Model. SPT maintains high fidelity and utility while achieving substantially improved diversity, highlighting its ability to generate synthetic data that is both realistic and practically useful for downstream tasks.

4.3 RQ2: Effect of Less-Constrained Prior

To investigate the effect of our regularization term, we compare three models with different forms of latent regularization: (1) AutoDiff [Suh et al., 2023], which applies no regularization to the latent space; (2) TabSyn, which imposes a KL-divergence between the aggregated posterior and a fixed Gaussian prior; and (3) SPT, which regularizes the posterior distribution using a self-regularization scheme. Note that the same latent dimensionality (d = 4) is used for AutoDiff, TabSyn, and SPT.

Latent space analysis. Figure 2 visualizes the resulting latent distributions p(z; η) and learned posteriors q(z; ϕ), and together with the corresponding class labels on the redwine dataset. The middle row in Figure 2 shows that the additional latent regularization (6) constrains the effective support of the learned posterior, creating large near-zero-density regions (i.e., holes) in latent space, consistent with our theoretical coverage bound in Section 3.2.

AutoDiff, which applies no explicit regularization, yields a latent space in which latent variables corresponding to the same class label form clearly separated clusters. TabSyn, due to the additional regularization in (6), produces much more compact class-wise clusters than AutoDiff, resulting in a strongly constrained latent support where large regions of the latent space carry almost no probability mass. In contrast, because the additional latent-support constraint induced by a fixed prior is eliminated, SPT yields a latent space in which class-wise clusters are connected without near-zero-density regions. Consistently, PCA scores in Figure 2 indicate that TabSyn exhibits substantially smaller values than SPT due to KL-divergence regularization toward a fixed standard Gaussian prior. Therefore, our proposed method enables smoother transitions within the effective latent support and thereby facilitates the generation of more diverse samples.

Comparison of regularization methods. We investigate how different regularization strategies influence synthetic data generation performance, as summarized in Table 2. Compared to AutoDiff (no latent regularization), TabSyn (KL regularization to a fixed Gaussian prior) improves performance across all evaluation dimensions, indicating that introducing a structured latent constraint can enhance synthetic data generation quality. However, additionally replacing the fixed-prior regularization with our self-regularized posterior (SPT), we observe substantially larger gains on nearly all metrics. SPT achieves the best values on 9 out of 10 metrics and yields the overall improvement under TOPSIS (0.922 → 0.969, +5.1%). Specifically, SPT yields substantial gains in diversity-related metrics (β-recall, Coverage, and CC), indicating improved coverage of minority modes, with only a slight drop in α-precision, reflecting a mild trade-off toward broader coverage.

These results support our argument that a fixed prior is not essential for effective two-stage generative modeling and that eliminating the additional constraint on latent support leads to improved synthetic data quality.

4.4 RQ3: Ablation Studies

Encoder variance. During stage 1, we fix the encoder variance to a constant for training stability, as the regularization term in (8) allows variance inflation to reduce the objective, potentially leading to unstable dynamics; empirically, treating the encoder variance as a non-trainable hyperparameter also consistently improves performance.

In Figure 3, we present empirical results comparing the performance of SPT with a trainable encoder variance to that of models using fixed variance values. When the encoder variance is fixed at σ2 = 0.1, SPT achieves an 11.6% improvement in MMD, indicating enhanced statistical fidelity. In terms of diversity, β-recall and Coverage improve by 26.4% and 2.3% under the same fixed variance setting, respectively. Synthetic data utility metrics also benefit from fixing the variance at σ2 = 0.1, with Feature increasing by 4.4%. We further observe consistent performance gains for σ2 = 0.1 across the remaining evaluation metrics (the complete experimental results are reported in Appendix D.3). Consequently, the TOPSIS-based evaluation consistently selects the fixed-variance setting with σ2 = 0.1 across all datasets.

Adaptive regularization scheduling. For KL-divergence regularization in stage 1, we adopt a scheduling strategy for the decoder variance parameter β. The detailed regularization scheduling algorithm and its implementation details are provided in Algorithm 1 in the Appendix.

As shown in Table 3, we observe that β-annealing does not merely improve one aspect (e.g., fidelity) at the expense of another (e.g., diversity); instead, it shifts SPT toward a better operating point where fidelity, diversity, and utility improve simultaneously. The most pronounced effects appear in fidelity metrics (GoF, MMD, PCD) and diversity metrics

(β-recall, CC). These results highlight the importance of gradually relaxing the regularization strength during training, especially in variational models where structured latent representations play a key role. The annealing process effectively balances reconstruction fidelity and latent space regularity, thereby improving the quality of generated samples.

4.5 Privacy Preservability

We evaluate our model's privacy preservability. For privacy evaluation, we measure k-anonymity property (k-anonymity) [Sweeney, 2002], Attribute Disclosure (AD) [Choi et al., 2017], and Distance to Closest Records (DCR) [Park et al., 2018].

Trade-off between privacy and quality. Figure 4 illustrates this trade-off by reporting the synthetic data quality metric Model and privacy preservability metric AD under varying noise strengths. We observe that the Model metric remains competitive as the noise scaling factor c increases from 1 to 2, indicating that moderate noise injection can improve privacy without significantly compromising Model. However, when c exceeds this range, Model gradually deteriorates relative to other baselines, even though AD remains consistently strong. These results demonstrate that our framework enables a controllable trade-off between privacy and synthetic data quality. Descriptions of the evaluation metrics and additional experiments are provided in the Appendix.

5 RelatedWorks

Two-stage generativemodeling. In this framework, a firststage model extracts a compact latent representation and a second-stage model learns its distribution [Li et al., 2015; Engel et al., 2018].[Dai and Wipf, 2019] showed that using a VAE in the first stage and a second VAE to model the latent distribution yields higher-quality samples than a single VAE. [Van Den Oord et al., 2017; Razavi et al., 2019] further demonstrated that discretizing the latent space with vector quantization in the first stage and utilizing an autoregressive generative model in the second stage produces a more meaningful representation and enables high-fidelity image synthesis. Notably, the latent diffusion model of [Rombach et al., 2022] achieves high-resolution image generation more efficiently than pixel-space DDPMs [Ho et al.,

2020], while preserving details and enabling flexible conditioning. This two-stage strategy has also shown strong performance in synthetic tabular data generation [Choi et al., 2017; Apell´aniz et al., 2024; Zhang et al., 2024]. Motivated by these successes, we also propose a two-stage generative framework that extends this line of work.

Over-regularization. [Bowman et al., 2016] showed that using a simple prior can cause posterior collapse, leading the decoder to ignore the latent representation and reducing the diversity of generated samples. [Dilokthanakul et al., 2016] demonstrated that over-regularization of the latent space in unsupervised clustering leads to the collapse of distinct cluster structures, where the model fails to separate data into meaningful groups and instead merges all data points into a single large cluster. [Kim et al., 2024] also showed that an overly simple prior imposes excessive regularization on the latent space, under-representing low-density regions and thereby degrading the quality of synthesized data.

Tabular data synthesis. Before the emergence of diffusion models and Transformers, GANs and VAEs dominated tabular data synthesis [Park et al., 2018; Xu et al., 2019; Zhao et al., 2023; An and Jeon, 2023]. Recently, Transformerbased approaches have gained attention. GReaT [Borisov et al., 2023] reformulates each table row as a text input and models its distribution using GPT-2. Other methods, such as TabMT [Gulati and Roysdon, 2023] and MaCoDE [An et al., 2024], adopt Transformer encoder architectures with masked language modeling. Furthermore, diffusion models have advanced significantly, with improved architectures and training protocols, and have often been highlighted for their ability to capture better multi-modal distributions, which is closely related to sample diversity. TabDDPM [Kotelnikov et al., 2023], STaSy [Kim et al., 2023], and CoDi [Lee et al., 2023] have adopted diffusion-based approaches. AutoDiff [Suh et al., 2023] and TabSyn [Zhang et al., 2024] employ a diffusion model in a latent space.

6 Conclusion and Limitation

We introduce a new two-stage generative modeling objective with a less-constrained effective latent support. This objective is derived directly from a single KL-divergence between the ground-truth distribution and the generative model, and our proposed approach relies on a single prior distribution, unlike existing methods. Through extensive experiments, we empirically demonstrate that relaxing the constraint on the effective latent support leads to significantly improved diversity in synthetic data, while improving or maintaining other quality measures.

Among many hyperparameters, we observed that the synthetic data quality of the model is particularly sensitive to the encoder variance. The dependency on the choice of this hyperparameter can be considered a limitation of our proposed model. Moreover, the diversity of generated samples is crucial not only in the tabular domain but also in other domains such as images [Razavi et al., 2019; Naeem et al., 2020]. An important direction for future work is to investigate whether the proposed less-constrained prior can similarly enhance synthetic sample diversity in other domains.

Acknowledgments

Sanghun Park and Jaesung Lim were supported by the National Research Foundation of Korea (NRF) grant funded by the Ministry of Science and ICT (MSIT) (No. RS2022-NR068754). Jong-June Jeon was supported by the National Research Foundation of Korea (NRF) grant funded by the Ministry of Science and ICT (MSIT) (No. RS-2022NR068754, No. RS-2025-23525436). Seunghwan An was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. RS-2026-25480090). Moreover, the authors acknowledge the Urban Big data and AI Institute of the University of Seoul supercomputing resources (http://ubai.uos.ac.kr) made available for conducting the research reported in this paper.

Contribution Statement

Sanghun Park and Jaesung Lim are co-first authors.