VGA-BenchV2: An Expanded Unified Benchmark and Multi-Model Framework for Evaluating Video Aesthetics and Generation Quality

1 Introduction

Text-to-video generation has rapidly evolved from early proof-of-concept systems into increasingly capable generative models that can synthesize coherent, temporally stable, and visually compelling videos from natural language prompts [Blattmann et al., 2023a; Blattmann et al., 2023b; Luo et al., 2023; Khachatryan et al., 2023]. Recent advances in diffusion models [Blattmann et al., 2023a; Song et al., 2020], transformer-based architectures [Liu et al., 2022; Selva et al., 2023], and large-scale vision-language pretraining [Chen et al., 2023; Wang et al., 2023b; Dou et al., 2022] have further accelerated this progress, enabling state-of-theart systems [Wan et al., 2025; Kong et al., 2024; Tang et al., 2025; Liu et al., 2024; HaCohen et al., 2024; Team, 2024; Ma et al., 2024; Yang et al., 2024; Wang et al., 2023a; Zhang et al., 2025; Wang et al., 2025a; Guo et al., 2023; Blattmann et al., 2023a] to support increasingly diverse creative scenarios. As these models move closer to real-world use in digital art, film production, advertising, and virtual reality, evaluation is no longer limited to verifying whether a video is technically plausible. It must also measure whether the generated content is aesthetically appealing, stylistically controllable, and aligned with human visual preferences.

However, most existing evaluation protocols are still designed around technical correctness. Metrics such as FVD [Unterthiner et al., 2019] and CLIP Score [Hessel et al., 2021], together with their advanced variants [Liu et al., 2023], primarily quantify distributional similarity, temporal consistency, prompt alignment, or low-level visual artifacts. These measurements are useful for diagnosing basic generation quality, but they provide limited insight into perceptual and artistic factors such as composition, lighting, color harmony, cinematic expression, and style controllability. More importantly, conventional metrics usually serve as passive evaluation tools: they can rank or compare models, but are less effective at providing human-aligned supervision that can be directly used to improve video generators.

A series of video generation benchmarks have been proposed to address this evaluation gap. V-Bench [Huang et al., 2024] is a representative effort toward standardized multidimensional evaluation, but its treatment of video aesthetics remains coarse and depends heavily on off-the-shelf scoring models such as MUSIQ [Ke et al., 2021] and DINO [Caron et al., 2021]. VGA-Bench [Jiang et al., 2026] further advances this direction by introducing a fine-grained taxonomy for jointly evaluating aesthetic quality, aesthetic tags, and generation quality. Nevertheless, as video generation models become stronger and more widely used, a benchmark alone is insufficient: reliable aesthetic evaluation requires substantially larger human-labeled supervision, more expressive evaluator architectures, and a mechanism to convert human preference judgments into actionable optimization signals. These requirements motivate VGA-BenchV2, which extends VGA-Bench from a fine-grained evaluation benchmark into a human-aligned framework that unifies large-scale annotation, hybrid automated evaluation, and reinforcement learning-based model optimization.

Notably, V-Bench [Huang et al., 2024] represents a pioneering systematic effort to evaluate AIGC videos across multiple dimensions, marking a significant step toward standardized evaluation. However, it reduces the multifaceted nature of video aesthetics into a limited set of scalar measurements and relies heavily on off-the-shelf scoring models, such as MUSIQ [Ke et al., 2021] and DINO [Caron et al., 2021]. This reliance inevitably leads to coarse granularity, potential domain bias, and limited interpretability for improving generation models. VGA-Bench [Jiang et al., 2026] further advances this direction by introducing a fine-grained benchmark for jointly evaluating video aesthetic quality, aesthetic tags, and generation quality. Nevertheless, its evaluator training is still limited by the scale of human-labeled supervision, and the benchmark mainly focuses on evaluation rather than forming a closed loop from human preference modeling to generative model optimization.

To address these limitations, this paper introduces VGABenchV2 (as shown in Figure 1), an extended humanaligned benchmark and optimization framework for video aesthetic and generation quality assessment. Built upon VGA-Bench [Jiang et al., 2026], VGA-BenchV2 preserves the original fine-grained taxonomy with two primary dimensions, Aesthetic and Generation, and 52 sub-dimensions, while substantially extending the human annotation scale, evaluator architecture, and optimization capability. Our main contributions are summarized as follows:

• Large-scale human annotation expansion. Compared with VGA-Bench [Jiang et al., 2026], VGABenchV2 substantially expands human-labeled supervision by adding 36,000 newly collected task-level anno-

tations, including 16,200 annotations for aesthetic quality assessment, 13,200 annotations for aesthetic tagging, and 6,600 annotations for generation quality assessment. This corresponds to 13.46×, 11.15×, and 1.55× scale-ups over VGA-Bench for the three tasks, respectively, transforming the benchmark from an evaluationoriented suite into a large-scale human-aligned training infrastructure.

• Human-supervised hybrid evaluator architecture. Leveraging the enlarged annotation corpus, we develop a hybrid evaluator system consisting of VAQANet for continuous aesthetic scoring and two Qwenbased Large Vision-Language Model evaluators, VTagNet and VGQA-Net, for aesthetic tagging and generation quality assessment, respectively. This design combines specialized aesthetic regression with LVLM-based semantic reasoning, enabling scalable, interpretable, and human-aligned evaluation across diverse generation models.
• Evaluation-to-optimization via reinforcement learning. Beyond passive evaluation, VGA-BenchV2 introduces an evaluation-to-optimization pipeline in which the learned aesthetic evaluator serves as a reward model for reinforcement learning-based fine-tuning of video generators. By using the aesthetic score as a reward signal, the framework converts human aesthetic preferences into actionable optimization objectives, demonstrating that VGA-BenchV2 can directly guide generative models toward higher aesthetic quality and better human preference alignment.
• Unified benchmark, evaluator, and optimization infrastructure. VGA-BenchV2 integrates 52 fine-grained evaluation dimensions, 1,016 diverse prompts, over 60,000 videos generated by 12 mainstream models, large-scale human annotations, hybrid automated evaluators, and reinforcement learning-based optimization into a unified framework. This enables systematic analysis, fair cross-model comparison, and closed-loop improvement of video generation models.

We believe that VGA-BenchV2 serves not only as a rigorous evaluation platform, but also as a key infrastructure for advancing the next generation of video generation systems with enhanced aesthetic intelligence, artistic controllability, and human preference alignment.

2 RelatedWork

Evaluating video generation models remains challenging despite their rapid progress. Early metrics such as FVD [Unterthiner et al., 2019], CLIP Score [Hessel et al., 2021], and their advanced variants [Liu et al., 2023] mainly measure distributional similarity, temporal consistency, prompt alignment, or low-level visual artifacts. While useful for assessing basic technical quality, these metrics are insufficient for capturing fine-grained perceptual factors such as composition, lighting, color harmony, artistic style, and overall aesthetic appeal.

Recent benchmarks have promoted more systematic evaluation of video generation models. V-Bench [Huang et al., 2024] provides a representative multi-dimensional evaluation protocol, and V-Bench2 [Zheng et al., 2025] further extends the evaluation scope. Other benchmarks, including ChronoMagic-Bench [Yuan et al., 2024], T2VCompBench [Sun et al., 2025], and StoryEval [Wang et al., 2025b], focus on complementary aspects such as temporal coherence, compositional binding, and narrative consistency. Compared with these works, VGA-Bench [Jiang et al., 2026] is distinguished by its fine-grained treatment of video aesthetics, jointly modeling aesthetic quality, aesthetic tags, and generation quality within a unified taxonomy.

VGA-BenchV2 further extends VGA-Bench [Jiang et al., 2026] from a fine-grained evaluation benchmark into a human-aligned evaluation and optimization framework. Specifically, it strengthens human-labeled supervision, introduces a hybrid evaluator architecture that combines specialized aesthetic scoring with LVLM-based tagging and generation-quality assessment, and further connects evaluation with reinforcement learning-based generator optimization. In this way, VGA-BenchV2 moves beyond passive benchmarking and forms a closed-loop framework that links benchmark design, human annotation, automated evaluation, and model improvement.

3 VGA-BenchV2 Construction

3.1 Inherited Evaluation Taxonomy

VGA-BenchV2 inherits the fine-grained evaluation taxonomy of VGA-Bench [Jiang et al., 2026] to ensure continuity, reproducibility, and cross-version comparability. Rather than redesigning the benchmark dimensions, we preserve the original taxonomy as the evaluation backbone and focus the V2 extension on human annotation expansion, hybrid evaluator training, and evaluation-to-optimization. Overall, the taxonomy contains 52 sub-dimensions organized into two primary categories: Aesthetic and Generation.

The Aesthetic category consists of two complementary perspectives.

• Aesthetic Quality measures the perceptual appeal of generated videos as continuous scores, covering ten dimensions adapted from VADB [Qiao et al., 2025]: Overall Score (Ovr), Composition (Com), Shot Size (SS),

Lighting (Lig), Visual Tone (VT), Color (Col), Depth of Field (DoF), Expression (Exp), Costume (Cos), and Makeup (Mak).

• Aesthetic Tagging describes discrete visual and stylistic attributes as classification labels. Following photographic and cinematographic principles [Qiao et al., 2025; Matbouly, 2022; Deren, 1960; Brown, 2016], it contains eleven tag dimensions: Composition Types (CT), Shot Type (ST), Number of Light Sources (NoLS), Light Source Position (LSP), Light Quality (LQ), Light Color (LC), Color Temperature (ColT), Saturation (Sat), Brightness (Bri), Contrast (Con), and Depth of Field (DoF). These 21 aesthetic-related dimensions support both quantitative model ranking and interpretable analysis of visual style controllability.

The Generation category evaluates whether generated videos are semantically faithful, physically plausible, and visually stable. Following VGA-Bench [Jiang et al., 2026] and building upon V-Bench [Huang et al., 2024], VGA-BenchV2 retains 31 generation-quality sub-dimensions grouped into three aspects.

• Video-Text Consistency includes Character-Text Consistency (Cha-T), Action-Text Consistency (Act-T), Object-Text Consistency (Obj-T), Scene-Text Consistency (Sce-T), Object Position-Text Consistency (PosT), Camera Movement-Text Consistency (Cam-T), Object Attribute-Text Consistency (Att-T), Video ContentText Consistency (Cnt-T), Video Style-Text Consistency (Sty-T), and Video Speed-Text Consistency (Spd-T).
• Reality & Plausibility includes Rigid Body Collision Realism (Rig-R), Fluid Motion Realism (Flu-R), Gaseous Motion Realism (Gas-R), Gradual Change Realism (Gra-R), Object Trajectory Realism (Tra-R), Action Realism (Act-R), Scene Realism (Sce-R), Object Realism (Obj-R), Character Generation Quality (ChaR), Weather Representation Realism (Wea-R), Time Period Representation Realism (Tim-R), Lighting & Shadow Realism (Lig-R), Moving Scene Reasonableness (Dyn-R), Textual Attribute Representation (Att-R), Overall Realism (Glo-R), and Unrealistic Description Imaginative Presentation (Ima-R).

AesthetiĐ QualitLJ

Model：AŶiŵateDiff-ǀϮ

Proŵpt：A ĐoŶĐeptual ŵakeup

ǀideo ǁith a stroŶg artistiĐ feel,

usiŶg high-ĐoŶtrast ďlaĐk aŶd

ǁhite throughout to highlight

the struĐture aŶd ……

DiŵeŶtioŶs：VisualtoŶe +

Costuŵe

AesthetiĐ TaggiŶg

Model：Latte-ϭ

Proŵpt：The Ŷight ŵarket stalls are agloǁ ǁith flashiŶg lights, ĐreatiŶg a Đoŵpledž lightiŶg sLJsteŵ froŵ ŵultiple sourĐes……

DiŵeŶtioŶs：CoŵpositioŶ +

Light SourĐe Nuŵďer + Light

SourĐe Color + ……

GeŶeratioŶ QualitLJ

Model：HuŶYuaŶ

Proŵpt：OŶ the ArĐtiĐ

glaĐier, reǀerse plaLJďaĐk shot

of aŶ iĐe ďloĐk falliŶg iŶto the

ǁater ǁith a statiĐ Đaŵera.

DiŵeŶtioŶs：SĐeŶe-Tedžt CoŶsisteŶĐLJ + SĐeŶe Realisŵ + Video Speed-Tedžt CoŶsisteŶĐLJ + Caŵera MoǀeŵeŶt-Tedžt CoŶsisteŶĐLJ

• Basic Quality includes Video Noise-Free (Noi), Abnormal Lighting Detection (AbL), Video Clarity (Cla), Static Content Stability (Sta), and Static Content Nondistortion (Dis). We use the suffix "-T" to denote videotext consistency dimensions and "-R" to denote reality or plausibility dimensions, while basic quality dimensions use standalone mnemonic codes. Detailed definitions and evaluation criteria are provided in the Appendix.

3.2 Prompt Suite and Video Pool

VGA-BenchV2 inherits the prompt suite from VGABench [Jiang et al., 2026] to ensure cross-version comparability and stable dimension-wise evaluation. The prompt suite is designed under the principle of explicit attribute specification: each prompt explicitly triggers one or more target dimensions, such that annotators and automated evaluators assess only the attributes that are semantically grounded in the prompt. This design reduces ambiguity during both human annotation and automated evaluation.

The prompt suite contains 1,016 prompts, including 200 prompts for aesthetic quality, 220 for aesthetic tagging, and 596 for generation quality. Each dimension is covered by at least 50 prompts, and the suite supports both singledimension and multi-dimensional combinations. We also retain two lightweight subsets with 508 and 127 prompts for efficient evaluation under different computational budgets.

In VGA-BenchV2, the prompt suite serves as a unified anchor for three downstream components: large-scale human annotation, supervised evaluator training, and evaluation-tooptimization experiments. By reusing the same dimensionaligned prompts as VGA-Bench, VGA-BenchV2 isolates the effect of expanded human supervision and improved evaluator design, while preserving a consistent benchmark protocol for comparing video generation models.

3.3 Large-scale Human Annotation Expansion

A central goal of VGA-BenchV2 is to transform VGA-Bench from an evaluation-oriented benchmark into a human-aligned training infrastructure for automated video evaluators. To this end, we substantially expand the task-level human annotations over VGA-Bench [Jiang et al., 2026]. Specifically, VGA-BenchV2 introduces 36,000 newly collected annotations, including 16,200 annotations for aesthetic quality assessment, 13,200 annotations for aesthetic tagging, and 6,600 annotations for generation quality assessment. These annotations provide direct supervision for evaluator training, alignment verification, and reward modeling.

Annotation Protocol and Quality Control

We follow an expert-guided and multi-annotator annotation protocol. Domain experts first provide exemplar annotations and detailed rating guidelines for each task. Trained annotators then label the remaining samples according to these exemplars, while experts conduct batch-wise audits to ensure annotation consistency. If a batch fails the quality check, it is rejected and re-annotated.

For aesthetic quality, each sample is scored on a 0-10 scale, and the final score is obtained by averaging multiple independent ratings. For aesthetic tagging, annotators assign discrete visual labels such as composition type, light source position, shot type, color temperature, and contrast, with final labels determined by majority voting. For generation quality, each dimension is formulated as a task-specific question with structured ordinal options, enabling the labels to capture different levels of semantic consistency, realism, and basic visual quality.

Importantly, all annotation tasks follow the explicit-trigger principle inherited from the prompt suite: annotators evaluate only the dimensions explicitly specified by the prompt. This prevents subjective speculation on unmentioned attributes and ensures that each label is semantically grounded in the prompt-video pair.

3.4 Hybrid Automated Evaluation Networks

Leveraging the expanded human-annotated corpus, VGABenchV2 moves beyond lightweight task-specific evaluators to train a robust, human-aligned hybrid evaluator system. The architecture combines specialized aesthetic regression with LVLM-based semantic reasoning, consisting of VAQA-Net for continuous aesthetic scoring, VTag-Net for aesthetic tagging, and VGQA-Net for generation quality assessment.

VAQA-Net (Aesthetic Scoring): We adopt a two-stage transfer learning strategy. First, the video encoder is initialized from the VADB pre-trained model to inherit aesthetic representations. Second, the model is fine-tuned on a largescale generated video dataset, including 8,366 videos from 12 mainstream generation models with high-quality human annotations. Evaluation is performed on a held-out set of 1,186 generated videos to assess generalization.

VTag-Net & VGQA-Net (LVLM-based Evaluators): To enable semantic reasoning and detailed assessment, we leverage Qwen3-VL-32B [Bai et al., 2025] as the backbone and apply instruction tuning for task-specific adaptation.

• VTag-Net: Trained on a combination of real VADB videos and 11,100 generated videos with humanannotated aesthetic tags, outputting discrete visual labels.
• VGQA-Net: Trained on 7,054 generated videos with human-annotated question-answer pairs, optimized for coherent reasoning and generation quality assessment.

The training set includes representative samples from all 12 source models to ensure generalization across generation paradigms.

The network architectures of VAQA-Net, VTag-Net, and VGQA-Net are illustrated in Figure 5.

3.5 Evaluation-to-Optimization Interface

Beyond serving as passive evaluation metrics, the learned evaluators in VGA-BenchV2 can be used as reward models for generator optimization. In particular, the aesthetic score predicted by VAQA-Net provides a scalar reward reflecting human-aligned visual preference. This reward can be incorporated into reinforcement learning-based fine-tuning pipelines, enabling video generators to optimize toward higher aesthetic quality while maintaining stability.

Formally, given a prompt p, the current policy πθ generates a video v ∼ πθ(·|p), and VAQA-Net outputs a scalar reward:

with ϕ denoting the frozen evaluator parameters. To stabilize training, KL regularization is applied:

and the normalized advantage is

where µB and σB denote the batch mean and standard deviation.

For policy optimization, the unclipped and clipped advantages are defined as

and the clipped GRPO objective is

This formulation encourages higher-reward samples to contribute stronger updates while constraining policy drift. Detailed implementation and training parameters are described in Section 4.

4 Experiments and Results

4.1 Validation of Automated Evaluator

To ensure the reliability of VGA-BenchV2, we rigorously validated our automated evaluators against human ground truth on a held-out test set. For aesthetics, Table 3 shows that VAQA-Net achieves a high SROCC of 87.6% on the overall score, demonstrating strong ranking consistency with human experts. Simultaneously, VTag-Net delivers robust classification performance, particularly in objective dimensions like Light Color (93.6%) and Number of Light Sources (83.7%), validating its precision in identifying visual elements. For generation quality, VGQA-Net (Table 4) exhibits consistent alignment across all 31 fine-grained dimensions, with an average accuracy approximately 71.3%. High scores in challenging categories like Scene Realism (89.2%) and Abnormal Lighting Detection (85.7%) confirm that our MLLM-based evaluators serve as effective, scalable proxies for human assessment.

These high scores not only demonstrate the effectiveness of our models but also indicate that their predictions are closely aligned with expert human annotations, corroborating the human-aligned supervision highlighted in Table 1.

4.2 VGA-BenchV2 Evaluation Results

We evaluate all generative models using the trained VAQANet, VTag-Net, and VGQA-Net to assess their performance across diverse aesthetic and quality dimensions. To ensure a fair and unbiased ranking, all generated videos used for evaluation are strictly held out from the evaluator training process, preventing data leakage. The evaluated models are listed chronologically by release date: Stable Video Diffusion (SVD) [Blattmann et al., 2023a], AnimateDiff-v2 [Guo et al., 2023], LaVie [Wang et al., 2025a], Show-1 [Zhang et al., 2025], ModelScope [Wang et al., 2023a], CogVideoX [Yang et al., 2024], Latte-1 [Ma et al., 2024], Mochi [Team, 2024], LTXVideo [HaCohen et al., 2024], HunyuanVideo [Kong et al., 2024], Wan2.2 [Wan et al., 2025], and Sora2 [Liu et al., 2024].

The ranking metrics are defined as follows:

• Aesthetic Quality: Models are ranked by their average predicted score (higher is better).
• Aesthetic Tagging: We utilize alignment accuracy as the metric (i.e., whether the generated video successfully reflects the prompted aesthetic tag), where higher mean accuracy indicates superior controllability.
• Generation Quality: Models are ranked by the mean score across all sub-dimensions, where a higher average indicates better generation fidelity and consistency.

The final aggregated results, obtained by normalizing and averaging scores across all sub-dimensions within the three main categories, are presented in Table 5. For detailed performance breakdowns of each model across specific subdimensions, please refer to the Appendix.

4.3 Aesthetic Optimization via Reinforcement Learning

To validate the practical effect of our evaluators, we finetune Wan2.1 using the evaluation-to-optimization interface described in Section 3.5. As shown in Figure 6, the VAQANet Overall Score is used as the reward, and Flow-GRPO [Liu et al., 2025] is applied with LoRA and ODE-to-SDE conversion.

Quantitative results on a held-out test set show that the average aesthetic score increases from 0.49 to 0.52. Qualitative results indicate that fine-tuned videos exhibit more appealing visual composition and higher overall aesthetic quality. These results demonstrate that VGA-BenchV2 evaluators can support not only passive evaluation but also reward-driven video generator optimization.

5 Conclusion

We present VGA-BenchV2, an extended human-aligned benchmark and optimization framework that builds upon VGA-Bench by preserving 52 sub-dimensions, 1,016 prompts, and over 60,000 videos while introducing three key advancements: expanded human-labeled supervision for aesthetic quality, aesthetic tagging, and generation quality; a hybrid evaluator architecture combining VAQA-Net and LVLM-based VTag-Net and VGQA-Net for accurate, interpretable, and scalable assessment; and an evaluation-tooptimization pipeline enabling RL-based fine-tuning of video generators to align outputs with human aesthetic preference. Experiments demonstrate that VGA-BenchV2 provides more precise, human-aligned evaluation and facilitates multidimensional optimization. In the future, it can be extended to higher-resolution videos, additional aesthetic dimensions, cross-modal generation, and controllable creative applications, supporting next-generation AIGC video systems that harmonize fidelity, artistry, and human preference.

Ethical Statement

We strictly ensure that all prompts and generated videos within this benchmark have been rigorously screened to exclude pornographic, violent, or otherwise offensive content. Furthermore, all human annotation procedures comply with ethical guidelines and have been reviewed and approved by the relevant Institutional Review Board (IRB).

Acknowledgments

This work was supported by the Ant Group Research Fund, the National Natural Science Foundation of China under Grant No.62072014, and the Opening Project of the State Key Laboratory of General Artificial Intelligence, BIGAI/Peking University, Beijing, China (Project No.SKLAGI2025OP01).