Hybrid-Adaptive Thread Tuning to Mitigate Simulation Execution Bottlenecks in High-Performance Reinforcement Learning Inference

1 Introduction

As artificial intelligence is increasingly used for decisionmaking in complex dynamic systems, reinforcement learning with simulation-in-the-loop (SiL-RL) has become a core paradigm for wireless communication network optimization and swarm intelligence control[Feriani and Hossain, 2021; Hu et al., 2024]. Its operational efficiency, however, directly limits real-time decision-making. SiL-RL performance depends on three components: simulation-side model computation (e.g., social network dynamic evolution models[Wu et al., 2022]), agent-side algorithm inference (e.g., Deep Q-Network decision-making[Zhang et al., 2021]), and their communication/synchronization. As simulation models grow increasingly complex (e.g., high-precision channel interference models[Eldeeb and Alves, 2025] in communication networks or high-fidelity aerodynamic models[Riboldi et al., 2025] in UAV swarms), the simulation side becomes the critical bottleneck[Morgado et al., 2025], increasing end-to-end latency and impairing real-time decisions. This paper therefore accelerates simulation-side performance to break this bottleneck and support efficient SiL-RL systems.

Existing SiL-RL simulation environments typically adopt Discrete Event Simulation (DES) and use multithreading on multicore processors for parallel acceleration[Belsare et al., 2022]. They mainly employ two architectures: centralized queues with thread pools and distributed queues with work stealing[Risco-Mart´ın et al., 2022]. The centralized queue, with dynamic load balancing and lower synchronization overhead, is well suited to SiL-RL inference, where agent activation rates and task loads fluctuate across simulation cycles[Lin et al., 2025]. This work therefore focuses on simulation acceleration under the centralized queue architecture.

However, centralized queues are highly sensitive to the number of worker threads (WTs): all WTs share one task queue, and task acquisition and distribution are handled serially by the scheduler[Samadi et al., 2024]. Thus, the scheduler may become a serial bottleneck: as WTs increase, scheduler contention, lock waiting, and busy waiting rise sharply, negating or even reversing parallel gains. Selecting WTs according to workload characteristics is therefore key to improving simulation-side performance[Li et al., 2025]. First, the optimal WT count depends on workload intensity and hardware factors such as core frequency, memory bandwidth, and cache hierarchy, which vary across platforms and are hard to know a priori. Second, SiL-RL workloads are highly dynamic; agent activation rates, task execution times, and interaction intensities fluctuate across simulation stages, making fixed WT counts suboptimal. Runtime noise and external disturbances further obscure transient interference versus persistent workload shifts. Together, these factors prevent precise WT allocation before or during runtime.

To address this, we propose AutoThread, a hybrid adaptive thread-tuning method for SiL-RL simulation acceleration. The core idea is to construct a WT count predictor where a neural network is guided and constrained by a theoretical performance model. By introducing a dynamic load-aware adaptive fine-tuning strategy, we achieve significant improvements in simulation performance, thereby supporting highperformance inference in RL decision-making systems. The main contributions of this paper are as follows:

• We open-source a large-scale multithreaded trajectory dataset for DES environment. Through empirical analysis, we derive three findings regarding the relationship between WT count and simulation performance.
• We develop a Physics-Informed Neural Operator (PINO) thread predictor, utilizing a queueing model to constrain and guide the training process.
• We propose AutoThread, which leverages PINO-based thread prediction and load-aware online fine-tuning to achieve fast and adaptive WT count allocation.

2 Empirical Study

We first introduce the dataset, then empirically reveal the relationship between WT count and execution performance to guide AutoThread's model construction and tuning strategy.

2.1 Dataset

We constructed the multithreaded trajectory dataset for the DES environment by collecting over 10,000 multithreaded execution samples across both Intel and AMD CPU hardware platforms. The dataset includes two application cases:

PCS: The Personal Communication System(PCS) [Carothers et al., 2002] is a benchmark DES environment that simulates wireless communication processes within a network topology and is widely used to test decision algorithms in communication networks. In this application, each agent represents a communication node, continuously generating five types of communication tasks.

UAV: This is a case study based on the DES paradigm, simulating the autonomous navigation and obstacle avoidance of multiple unmanned aerial vehicles (UAVs) within a specific area. The paths are derived from real-world flight data[FlightAware, 2025]. Each agent represents a UAV, involving both perception and navigation tasks.

To emulate high-fidelity simulation, we embedded array dot products and iterative power computations into each task, producing execution times between 10µs and 800µs. With profiling tools, we captured each sample's execution trajectories under various WT counts. The data include hardware utilization, application logs, and micro-architectural features, totaling 34 variables. To our knowledge, this is the first public DES trajectory dataset for RL systems and among the largest single-node thread-level trace sets; we release it to support studies on multithreaded behavior and DES acceleration.

2.2 Empirical Observations

We conduct three experiments to characterize how workload, hardware, and micro-architectural metrics jointly affect performance. Our analysis yields the following observations.

Dependence of Optimal WTs on Workload Intensity

Figure 2 shows speedup versus WT count under different agent populations and task execution times. In UAV, each agent always generates two fixed tasks with execution time t; in PCS, each agent generates a variable number of tasks whose execution times follow a Poisson distribution with mean t. The optimal WT count is positively correlated with average task execution time: higher workload intensity increases the optimal WT count because longer tasks enlarge parallelizable regions and let more threads improve performance. Conversely, with fixed execution time, the optimal WT count remains nearly unchanged regardless of agent scale(Figure 2(b)). The variance in Figure 2(d) arises because changing the number of PCS agents alters the task-queue execution-time distribution and thus its mean.

Contention-Induced Waiting as the Cause of Degradation

We conduct detailed analysis on a subset of samples using VTune. Figure 3 displays the distribution of hotspot functions and thread time consumption. As shown in Figure 3(a), the proportion of time spent on lock operations rises rapidly as the WT count increases. Once the lock-wait time exceeds the task execution time, system performance begins to degrade. Figure 3(b) further shows that the critical point at which busy waiting starts to increase sharply around the optimal WT count is insensitive to inactive wait times such as inactive synchronization and preemption.

Although busy waiting time is key to the optimal WT count, measuring it online is challenging[Jancauskas et al., 2019]. We therefore use lightweight metrics to indirectly reflect busy-waiting status, as shown in Fig 4. Figure 4(a) shows that usr cpu peaks near the optimal WT count, where CPU cycles are most efficiently used for parallel tasks. Once the WT count exceeds the optimum, system cpu (Figure 4(b)) rises sharply, indicating a shift from effective computation to OS scheduling and thread waiting. Since busywaiting threads still occupy physical cores, aggregate CPU utilization does not accurately reflect the optimal WT count (see Figure 4(c)). Figure 4(d) further shows that the context switch rate reaches a local minimum near the optimum, suggesting that an appropriate thread count reduces unnecessary scheduling switches. In contrast, cache miss rate (Figure 4(e)) and memory consumption (Figure 4(f)) affect DES performance but not optimal WT selection.

Linear Interference Under Multicore Concurrency

Figure 5 records the task execution time and task scheduling time for an individual working thread under varying numbers of active threads. The results show that as the number of active threads increases, both the execution and scheduling times for a single thread grow linearly. This indicates that the performance degradation of individual threads in a multithreaded environment can be approximated by a linear model, which captures the aggregate pressure exerted by all threads on a single thread's runtime. The slope and intercept of these linear functions depend on hardware characteristics

(e.g., core frequency, cache size) and workload intensity.

3 Methodology

Based on the empirical findings in Section 2, we propose AutoThread. AutoThread consists of two core components: a PINO predictor and a dynamic tuner. This approach adopts a two-stage optimization strategy of online prediction followed by dynamic adjustment: during the online prediction stage, it rapidly estimates the optimal initial thread count based on the characteristics of the current workload; during the dynamic adjustment stage, it performs fine-grained adaptive thread tuning according to runtime feedback.

3.1 Model-Informed PINO Predictor

Performance Model. In typical SiL-RL inference workflows, the simulator must process a large number of computation tasks generated by multiple agents within each simulation step. We decompose the execution of a single simulation cycle into three sequential phases: task execution, task scheduling, and synchronization. WT reconfiguration is performed at the granularity of simulation cycles, during which the WT count remains fixed.

During the task execution phase, WTs process assigned tasks in parallel. Since the system must wait for all threads to finish before synchronization, the total execution time is The above equation shows that execution time is a piecewisecontinuous, non-smooth function of the WT count k. Numerical analysis reveals that the optimal configuration occurs at k∗ ≈ µb/µe, indicating optimal performance when scheduler service rate balances with task execution rate.

determined by the slowest thread:

where N is the total number of tasks within the current simulation step; k is the number of WTs; and µe denotes the average service rate of tasks within the current simulation step.

Scheduling is inherently serial because the shared scheduler serves at most one WT at a time. We model the scheduler as an M/M/1 server and WTs as customers in a finite-source closed queueing system, where a WT requests scheduling only after finishing its current task, yielding an effective arrival rate λ = kµe. When the system is stable, steady-state Continuous-Time Markov Chain (CTMC) analysis gives:

where µb denotes the scheduler service rate and Po is the idle probability of the scheduler. When kµe exceeds µb, the scheduler becomes the dominant bottleneck and the total time is bounded by the scheduler capacity:

This captures the saturation regime under oversubscription.

The synchronization phase (e.g., global state update and clock advance by the master thread) is treated as constant with respect to k when focusing on WT selection.

Based on Section 2.2.3, resource contention causes performance degradation that grows linearly with active WT count:

where a1 and a2 denote contention intensity coefficients for scheduling and execution, respectively, and tbase1, tbase2 represent baseline times under single-thread execution. These parameters are determined by hardware platform and application context.

Combining the above, the total execution time of a single simulation cycle can be expressed as:

PINO Predictor. While µe can be estimated online from probe-collected execution times, directly measuring µb is difficult due to microsecond-scale sensitivity and measurement overhead. We therefore introduce a Physics-Informed Neural Operator (PINO) to fuse the above model with historical traces for fast online prediction.

We define a neural operator Gθ : a → k∗ = Gθ (a), which maps system state parameters a to the optimal WT number k∗ . We construct the feature vector a = {N, µe, kobs, usr cpu, system cpu, V SZ, cswch} from both application and hardware features. Although virtual memory usage (VSZ) has limited impact on thread decisions, excessive memory consumption may trigger resource contention or even abnormal program termination; thus, it is included as an input to enhance robustness.

As illustrated in Figure 6, PINO predictor employs a multioutput head cross-attention architecture. Encoders separately process application and hardware features, cross-attention captures their deep interactions, and FC layers decode fused features into optimal WT count k∗ and contention coefficients a1, b1. Physical loss:

Supervised loss and Regularization term:

where T represents the theoretical performance model (Equation 1), µb is derived from the contention model as µb = a1k + b1, and Tobs and kobs denote the observed task queue execution time and iteratively measured optimal WT number, respectively. The weight α normalizes contributions of the two physical components. The overall training objective is Lloss = Lphy + λLsup + Lreg. We incorporate ResidualBased Attention (RBA) [Anagnostopoulos et al., 2024] to dynamically adjust λ between physical and supervised losses. In out-of-distribution scenarios, RBA automatically increases the physical-loss weight, strengthening reliance on the performance model and improving prediction robustness.

3.2 Adaptive Dynamic Tuner

Based on the trained PINO predictor, we design an adaptive dynamic tuner adopting a three-tier progressive adjustment strategy. The tuner aims to establish a closed-loop control process of prediction-execution-feedback-readjustment, so as to mitigate adverse effects caused by prediction errors, system noise, and measurement bias. The complete workflow is presented in Algorithm 1.

Load-Aware Fast Prediction. During each execution cycle, the system automatically records the average service time of the current task queue tcur using probe pointer and computes the average processing time of the queue Tcur (lines 1-3). When a change in tcur exceeding a preset threshold is detected, the tuner immediately invokes PINO to predict the optimal WT count and update it (lines 4-6). This mechanism ensures fast responsiveness to sudden load changes.

CPU-Aware Periodic Adjustment. To avoid continuous prediction overhead and handle gradually varying loads, a periodic checking mechanism is introduced. The check interval is dynamically adaptive: if prediction was triggered in the previous cycle, a short interval C1 is used to verify and correct predictions quickly; otherwise, a longer interval C2 is adopted to reduce monitoring overhead (line 7). At a checkpoint, if the observed system cpu exceeds the usr cpu, the algorithm proactively reduces the number of WTs to mitigate contention (lines 8-16). This mechanism effectively maps CPU indicators of resource contention into direct signals for thread adjustment, thereby preventing performance degradation due to oversubscription.

Throughput-Driven Fine-tuning. When the system is detected to be in a high-contention state, the tuner switches to a fine-tuning phase. In this phase, throughput of the task queue is directly optimized as the adjustment objective. If reducing threads leads to improved throughput, the adjustment is accepted and further attempts are made; otherwise, the change is rolled back and the phase is terminated (lines 17-25). Considering that undersubscription has a less detrimental effect on performance than the severe contention caused by oversubscription, the fine-tuning phase only considers reducing threads. Moreover, to avoid erroneous adjustments due to random fluctuations or transient anomalies, the tuner exits this phase only if throughput fails to improve over three consecutive simulation steps or if the thread count reaches the minimum allowed value (line 20).

4 Evaluation

In this section, we first introduce the experimental setup and current thread tuning methods. Subsequently, we evaluate the proposed method against these baselines in terms of system acceleration performance and adaptability. Finally, we conduct ablation studies to quantify the contributions of each optimization module and verify the robustness of our method with respect to various hyperparameters.

4.1 Setup

The training set is derived from averaged statistics over the entire execution under constant-load scenarios, where the

Algorithm 1 Three-Level Progressive Thread Adjustment

Input: δ (load change threshold), C1 (short period), C2 (long period) Output: WT count k for each simulation step 1: while simulation not finished do 2: tcur ← PROBETASKTIME() 3: Tcur ← EXECUTECYCLE() 4: if |tcur - tprev|/(tprev + ϵ) > δ and tuning = 0 then 5: k ← PINO-PREDICT 6: predicted ← TRUE 7: else if predicted and (tick mod C1 = 0) then 8: (system cpu, usr cpu) ← GETCPUMETRICS() 9: if usr cpu < system cpu then 10: k ← k -1, tuning ← 1 11: end if 12: predicted ← FALSE 13: else if ¬predicted and (tick mod C2 = 0) then 14: if usr cpu < usr cpu then 15: k ← k -1, tuning ← 1 16: end if 17: else if tuning ≥ 1 and k > kmin then 18: if Tcur < Tprev then ▷ Throughput improved 19: k ← k -1, tuning ← 1 20: else if tuning < 3 then ▷ Tolerate fluctuation 21: tuning ← tuning + 1 22: else ▷ Rollback and exit tuning 23: k ← k +1, tuning ← 0 24: end if 25: end if 26: tprev ← tcur, Tprev ← Tcur 27: SYNCHRONIZE(); NEXTTICK() 28: end while

execution time of each task remains unchanged throughout the simulation horizon. We split the dataset into training/testing/validation sets with a ratio of 0.7/0.2/0.1 to offline train different predictors. During training, each feature vector is taken as the average over the whole execution; since the load is constant, the optimal thread count label for each sample can be obtained by sweeping different thread counts and recording the total execution time.

In the testing phase, we use dynamic workloads to evaluate the speedup and adaptivity of each method. Specifically, within each simulation step, task execution times follow a Poisson distribution parameterized by t, while the parameter t varies across steps as a square wave with frequency roh (switching among [t, 1.5t, 2t]). Each method adjusts based on instantaneous values of the feature variables at a single time point, so as to emulate a practical deployment setting with offline training and online testing.

Experiments are conducted on two multicore architectures: AMD EPYC 9734 (112 Cores/224 Threads) and Intel Xeon Gold 6338 (32 Cores/64 Threads). The baselines include ADAPT-T[Costa et al., 2019] based on runtime feedback control; Otter[Luan et al., 2022] employing search strategies; Thread Reinforcer[Pusukuri et al., 2011] utilizing system resource utilization; Xgboost[Akash et al., 2021] as an advanced representative of machine learning-based prediction.

Additionally, a fixed thread count configuration was used as a control experiment, denoted as Static. All methods were implemented on the Repast HPC system[Zhu et al., 2017]. Each test case was executed at least five times to obtain optimal measurement data.

4.2 Performance Experiments

We run the UAV benchmark on the AMD platform with the execution horizon set to 5000 and the load threshold δ = 0.3, and record the total runtime from end to-end under different thread adjustment schemes; the results are shown in Fig. 7. Compared to ADAPT-T, AutoThread saves up to 83.8% of runtime in different scenarios. Over 20 test scenarios, AutoThread achieves an average runtime of 149.32 s and delivers 10.3% and 17.2% higher speedup than the state-ofthe-art Otter and Reinforcer, respectively. Notably, we set the thread count of the fixed strategy to 20, i.e., the average output of all tuning schemes. Even when accounting for the additional overhead introduced by frequent thread adjustments, AutoThread still attains, on average, an 18.4% higher speedup than the fixed strategy with prior knowledge, with a maximum performance improvement of 62.6%.

4.3 Adaptiveness Experiments

We run the PCS and UAV applications on both Intel and AMD platforms under different load-variation intensities (frequency roh, amplitude t) , and record the system throughput over environment steps to evaluate the adaptability of different schemes, where throughput denotes the number of tasks processed per millisecond. Figure 8 shows that, in practical scenarios, when competing methods successively exhibit non-convergent oscillations and extrapolation failures, AutoThread still maintains strong parallel speedup. On the PCS case, AutoThread attains an average throughput of 536.83, improving over ADTPT-T and Xgboost by 37.7% and 2.3%, respectively, while remaining below Reinforcer's 718.55. Under the UAV case with larger workload fluctuations, AutoThread's average throughput is 1.8× and 1.7× that of Reinforcer and Xgboost, respectively.

4.4 Ablation Experiments

We ablate the PINO predictor and the Adaptive Dynamic Tuner to investigate the performance contribution of each component. The compared variants include: using only the physics-informed neural operator network prediction (PINO); using the dynamic tuner with the maximum thread count (Dyn); applying the Dyn tuner on top of the Xgboost output (Xgboost+Dyn); and pairing the ADAPT-T tuner with the output of PINO (ADAPT+PINO). Figure 9 reports the average throughput of different schemes when running the UAV application on the ADM platform. The results show that both the PINO predictor and the Adaptive Dynamic Tuner effectively improve runtime performance. Across 10 test scenarios, the Dyn technique increases the task processing speed of Xgboost by 87.8% on average, with a maximum improvement of 55.2×. PINO increases the task processing speed of the ADAPT-T tuner by 3.9× on average. Moreover, AutoThread improves average throughput by 1.3× and 2.0× over Xgboost+Dyn and ADAPT+PINO, respectively, demonstrating the combined benefits of the two techniques.

4.5 Robustness Experiment

We further examine the impact of parameters δ, C1 and C2 on AutoThread's performance. The parameter δ represents the maximum tolerable degree of workload change for thread adjustment. An overly small value leads to frequent thread changes due to random hardware jitter, whereas an overly large value prevents timely adaptation to workload changes. AutoThread's robustness to δ is shown in Figure 10. Benefiting from the theoretical constraints of the physical model and the error-correction capability of the dynamic tuner, AutoThread provides stable speedup even under workload inputs with substantial randomness. In contrast, an excessively large δ weakens the predictor's sensitivity to workload changes, resulting in performance degradation.

Figure 10 shows the impact of different C1/C2 combinations on system performance. We find that a smaller C1 enables faster response; however, an overly small C2 causes frequent periodic checks and increases system overhead, while an overly large C2 may cause the system to miss critical tuning opportunities. The optimal ratio is approximately 2-4, within which short-cycle validation and long-cycle monitoring achieve a good balance-responding quickly to workload changes while avoiding excessive monitoring overhead.

4.6 Discussion

The experimental results clearly demonstrate the advantages of AutoThread in sustaining parallel speedup and substantially reducing execution time, particularly in environments with fluctuating workloads. The key strength of AutoThread lies in the deep integration of data-driven representational capacity with the structural constraints imposed by physical models: on the one hand, it learns latent prior patterns from runtime data to enable rapid prediction under dynamic workloads; on the other hand, by introducing interpretable theoretical models as inductive biases and training constraints, it anchors the prediction process in consistent mechanistic logic, thereby significantly improving robustness and the achievable performance upper bound while preserving interpretability and transferability. This form of mechanism-guided learning coupled with data-corrected inference is central to AutoThread's superior effectiveness.

Specifically, first, guided by a performance model, PINO enforces queueing-theoretic constraints on the neural network's predictive outputs, endowing the predictions with strong generalization and physical plausibility; consequently, it can provide reliable initial estimates of thread counts even under workload conditions whose distributions differ from those seen during training. Second, the dynamic tuning mechanism employs a three-level strategy to perform online corrections to the predicted results, while accounting for the overhead of monitoring and adjustment. This enables the system to rapidly converge to a near-optimal thread configuration when the workload changes, thereby accelerating the simulation environment's runtime efficiency.

5 RelatedWork

Many WT tuning methods have been proposed across domains; however, compared with this study, they insufficiently address dynamic workload fluctuations or generalizability. Static modeling methods set fixed thread counts through empirical heuristics or offline analysis, such as using a multiple of physical cores[Montesano et al., 2024] or estimating optimum from the ratio of thread creation to maintenance overhead [Luo et al., 2025]. Although simple and efficient, these strategies struggle in complex, volatile environments because they cannot adjust to workload fluctuations in real time.

Machine learning has enabled data-driven threadconfiguration prediction, including Reinforcement Learning[Xu et al., 2012], Ensemble Learning [Chao et al., 2025], and online search optimization[Stetsenko and Dyfuchyna, 2019], with applications in large-scale cluster management. However, these methods often rely on highquality training data; under dynamic scenarios absent from training, pure data-driven models may generalize poorly. Another common category uses adaptive regulation based on online feedback control [Albalawi, 2025]. These methods incrementally adjust WT count, one thread at a time, using metrics such as request arrival rates and service times until performance no longer improves. Although practical for cloud services, such feedback-driven regulation often suffers from slow convergence, latency, and performance oscillations in large thread spaces or under frequent workload variations [Bahadur et al., 2025].

6 Conclusion

This paper proposes AutoThread, a thread-count tuning method for ABS systems to accelerate RL inference. The core contributions are as follows: (1) we open-source a multithreaded trajectory dataset collected from a DES simulation environment, and empirically derive three key insights; (2) we design a PINO-based thread predictor and establish a theoretical model relating the number of WTs to runtime, which is used to constrain and guide model training; and (3) we develop an efficient hybrid thread adjustment method that incorporates a load-aware adaptive tuning strategy on top of PINO's predictions. Experimental results show that AutoThread delivers excellent parallel speedup and strong workload adaptivity. The proposed approach is portable and can be applied to accelerate other systems based on a shared event queue. Future work will explore batch-sample thread tuning methods for accelerating RL training.

Acknowledgments

We thank the anonymous IJCAI reviewers for their insightful comments. This work was partially supported by the National Natural Science Foundation of China (NSFC) under Grant No.62503491.

Contribution Statement

Jiming Su and Hantao Hua contributed equally to this work. Feng Zhu is the corresponding author.