Publications | Minglai Yang

2026

TMLR
AlignSAE: Concept-Aligned Sparse Autoencoders

Minglai Yang^*, Xinyu Guo, Zhengliang Shi, Jinhe Bi, Steven Bethard, Mihai Surdeanu^*, and Liangming Pan^*

Transactions on Machine Learning Research (TMLR), 2026

Abs arXiv Bib HTML Code Slides Website

Large Language Models (LLMs) encode factual knowledge within hidden parametric spaces that are difficult to inspect or control. While Sparse Autoencoders (SAEs) can decompose hidden activations into more fine-grained, interpretable features, they often struggle to reliably align these features with human-defined concepts, resulting in entangled and distributed feature representations. To address this, we introduce AlignSAE, a method that aligns SAE features with a predefined ontology through a "pre-train, then post-train" curriculum. After an initial unsupervised training phase, we apply supervised post-training to bind specific concepts to dedicated latent slots while preserving the remaining capacity for general reconstruction. This separation creates an interpretable interface where specific concepts can be inspected and controlled without interference from unrelated features. Empirical results demonstrate that AlignSAE enables precise causal interventions, such as reliable "concept swaps", by targeting single, semantically aligned slots, and further supports multi-hop reasoning and a mechanistic probe of grokking-like generalization dynamics.
@article{yang2025alignsaeconceptalignedsparseautoencoders, title = {AlignSAE: Concept-Aligned Sparse Autoencoders}, author = {Yang, Minglai and Guo, Xinyu and Shi, Zhengliang and Bi, Jinhe and Bethard, Steven and Surdeanu, Mihai and Pan, Liangming}, journal = {Transactions on Machine Learning Research (TMLR)}, year = {2026}, eprint = {2512.02004}, archiveprefix = {arXiv}, primaryclass = {cs.LG}, url = {https://arxiv.org/abs/2512.02004}, google_scholar_id = {Tyk-4Ss8FVUC}, }
ArXiv

Peeking inside the Black-Box: Reinforcement Learning for Explainable and Accurate Relation Extraction

Xinyu Guo, Zhengliang Shi, Minglai Yang, Mahdi Rahimi, and Mihai Surdeanu

In Submission to ICLR , 2026

Abs arXiv PDF Code

This paper introduces a framework for relation extraction (RE) that enhances both accuracy and explainability. The framework has two key components: (i) a reasoning mechanism that formulates relation extraction as a series of text-processing steps inspired by cognitive science, and (ii) an optimization process driven by reinforcement learning (RL) with a novel reward function designed to improve both task accuracy and explanation quality. We call our approach CogRE. Our framework addresses the lack of supervision for language-based explanations in traditional RE by promoting outputs that include important relation keywords. These keywords are drawn from a high-quality dictionary that is automatically constructed using an LLM. We evaluate our approach for the task of one-shot RE using two LLMs and two RE datasets. Our experiments show that CogRE improves explanation quality by addressing two common failure patterns in one-shot RE: poor attention focus and limited one-shot learning capability. For example, our cognitive-structured reasoning with Qwen2.5-15B-Instruct on One-shot NYT29 achieves 24.65% F1, surpassing prior reasoning-based designs. Optimizing this approach with RL using our reward further improves performance by +23.46% (absolute). Finally, human evaluation shows that our best model generates relational keywords closely aligned with gold labels, increasing human explanation quality ratings by 54% (relative).
ArXiv
VEFX-Bench: A Holistic Benchmark for Generic Video Editing and Visual Effects

Xiangbo Gao, Sicong Jiang, Bangya Liu, Xinghao Chen, Minglai Yang, Siyuan Yang, Mingyang Wu, Jiongze Yu, Qi Zheng , Haozhi Wang , and 5 more authors

In Submission to COLM , 2026

Abs arXiv Bib

As AI-assisted video creation becomes increasingly practical, instruction-guided video editing has become essential for refining generated or captured footage to meet professional requirements. Yet the field still lacks both a large-scale human-annotated dataset with complete editing examples and a standardized evaluator for comparing editing systems. We introduce VEFX-Dataset, a human-annotated dataset containing 5,049 video editing examples across 9 major editing categories and 32 subcategories, each labeled along three decoupled dimensions: Instruction Following, Rendering Quality, and Edit Exclusivity. Building on VEFX-Dataset, we propose VEFX-Reward, a reward model designed specifically for video editing quality assessment. We further release VEFX-Bench, a benchmark of 300 curated video-prompt pairs for standardized comparison of editing systems. Experiments show that VEFX-Reward aligns more strongly with human judgments than generic VLM judges and prior reward models.
@misc{gao2026vefxbench, title = {VEFX-Bench: A Holistic Benchmark for Generic Video Editing and Visual Effects}, author = {Gao, Xiangbo and Jiang, Sicong and Liu, Bangya and Chen, Xinghao and Yang, Minglai and Yang, Siyuan and Wu, Mingyang and Yu, Jiongze and Zheng, Qi and Wang, Haozhi and Zhang, Jiayi and Yang, Jie and Wang, Zihan and Yin, Qing and Tu, Zhengzhong}, year = {2026}, eprint = {2604.16272}, archiveprefix = {arXiv}, primaryclass = {cs.CV}, url = {https://arxiv.org/abs/2604.16272}, }
ArXiv
Justified or Just Convincing? Error Verifiability as a Dimension of LLM Quality

Xiaoyuan Zhu, Kimberly Le Truong, Riccardo Fogliato, Gokul Swamy , Weijian Zhang, Minglai Yang, Longtian Ye, Bangya Liu, Minghao Liu, Andrew Ilyas , and 1 more author

In Submission to COLM , 2026

Abs arXiv Bib

As LLMs are deployed in high-stakes settings, users must judge the correctness of individual responses, often relying on model-generated justifications such as reasoning chains or explanations. Yet, no standard measure exists for whether these justifications help users distinguish correct answers from incorrect ones. We formalize this idea as error verifiability and propose v_bal, a balanced metric that measures whether justifications enable raters to accurately assess answer correctness, validated against human raters who show high agreement. We find that neither common approaches, such as post-training and model scaling, nor more targeted interventions improve verifiability. We introduce two methods that succeed at improving verifiability: reflect-and-rephrase (RR) for mathematical reasoning and oracle-rephrase (OR) for factual QA. Together, our results establish error verifiability as a distinct dimension of response quality that does not emerge from accuracy improvements alone.
@misc{zhu2026justified, title = {Justified or Just Convincing? Error Verifiability as a Dimension of LLM Quality}, author = {Zhu, Xiaoyuan and Truong, Kimberly Le and Fogliato, Riccardo and Swamy, Gokul and Zhang, Weijian and Yang, Minglai and Ye, Longtian and Liu, Bangya and Liu, Minghao and Ilyas, Andrew and Wu, Steven}, year = {2026}, eprint = {2604.04418}, archiveprefix = {arXiv}, primaryclass = {cs.CL}, url = {https://arxiv.org/abs/2604.04418}, }
ArXiv
Dr. DocBench: A Comprehensive Benchmark for Expert-Level and Difficult Document Parsing

Minglai Yang^*, Xinyan Velocity Yu^*, Pengyuan Li, Xinyu Guo, Zhenting Qi, Konwoo Kim, Longtian Ye, Xiaolong Luo, Jinhe Bi , Henry Zhang , and 15 more authors

In Submission to EMNLP , 2026

Abs arXiv Bib

Document parsing and recognition are fundamental capabilities for vision-language models (VLMs) and document processing systems. However, existing Optical Character Recognition (OCR) and document parsing benchmarks are increasingly limited in coverage and difficulty: many focus on common document genres or uniformly sampled pages where modern parsers already perform strongly. Dr. DocBench provides expert-level, difficult document parsing evaluation with thousands of annotated pages from long documents, spanning 52 BISAC subject domains, selecting challenging documents through parser-failure-based sampling.
@misc{yang2026drdocbench, title = {Dr. DocBench: A Comprehensive Benchmark for Expert-Level and Difficult Document Parsing}, author = {Yang, Minglai and Yu, Xinyan Velocity and Li, Pengyuan and Guo, Xinyu and Qi, Zhenting and Kim, Konwoo and Ye, Longtian and Luo, Xiaolong and Bi, Jinhe and Zhang, Henry and Riaz, Haris and Zhang, Xuan and Xiao, Yunze and Liu, Bangya and Tang, Tom and Zhao, Yunfei and Lin, Qunshu and Wang, Zihan and Liu, Minghao and Li, Michael Lingzhi and Du, Yilun and Thomason, Jesse and Feris, Rogerio and Pentland, Alex and He, Zexue}, year = {2026}, eprint = {2606.01393}, archiveprefix = {arXiv}, primaryclass = {cs.CL}, url = {https://arxiv.org/abs/2606.01393}, }
ICML
EchoRL: Reinforcement Learning via Rollout Echoing

Jinhe Bi, Aniri, Minglai Yang, Xingcheng Zhou, Wenke Huang, Sikuan Yan , Yujun Wang, Zixuan Cao, Michael Färber, Xun Xiao , and 2 more authors

ICML 2026 , 2026

Abs arXiv Bib

Reinforcement Learning with Verifiable Rewards is an effective route for post-training to strengthen the reasoning capability of large language models. However, as training proceeds, the learning signal can collapse thus makes the training gain become marginal and ineffective. Specifically, a growing fraction of prompts’ rollouts become advantage-degenerated: all the self-generated rollouts show vanishing advantage. EchoRL revives these prompts via rollout echoing.
@article{bi2026echorl, title = {EchoRL: Reinforcement Learning via Rollout Echoing}, author = {Bi, Jinhe and Aniri and Yang, Minglai and Zhou, Xingcheng and Huang, Wenke and Yan, Sikuan and Wang, Yujun and Cao, Zixuan and Färber, Michael and Xiao, Xun and Tresp, Volker and Ma, Yunpu}, year = {2026}, eprint = {2605.31228}, archiveprefix = {arXiv}, primaryclass = {cs.LG}, url = {https://arxiv.org/abs/2605.31228}, }
ArXiv
Opening the Black Box: A Survey on the Mechanisms of Multi-Step Reasoning in Large Language Models

Liangming Pan, Jason Liang, Jiaran Ye, Minglai Yang, Xinyuan Lu, and Fengbin Zhu

2026

Abs arXiv Bib

Large Language Models (LLMs) have demonstrated remarkable abilities to solve problems requiring multiple reasoning steps, yet the internal mechanisms enabling such capabilities remain elusive. Unlike existing surveys that primarily focus on engineering methods to enhance performance, this survey provides a comprehensive overview of the mechanisms underlying LLM multi-step reasoning.
@misc{pan2026openingblackbox, title = {Opening the Black Box: A Survey on the Mechanisms of Multi-Step Reasoning in Large Language Models}, author = {Pan, Liangming and Liang, Jason and Ye, Jiaran and Yang, Minglai and Lu, Xinyuan and Zhu, Fengbin}, year = {2026}, eprint = {2601.14270}, archiveprefix = {arXiv}, primaryclass = {cs.CL}, url = {https://arxiv.org/abs/2601.14270}, }

ArXiv

Knowledge Index of Noah’s Ark

Sheng Jin, Minghao Liu, Yunze Xiao, Zeqi Zhou, Heli Qi, Yifan Yao, Meishu Song, Kaijing Ma , Xuan Zhang, Sicong Jiang , and 17 more authors

2026

Abs arXiv Bib

Knowledge benchmarks for LLMs face three issues: scaling-driven designs that do not operationalize disciplinary representativeness; flat-payment annotation that permits lazy consensus; and unaudited ranking instability under bounded test budgets. We introduce KINA, an 899-item benchmark across 261 fine-grained disciplines.

@misc{jin2026kina,
  title = {Knowledge Index of Noah's Ark},
  author = {Jin, Sheng and Liu, Minghao and Xiao, Yunze and Zhou, Zeqi and Qi, Heli and Yao, Yifan and Song, Meishu and Ma, Kaijing and Zhang, Xuan and Jiang, Sicong and Li, Yizhe and Ma, Ningshan and Wei, Jie and Li, Ziniu and Yang, Minglai and Liu, Bangya and Liang, Yiming and Fang, Xiao and Zeng, Qingcheng and Liu, Jiarui and Yang, Rui and Yan, Shen and Huang, Wenhao and Liu, Jiaheng and Wang, Zihan and Xuan, Weihao and Zhang, Ge},
  year = {2026},
  eprint = {2606.05104},
  archiveprefix = {arXiv},
  primaryclass = {cs.CL},
  url = {https://arxiv.org/abs/2606.05104},
}

ArXiv
Triaging Threats to Specialized Guardrails

Wenjie Jacky Mo, Xiaofei Wen, Rui Cai, Boyu Zhu, Sicong Jiang , Zihan Wang, Minglai Yang, Zhe Zhao, and Muhao Chen

2026

Abs arXiv Bib

Building robust safety guardrails is essential for deploying Large Language Models across diverse real-world applications. However, this goal remains challenging because safety risks span heterogeneous threat domains, while existing datasets cover only fragmented risk subsets and rely on inconsistent taxonomies.
@misc{mo2026triaging, title = {Triaging Threats to Specialized Guardrails}, author = {Mo, Wenjie Jacky and Wen, Xiaofei and Cai, Rui and Zhu, Boyu and Jiang, Sicong and Wang, Zihan and Yang, Minglai and Zhao, Zhe and Chen, Muhao}, year = {2026}, eprint = {2605.30693}, archiveprefix = {arXiv}, primaryclass = {cs.CL}, url = {https://arxiv.org/abs/2605.30693}, }

2025

EMNLP
How Is LLM Reasoning Distracted by Irrelevant Context? An Analysis Using a Controlled Benchmark

Minglai Yang^*, Ethan Huang , Liang Zhang, Mihai Surdeanu, William Wang, and Liangming Pan^*

Oral Presentation

EMNLP Main Conference , 2025

Abs arXiv Bib PDF Code Poster Slides Website

We introduce Grade School Math with Distracting Context (GSM-DC), a synthetic benchmark to evaluate Large Language Models’ (LLMs) reasoning robustness against systematically controlled irrelevant context (IC). GSM-DC constructs symbolic reasoning graphs with precise distractor injections, enabling rigorous, reproducible evaluation. Our experiments demonstrate that LLMs are significantly sensitive to IC, affecting both reasoning path selection and arithmetic accuracy. Additionally, training models with strong distractors improves performance in both in-distribution and out-of-distribution scenarios. We further propose a stepwise tree search guided by a process reward model, which notably enhances robustness in out-of-distribution conditions.
@article{yang2025llmreasoningdistractedirrelevant, title = {How Is LLM Reasoning Distracted by Irrelevant Context? An Analysis Using a Controlled Benchmark}, author = {Yang, Minglai and Huang, Ethan and Zhang, Liang and Surdeanu, Mihai and Wang, William and Pan, Liangming}, year = {2025}, eprint = {2505.18761}, archiveprefix = {arXiv}, primaryclass = {cs.CL}, url = {https://arxiv.org/pdf/2505.18761}, google_scholar_id = {UeHWp8X0CEIC}, }
EMNLP
CopySpec: Accelerating LLMs with Speculative Copy-and-Paste Without Compromising Quality

Razvan-Gabriel Dumitru, Minglai Yang, Vikas Yadav, and Mihai Surdeanu

Oral Presentation

EMNLP Main Conference , 2025

Abs arXiv Bib PDF Code

We introduce CopySpec, a simple yet effective technique to tackle the inefficiencies LLMs face when generating responses that closely resemble previous outputs or responses that can be verbatim extracted from context. CopySpec identifies repeated sequences in the model’s chat history or context and speculates that the same tokens will follow, enabling seamless copying without compromising output quality and without requiring additional GPU memory. To evaluate the effectiveness of our approach, we conducted experiments using seven LLMs and five datasets: MT-Bench, CNN/DM, GSM8K, HumanEval, and our newly created dataset, MT-Redundant. MT-Redundant, introduced in this paper, transforms the second turn of MT-Bench into a request for variations of the first turn’s answer, simulating real-world scenarios where users request modifications to prior responses. Our results demonstrate significant speed-ups: up to 2.35x on CNN/DM, 3.08x on the second turn of select MT-Redundant categories, and 2.66x on the third turn of GSM8K’s self-correction tasks. Importantly, we show that CopySpec integrates seamlessly with speculative decoding, yielding an average 49% additional speed-up over speculative decoding for the second turn of MT-Redundant across all eight categories. While LLMs, even with speculative decoding, suffer from slower inference as context size grows, CopySpec leverages larger contexts to accelerate inference, making it a faster complementary solution. Our code and dataset are publicly available at https://github.com/razvandu/copyspec.
@article{dumitru2025copyspecacceleratingllmsspeculative, title = {CopySpec: Accelerating LLMs with Speculative Copy-and-Paste Without Compromising Quality}, author = {Dumitru, Razvan-Gabriel and Yang, Minglai and Yadav, Vikas and Surdeanu, Mihai}, year = {2025}, eprint = {2502.08923}, archiveprefix = {arXiv}, primaryclass = {cs.CL}, url = {https://arxiv.org/abs/2502.08923}, google_scholar_id = {u-x6o8ySG0sC}, }
Workshop @ ICML
Improving the Data-efficiency of Reinforcement Learning by Warm-starting with LLM

Thang Duong, Minglai Yang, and Chicheng Zhang

Exploration in AI Today (EXAIT) Workshop @ ICML , 2025

Abs arXiv Bib PDF Code

We investigate the usage of Large Language Model (LLM) in collecting high-quality data to warm-start Reinforcement Learning (RL) algorithms for learning in some classical Markov Decision Process (MDP) environments. In this work, we focus on using LLM to generate an off-policy dataset that sufficiently covers state-actions visited by optimal policies, then later using an RL algorithm to explore the environment and improve the policy suggested by the LLM. Our algorithm, LORO, can both converge to an optimal policy and have a high sample efficiency thanks to the LLM’s good starting policy. On multiple OpenAI Gym environments, such as CartPole and Pendulum, we empirically demonstrate that LORO outperforms baseline algorithms such as pure LLM-based policies, pure RL, and a naive combination of the two, achieving up to 4\times the cumulative rewards of the pure RL baseline.
@article{duong2025improvingdataefficiencyreinforcementlearning, title = {Improving the Data-efficiency of Reinforcement Learning by Warm-starting with LLM}, author = {Duong, Thang and Yang, Minglai and Zhang, Chicheng}, year = {2025}, eprint = {2505.10861}, archiveprefix = {arXiv}, primaryclass = {cs.LG}, url = {https://arxiv.org/abs/2505.10861}, google_scholar_id = {d1gkVwhDpl0C}, }
ArXiv
Word2VecGD: Neural Graph Drawing with Cosine-Stress Optimization

Minglai Yang, and Reyan Ahmed

2025

Abs arXiv Bib

We propose a novel graph visualization method leveraging random walk-based embeddings to replace costly graph-theoretical distance computations. Using word2vec-inspired embeddings, our approach captures both structural and semantic relationships efficiently. Instead of relying on exact shortest-path distances, we optimize layouts using cosine dissimilarities, significantly reducing computational overhead.
@misc{yang2025word2vecgd, title = {Word2VecGD: Neural Graph Drawing with Cosine-Stress Optimization}, author = {Yang, Minglai and Ahmed, Reyan}, year = {2025}, eprint = {2509.17333}, archiveprefix = {arXiv}, primaryclass = {cs.LG}, url = {https://arxiv.org/abs/2509.17333}, }

2024

MCM

Dynamic Balances: Modelling Variable Sex Ratios in Lamprey Populations for Ecosystem Management

M. Yang, M. Abbasi, and S. Soporboev

Best Paper at University of Arizona Math Modeling

International Mathematical Contest in Modeling (MCM)., Tucson, Arizona. ** The Problem can be found here , May 2024

Abs PDF

We developped an Adaptive Population Control (ACC) model that explains the skewed sex ratio at high and low levels of resource availability as a control measure by lamprey to maximize their growth rate. We find that the advantage of a variable sex ratio is greatest under extremely poor or extremely suitable environmental conditions. Next, to account for the male skewed populations of lamprey at high densities, we develop a modified logistic growth model that allows for variable carrying capacity for males and females. Differential Carrying Capacity (DCC) model accurately predicts the optimal sex ratio to achieve the maximal growth rate at various population densities. In line with ACC, we use Aquatic Nutritional Network (ANN), a niche width differential equation model is developed by integrating concepts from Lotka-Volterra equations. to study the dynamics of lamprey and their native prey/predator. We find that variable sex ratios allow time for regeneration of the native prey in Lamprey.