Why TabularMath Matters
Existing math reasoning benchmarks still under-cover realistic table use: larger structured contexts, flawed tables, and the split between text and visual table inputs.
Mathematical reasoning is widely used to evaluate large language models, but table-based reasoning remains comparatively underexplored. In practice, systems must retrieve evidence from larger tables, remain robust when tables are incomplete or inconsistent, and generalize across both text-rendered and visually rendered inputs. TabularMath is introduced to make these pressures explicit rather than hiding them behind average-case scores.
AutoT2T: Automated Text-to-Table Generation
AutoT2T converts math word problems into validated tabular reasoning tasks without manual table annotation.
AutoT2T first formalizes the problem, then transforms it into a seed table, and finally applies controllable augmentations for difficulty and robustness testing.
Semantic Decoupling
Large language models extract variables, constraints, and goals from natural-language math problems, while formal solvers verify satisfiability and consistency.
Table Transformation
The verified formal state is rewritten into a structured seed table with explicit fields, entity rows, and tabularized assignments.
Table Augmentation
Row augmentation, column augmentation, order shuffling, and information modification create controllable splits for complexity, retrieval stress, and robustness.
A Structured Testbed for Table Reasoning
The benchmark is built from transformed GSM8K-style problems and covers both text-based and rendered image-based tables.
Compared with prior math and table reasoning datasets, TabularMath emphasizes larger tables, coupled retrieval-and-reasoning, and robustness to flawed inputs.
The paper names the benchmark TabularMath. The released Hugging Face dataset is available as TabularGSM, covering easy, medium, hard, and imperfect settings for controlled evaluation.
Low variation with simpler retrieval.
Shuffled tables with harder localization.
Larger tables with row and column expansion.
Missing or contradictory tables for robustness.
Three Core Experimental Messages
We analyze TabularMath from three main angles: table size and structural complexity, table quality, and modality-sensitive representation.
Table Size and Structural Complexity Drive Difficulty
Performance on top-complexity questions drops much more sharply than on average samples, showing that larger and more structurally demanding tables remain a major bottleneck.
Table Quality Matters, Especially for Imperfect Inputs
Informing models that a table may be problematic improves discrimination, but also hurts accuracy on ordinary well-defined problems, revealing a genuine robustness trade-off.
Modality and Representation Change the Difficulty Profile
Text-rendered tables remain easier than visual ones in general, and within text inputs, structured formats such as JSON or serialization stay more reliable than Markdown.
Retrieval Is Not the Same as End-to-End Reasoning
When the retrieval target is made explicit, models perform much better. The harder part is deciding what to retrieve while reasoning through the full problem.
Code Helps Less Than Clear Retrieval Guidance
Writing code does not automatically solve table reasoning; strong gains appear when the model is explicitly guided toward the right retrieval trajectory.
BibTeX
The citation below follows the current arXiv preprint metadata for
arXiv:2505.19563.
@article{tian2025tabularmath,
title = {TabularMath: Understanding Math Reasoning over Tables with Large Language Models},
author = {Tian, Shi-Yu and Zhou, Zhi and Dong, Wei and Yu, Kun-Yang and Yang, Ming and Cheng, Zi-Jian and Guo, Lan-Zhe and Li, Yu-Feng},
journal = {arXiv preprint arXiv:2505.19563},
year = {2025},
doi = {10.48550/arXiv.2505.19563},
url = {https://arxiv.org/abs/2505.19563}
}