Learning Computational Patterns — Arithmetics, Algebra, and More

From the abacus to the calculator to computer algebra systems (CAS), each generation of computational tools has expanded the range of computation humans can practically carry out. Does deep learning — Transformers in particular — open a genuinely new chapter in this history?

We study whether Transformers can learn to perform computational tasks that are notoriously hard for classical algorithms — from symbolic computer algebra, such as Gröbner bases and border bases, to arithmetic patterns like modular addition and input-sensitive functions such as QR code decoding. Along the way we also study how a model should learn: the order in which it processes or generates a computation can itself make a large difference to learnability.

Progress so far

We presented the first end-to-end attempt at learning to compute Gröbner bases at NeurIPS 2024, and have continued the line of work with attention-based border basis computation (NeurIPS 2025) and CALT, a library bridging computer algebra and Transformers (ISSAC 2025). More recently, we’ve extended this direction to arithmetic pattern learning: large-scale modular addition, input-sensitivity in QR code decoding, and learning-friendly orderings for arithmetic and sequential computation (ICML 2025/2026, AI for Math Workshop).

Hanato Kikuchi and collaborators’ “Learning Large-Scale Modular Addition with an Auxiliary Modulus” was accepted at NeurIPS 2026. The work studies learning large-scale modular addition with an auxiliary modulus.