Monthly AI Papers: April 2025

Contents

Trends
Top Papers in April 2025
Sources

Trends

April 2025’s theme is how computation is organized: the three selected papers explore numerical precision, selective parameter use, and executable generation procedures to improve model efficiency or task performance. BitNet trains with ternary weights, restricted to {−1, 0, +1}; Kimi-VL uses a Mixture-of-Experts decoder, which routes tokens through selected expert modules; Self-Steering Language Models generates programs that coordinate smaller models. Their results measure different outcomes, so memory savings, activated parameter counts, and constraint satisfaction require separate interpretation. 1, 2, 3

Additional inference computation produces task-dependent gains. Kimi-VL’s thinking extension generates explicit intermediate reasoning, with substantial improvements on visual mathematics but little additional MathVista improvement beyond a 4k-token thinking budget. Self-Steering Language Models instead allocates computation across candidate generations through executable search and scoring procedures. It improves exact constraint satisfaction, although additional candidates can worsen results when the generated scoring procedure penalizes required content.

The evidence also makes system-level measurement central to assessing efficiency. BitNet reports measured laptop decoding latency, but its weight-memory figure excludes embeddings, the learned token representations, and its energy result estimates arithmetic costs. Kimi-VL’s activated parameter count does not establish runtime memory or latency. In Self-Steering Language Models, prompting the larger planning model dominates estimated cost despite inexpensive execution by the smaller model. These findings support evaluating the complete deployment under its intended hardware, input, and output conditions.

Top Papers in April 2025

1. BitNet b1.58 2B4T Technical Report

https://arxiv.org/abs/2504.12285
BitNet b1.58 2B4T Technical Report | Summary
BitNet b1.58 2B4T Technical Report presents a 2-billion-parameter language model trained from scratch on 4 trillion tokens with ternary weights and 8-bit integer activations, the intermediate values passed between layers. Its 1.58-bit designation describes the weights rather than every training or inference value. Specialized BitLinear layers implement the quantized computation.
Table 1 reports an average benchmark score of 54.19 versus 55.23 for instruction-tuned Qwen2.5 1.5B, with 0.4 GB versus 2.6 GB of weight memory excluding embeddings. Decoding takes 29 ms versus 65 ms per output token on a Surface Laptop Studio 2 with an Intel Core i7-13800H and 8 central processing unit threads, averaged over 128 generated tokens. BitNet uses bitnet.cpp, while the competitors use llama.cpp.
The deployment result is promising within the tested small-model comparisons, but performance varies by task: BitNet scores 58.38 on GSM8K mathematics with four-example prompting, while trailing Qwen2.5 on MMLU knowledge evaluation and HumanEval+ code generation. Its reported 0.028 J energy figure estimates matrix-multiplication arithmetic at a sequence length of 512 tokens; it does not measure whole-device energy. Unequal training recipes and separate inference libraries limit attribution of the gains solely to weight precision.

2. Kimi-VL Technical Report

https://arxiv.org/abs/2504.07491
Kimi-VL Technical Report | Summary
Kimi-VL Technical Report combines MoonViT, an image encoder accepting varying resolutions, with an expert-routed language decoder for vision-language tasks, which process images alongside text. The decoder has 16B total parameters and 2.8B activated parameters, alongside a 400M-parameter vision encoder. Joint text and visual training extends the context window, the maximum input sequence length, to 131072 tokens.
Kimi-VL-A3B-Thinking adds supervised training on extended reasoning examples and reinforcement learning using answer-correctness rewards. On MathVision full, a visual mathematics competition benchmark, Pass@1—the success rate of one generated answer—rises from 21.4% for base Kimi-VL to 36.8%. MathVista mini rises from 68.7% to 71.3%, and MMMU validation, a college-level multimodal question benchmark, rises from 57.0% to 61.7%. These results follow the 10 April 2025 version, arXiv:2504.07491v1.
Long-input retrieval and additional thinking both have measurable limits. Needle-in-a-Haystack testing, which retrieves information inserted into long sequences, reports 100.0% recall through 65536 tokens, falling to 87.0% for text and 91.7% for video in the (65536, 131072] interval. MathVista reaches 70.9% at a 4k-token thinking budget and 71.3% at 16k. Deployment assessment therefore needs input sizes and output budgets alongside activated parameter counts; controlled experiments isolating the encoder, optimizer, data mixture, and reinforcement learning are absent.

3. Self-Steering Language Models

https://arxiv.org/abs/2504.07081
Self-Steering Language Models | Summary
Self-Steering Language Models introduces DISCIPL, Distributional Constraints by Inference Programming with Language Models. A larger Planner writes an executable generation and scoring procedure, which smaller Follower models execute without finetuning. The implementation uses LLAMPPL, a Python probabilistic-programming framework, to control generation, track probabilities, and enforce constraints such as exact lengths and required word positions.
On COLLIE sentence tasks, a constrained-writing benchmark, a GPT-4o Planner and Llama-3.2-1B Follower reach 0.84 expected Pass@1 with importance sampling, which weights independently generated candidates, at a budget of 32 candidates. Here Pass@1 measures the probability that the selected answer satisfies the constraints. Direct Follower sampling scores 0.07, GPT-4o with stepwise verbal reasoning scores 0.46, and o1 scores 0.96. The o1 reference uses one query per task with max_tokens=25000 and does not match DISCIPL’s computation budget.
Sequential Monte Carlo, which replaces low-scoring partial candidates with copies of promising ones, improves the paper’s fluency scores over importance sampling across all three evaluated domains. Both methods retain lower fluency scores than direct Follower generation, and generated programs can fail at runtime or execute incorrect logic. On one sentence task, constraint satisfaction decreases above 8 candidates because resampling removes low-probability required words. The engineering trade-off is therefore constraint validity against fluency and program reliability, with total cost also dependent on the larger Planner’s prompt.