Send us a text

Episode Paper: https://arxiv.org/pdf/2512.16848

In this episode, we dive into a cutting-edge AI research breakthrough that tackles one of the biggest challenges in training intelligent agents: how to explore effectively. Standard reinforcement learning (RL) methods help language model agents learn to interact with environments and solve multi-step tasks, but they often struggle when the tasks require active exploration—that is, learning what to try next when the best strategy isn’t obvious from past experience.

The new paper introduces LaMer, a Meta-Reinforcement Learning (Meta-RL) framework designed to give language agents the ability to learn how to explore. Unlike conventional RL agents that learn a fixed policy, LaMer’s Meta-RL approach encourages agents to flexibly adapt by learning from their own trial-and-error experiences. This means agents can better adapt to novel or more difficult environments without needing massive retraining.

We’ll explain:

• Why exploration is critical for long-horizon tasks with delayed or sparse rewards.
• How Meta-RL shifts the focus from fixed policies to adaptable exploration behavior.
• What LaMer’s results suggest about learned exploration and generalization in AI systems.

Whether you’re into reinforcement learning, multi-agent systems, or the future of adaptive AI, this episode breaks down how Meta-RL could help agents think more like explorers—not just pattern followers.

Send us a text

In this episode, we unpack DeepSearch, a new paradigm in reinforcement learning with verifiable rewards (RLVR) that aims to overcome one of the biggest bottlenecks in training reasoning-capable AI systems. Traditional reinforcement learning methods often plateau after extensive training because they rely on sparse exploration and limited rollouts, leaving critical reasoning paths undiscovered and unlearned.

DeepSearch turns this model training approach on its head by embedding Monte Carlo Tree Search (MCTS) directly into the training loop—not just at inference time. This fundamentally changes how models explore the space of possible solutions: instead of brute-force parameter scaling or longer training runs, DeepSearch uses structured, systematic exploration to dramatically improve learning efficiency.

We break down how DeepSearch:

• Injects tree search into training, enabling richer exploration of reasoning paths.
• Uses a global frontier strategy to prioritize promising reasoning trajectories.
• Improves training-time credit assignment, so models learn not only from success but from strategic exploration itself.
• Achieves impressive results on benchmarks for mathematical reasoning, setting new state-of-the-art performance and using fewer computational resources.

Whether you’re a machine learning researcher, an AI enthusiast, or just curious about the future of intelligent systems, this episode explores how search-augmented learning could redefine how future AI systems master complex reasoning problems.

DeepSearch: Overcome the Bottleneck of Reinforcement Learning with Verifiable Rewards via Monte Carlo Tree Search

Send us a text

In this episode we’re diving into “Transformer-Squared: Self-Adaptive LLMs” — a new framework for adapting large language models to unseen tasks on the fly by tuning only a small part of their weights. The central idea is Singular Value Fine-Tuning (SVF), a parameter-efficient fine-tuning technique that decomposes each weight matrix with Singular Value Decomposition (SVD) and then only trains a small vector that scales the singular values. These vectors become compact “expert” modules that specialize in different tasks and, unlike traditional methods like LoRA, can be composed, mixed, and reused because they’re in a principled, orthogonal basis.

During inference, Transformer-Squared runs a two-pass process — the first pass identifies the task or context, and the second pass combines the appropriate expert vectors to dynamically adapt the model’s behavior in real time. Across benchmarks and architectures, SVF consistently outperforms LoRA despite requiring orders of magnitude fewer parameters, and the framework even shows versatility on multimodal tasks like vision-language.

If you’re into efficient adaptation, reinforcement-learning optimization of model components, and self-organizing AI systems, this paper is a big step toward real-time adaptive foundation models. Read the full paper here: https://arxiv.org/pdf/2501.06252