Concept-Based Mechanistic Interpretability and Empirical Learning Mechanics
A personal overview of the questions that drive my research: what concepts LMMs represent, how those representations change during training, and how we can turn this understanding into better systems. I share our group's explorations of concept-based mechanistic interpretability and empirical learning mechanics, from theoretical foundations and analysis tools to practical applications.
Read MoreThe Crossroads of Detection, Reaction, and Annotation
As AI-generated content becomes increasingly difficult to distinguish from human-created work, how can we identify and act on the traces it leaves behind? Drawing on research in LanSE, Matryoshka Transcoders, and ForensicZoom, alongside experience in industrial forgery detection, I explore watermarks as a shared foundation for content detection, policy decisions, and annotation.
Read MoreA Tuṣita on Earth
A personal essay on three possible worldviews, the limits of scientific inquiry, and the roots of what we find meaningful. From a mathematical graph of meaning to my encounters with Buddhism, I reflect on Tuṣita and the kind of life I hope to lead.
Read MoreConcept-Based Mechanistic Interpretability Needs a Concrete Evaluation Paradigm
A walkthrough of our NeurIPS 2026 position paper. Reconstruction loss, automated interpretability scores, and the other standard metrics for sparse dictionary learning do not directly measure whether the ground-truth concepts are recovered. We show this with theory and experiments, propose four principles for a concrete evaluation paradigm, and sketch a capsule-based benchmark for concept localization.
Read MoreCapsule Lens: Locating and Tracking Concept Geometry in Model Representations
A walkthrough of Capsule Lens, a framework that matches concept geometry in model representation spaces with simple, interpretable shapes. We show how these geometric summaries can characterize concepts across layers and track how their representations change during pretraining, PPO post-training, and RLVR.
Read MoreA Unified Theory of Sparse Dictionary Learning in Mechanistic Interpretability: Piecewise Biconvexity and Spurious Minima
The first unified theoretical framework for Sparse Dictionary Learning in mechanistic interpretability, explaining why SDL methods suffer from dead neurons, polysemanticity, and feature absorption—and providing a principled fix grounded in piecewise biconvexity and non-identifiability.
Read MoreModality Gap–Driven Subspace Alignment Training Paradigm For Multimodal Large Language Models
A walkthrough of a paper that goes beyond the isotropic assumption to formally decompose the modality gap in CLIP-style models into a Stable Bias and Anisotropic Residuals. The paper introduces ReAlign, a closed-form alignment strategy, and ReVision, a training pipeline enabling text-only pretraining for MLLMs.
Read MoreBrainExplore: Large-Scale Discovery of Interpretable Visual Representations in the Human Brain
An automated framework that discovers thousands of interpretable visual concepts encoded across the human visual cortex—from "hands holding objects" to "forest scenes"—revealing fine-grained brain representations previously unreported.
Read MoreThe Evolution of Multimodal Model Architectures: A Taxonomy
A comprehensive walkthrough of Wadekar et al.'s taxonomy of multimodal architectures. Discover the four distinct architectural patterns—Type-A through Type-D—that define how models like GPT-4V, LLaVA, and Gemini combine vision, language, and other modalities.
Read MoreThe Linear Representation Hypothesis and the Geometry of Large Language Models
A walkthrough of Park et al.'s ICML 2024 paper on the linear representation hypothesis. We explore how high-level concepts are represented as linear directions in language model representation spaces, with formal definitions and theoretical foundations.
Read More