Learn the LLM’s computational topology instead of hand-engineering it: We propose a structure predictor adjoint to a LLM, which generates a context-dependent computational graph applied to the LLM. This allows the modular structure to emerge with minimal contraints during training.
Memory is central to autonomous, self-evolving agents. Leveraging Integrated Information Theory (IIT), we design time-variant memory that integrates past experience via event-driven updates gated by reward, effectively treating consolidation as learning in a standard RL framework.
A comprehensive, interdisplinary survey of interpretable and biologically plausible LLMs based on my tutorial delivered at CS 591 BAI seminar. See Talks for more information.
For a complete list of my past publications, please visit my Google Scholar profile.