Mrinal Verghese

I'm a senior software engineer at NVIDIA working on foundation models for humanoid robotics. I received my PhD from the Robotics Institute at Carnegie Mellon University where I worked with Professor Chris Atkeson on robot learning from human data. A copy of my thesis can be found here. I've also worked with the embodied AI group at Meta Reality Labs Research and Fundamental AI Research (FAIR) with Ruta Desai on task planning with large language models.

I received my B.S. in Mathematics-Computer Science from UC San Diego where I worked with Professor Michael Yip on surgical robotics and motion planning. I've also worked with Brain Corp on kinodynamic planning.

I'm always happy to chat about research and potential collaborations. Feel free to reach out!

Email  /  CV  /  Google Scholar  /  Twitter  /  BlueSky  /  Github  /  LinkedIn

TL;DR: Modeling human preferences in a reward function by learning to predict short-term point motion from large egocentric video datasets yields significant improvements in robot skill learning. We demonstrate this on a set of real-world reinforcement learning experiments.

TL;DR: Using optic flow as an embodiment-agnostic action representation lets us train world models across diverse robot embodiments and even humans. These world-models paired with a novel policy steering method require significantly less data to learn new robot skills.

TL;DR: We investigate the effectiveness of different internet data modalities to select templates to complete a large set of robot cooking skills. Our novel optic-flow encoder significantly outperforms existing representations in this task.

TL;DR: We present a first-of-its-kind user study on the ability of multimodal LLMs to construct high-level task plans for realistic cooking activities. Our study identifies the best current multimodal LLM architecture and highlights long-term challenges in grounding visual history.

TL;DR Memory-based learning approaches can significantly speed up the learning process for tasks with state-action constraints, a domain where traditional reinforcement learning methods struggle.

Here is a video about this work

TL;DR: Decomposing configuration space into subspaces significantly improves the efficiency and scalability of learned collision checkers. Doing this decomposition in the "Forward-Kinematics" space leads to better performance.

TL;DR: Estimating structured corrections to a jacobian from visual observations improves the control of continuum robot manipulators.

This seminar course will cover a mixture of modern and classical methods for robot cognition. We will review papers related to task planning and control using both symbolic and numeric methods. The goal of this course is to give students an overview of the current state of research on robot cognition.

This is a course about how to make robots move through and interact with their environment with speed, efficiency, and robustness. We will survey a broad range of topics from nonlinear dynamics, linear systems theory, classical optimal control, numerical optimization, state estimation, system identification, and reinforcement learning. The goal is to provide students with hands-on experience applying each of these ideas to a variety of robotic systems so that they can use them in their own research.

This course surveys perception, cognition, and movement in humans, humanoid robots, and humanoid graphical characters. Application areas include more human-like robots, video game characters, and interactive movie characters.

This course will cover the basics about neural networks, as well as recent developments in deep learning including deep belief nets, convolutional neural networks, recurrent neural networks, long-short term memory, and reinforcement learning. We will study details of the deep learning architectures with a focus on learning end-to-end models for these tasks, particularly image classification.

This course introduces the mathematical formulations and algorithmic implementations of the core supervised machine learning methods. Topics in 118A include regression, nearest neighborhood, decision tree, support vector machine, and ensemble classifiers.

template