publications

Publications

• DEFT: Dexterous Fine-Tuning for Real-World Hand Policies

  Aditya Kannan*, Kenneth Shaw*, Shikhar Bahl, Pragna Mannam, Deepak Pathak

  CoRL 2023

  website | arXiv | abstract | bibtex

  Dexterity is often seen as a cornerstone of complex manipulation. Humans are able to perform a host of skills with their hands, from making food to operating tools. In this paper, we investigate these challenges, especially in the case of soft, deformable objects as well as complex, relatively long-horizon tasks. Although, learning such behaviors from scratch can be data inefficient. To circumvent this, we propose a novel approach, DEFT (DExterous Fine-Tuning for Hand Policies), that leverages human-driven priors, which are executed directly in the real world. In order to improve upon these priors, DEFT involves an efficient online optimization procedure. With the integration of human-based learning and online fine-tuning, coupled with a soft robotic hand, DEFT demonstrates success across various tasks, establishing a robust, data-efficient pathway toward general dexterous manipulation.

  @article{kannan2023deft,
    title={DEFT: Dexterous Fine-Tuning for Real-World Hand Policies},
    author={Kannan, Aditya and Shaw, Kenneth and Bahl, Shikhar and Mannam, Pragna and Pathak, Deepak},
    journal={CoRL},
    year={2023}
  }

• 
  Learning from human videos for robotic manipulation

  Aditya Kannan

  Master's Thesis 2024

  website

• Learning dexterity from human hand motion in internet videos

  Kenneth Shaw*, Shikhar Bahl*, Aravind Sivakumar, Aditya Kannan, Deepak Pathak

  IJRR 2024

  website | arXiv | abstract | bibtex

  To build general robotic agents that can operate in many environments, it is often useful for robots to collect experience in the real world. However, unguided experience collection is often not feasible due to safety, time, and hardware restrictions. We thus propose leveraging the next best thing as real world experience: videos of humans using their hands. To utilize these videos, we develop a method that retargets any 1st person or 3rd person video of human hands and arms into the robot hand and arm trajectories. While retargeting is a difficult problem, our key insight is to rely on only internet human hand video to train it. We use this method to present results in two areas: First, we build a system that enables any human to control a robot hand and arm, simply by demonstrating motions with their own hand. The robot observes the human operator via a single RGB camera and imitates their actions in real-time. This enables the robot to collect real-world experience safely using supervision. See these results at https://robotic-telekinesis.github.io. Second, we retarget in-the-wild human internet video into task-conditioned pseudo-robot trajectories to use as artificial robot experience. This learning algorithm leverages action priors from human hand actions, visual features from the images, and physical priors from dynamical systems to pretrain typical human behavior for a particular robot task. We show that by leveraging internet human hand experience, we need fewer robot demonstrations compared to many other methods. See these results at https://video-dex.github.io

  @article{shaw_internetvideos,
    title={Learning dexterity from human hand motion in internet videos},
    author={Shaw, Kenneth and Bahl, Shikhar and Sivakumar, Aravind and Kannan, Aditya and Pathak, Deepak},
    journal={International Journal of Robotics Research},
    year={2024}
  }

• HypoRiPPAtlas as an Atlas of hypothetical natural products for mass spectrometry database search

  Yi-Yuan Lee, Mustafa Guler, Desnor N. Chigumba, Shen Wang, Neel Mittal, Cameron Miller, Benjamin Krummenacher, Haodong Liu, Liu Cao, Aditya Kannan, Keshav Narayan, Samuel T. Slocum, Bryan L. Roth, Alexey Gurevich, Bahar Behsaz, Roland D. Kersten, Hosein Mohimani

  Nature Communications 2023

  website | abstract

  Recent analyses of public microbial genomes have found over a million biosynthetic gene clusters, the natural products of the majority of which remain unknown. Additionally, GNPS harbors billions of mass spectra of natural products without known structures and biosynthetic genes. We bridge the gap between large-scale genome mining and mass spectral datasets for natural product discovery by developing HypoRiPPAtlas, an Atlas of hypothetical natural product structures, which is ready-to-use for in silico database search of tandem mass spectra. HypoRiPPAtlas is constructed by mining genomes using seq2ripp, a machine-learning tool for the prediction of ribosomally synthesized and post-translationally modified peptides (RiPPs). In HypoRiPPAtlas, we identify RiPPs in microbes and plants. HypoRiPPAtlas could be extended to other natural product classes in the future by implementing corresponding biosynthetic logic. This study paves the way for large-scale explorations of biosynthetic pathways and chemical structures of microbial and plant RiPP classes.

• Cost-effectiveness of Vitamin A supplementation among children in three sub-Saharan African countries: An individual-based simulation model using estimates from Global Burden of Disease 2019

  Aditya Kannan, Derrick Tsoi, Yongquan Xie, Cody Horst, James Collins, Abraham Flaxman

  PLOS ONE 2022

  website | abstract

  Vitamin A Supplementation (VAS) is a cost-effective intervention to decrease mortality associated with measles and diarrheal diseases among children aged 6–59 months in low-income countries. Recently, experts have suggested that other interventions like large-scale food fortification and increasing the coverage of measles vaccination might provide greater impact than VAS. In this study, we conducted a cost-effectiveness analysis of a VAS scale-up in three sub-Saharan African countries.