me Greetings from north of Asbury Park
Thomas Walsh


Since I am constantly on the move, the best way to contact me is via EMAIL: twalsh@mit.edu

News
I have graduated from Rutgers and am now working as a Postdoctoral Researcher with the Laboratory for Information and Decision Systems (LIDS) at MIT. For now you can still find all of my publications in the list below.

Thesis
Here it is, my thesis (also available without hyperlinks).

CV and Research
A copy of my CV and a research statement.


Job du jour:
I am currently a Postdoctoral Researcher with the Laboratory for Information and Decision Systems (LIDS) at MIT, where I am working on machine learning and planning techniques for sequential decision making problems.

Research:
I was formerly a member of the Rutgers Labratory for Real-Life reinforcement Learning (RL3) under the direction of Dr. Michael Littman. My primary research focus is using machine learning techniques to build models of environments and behaviors for agents that need to make decisions and take actions in the real world. My dissertation research focused on representation issues for agents in the Reinforcement Learning (RL) subsection of machine learning, where an agent must efficiently learn to act optimally in an environment based on feedback. In my post-doctoral work, I have focused on problems where learning is done through observations of people or another agent's behavior, and used these techniques to train systems in a wide range of areas, including teaching robots complicated commands and electronic tutoring. My current work focuses on adapting many of these techniques to real-world problems in educational domains.

PUBLICATIONS:

Journal Articles

Alborz Geramifard, Thomas J. Walsh, Stefanie Tellex, Girish Chowdhary, Nicholas Roy, and Jonathan P. How. A Tutorial on Linear Function Approximators for Dynamic Programming and Reinforcement Learning Foundations and Trends in Machine Learning (FTML), 2013.

Thomas J. Walsh, Michael L. Littman, and Alexander Borgida Learning Web-Service Task Descriptions from Traces Web Intelligence and Agent Systems, Volume 10, Number 4, (pages 397-421), 2012.

Lihong Li, Michael L. Littman, Thomas J. Walsh, and Alexander L. Strehl Knows what it knows: A framework for self-aware learning. Machine Learning, Volume 82, Number 3, (pages 399-443), 2011.

Fusun Yaman, Thomas J. Walsh, Michael L. Littman, Marie desJardins Democratic Approximation of Lexicographic Preference Models Artificial Intelligence, Special Issue on Representing, Processing, and Learning Preferences, Volume 175 (pages 1290-1307), 2011.

Thomas J. Walsh, Ali Nouri, Lihong Li, and Michael L. Littman Learning and Planning in Environments with Delayed Feedback In the Journal of Autonomous Agents and Multi-Agent Systems, Volume 18, Issue1, (pages 83-101), February, 2009.

Dennis D.Y. Kim, Thomas T.Y. Kim, Thomas Walsh, Yoshifumi Kobayashi, Tara C. Matise, Steven Buyske, and Abram Gabriel Widespread RNA Editing of Embedded Alu Elements in the Human Transcriptome Genome Res. 2004 14 (September): 1719-1725.

Conference Papers

Robert C. Grande, Thomas J. Walsh, Jonathan P. How. Sample Efficient Reinforcement Learning with Gaussian Processes In Proceedings of the International Conference on Machine Learning (ICML-14), Beijing, China, 2014.

  • Appendix available here

    Mark Cutler, Thomas J. Walsh, Jonathan P. How. Reinforcement Learning with Multi-Fidelity Simulators In Proceedings of the International Conference on Robotics and Automation (ICRA-14), Hong Kong, 2014.

    Alborz Geramifard, Thomas J. Walsh, Nicholas Roy, and Jonathan P. How. Batch-iFDD for Representation Expansion in Large MDPs In Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI-13), Bellevue, WA, 2013.

    Thomas J. Walsh and Sergiu Goschin. Dynamic Teaching in Sequential Decision Making Environments In Proceedings of theConference on Uncertainty in Artificial Intelligence (UAI-12), Catalina, CA, 2012.
  • Appendix available here.

    Thomas J. Walsh, Daniel Hewlett, and Clayton T. Morrison Blending Autonomous Exploration and Apprenticeship Learning In Proceedings of the Twenty-Fifth Annual Conference on Neural Information Processing Systems (NIPS-11), Granada, Spain, 2011.
  • Also appeared at the 2011 RSS Workshop on the State of Imitation Learning.

    Derek T. Green, Thomas J. Walsh, Paul R. Cohen and Yu-Han Chang. Learning a Skill-Teaching Curriculum with Dynamic Bayes Nets In Proceedings of the Twenty-Third Conference on Innovative Applications of Artificial Intelligence (IAAI-11), San Francisco, CA, 2011.

    Daniel Hewlett, Thomas J. Walsh, and Paul R. Cohen. Teaching and Executing Verb Phrases In Proceedings of the First Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics (ICDL-Epirob-11), Frankfurt, Germany, 2011.
  • Also appeared at the RSS Workshop on the State of Imitation learning and the AAAI Spring Symposium on Bridging the Gaps in Human-Agent Collaboration.

    Raquel Torres Peralta, Tasneem Kaochar, Ian R. Fasel, Clayton T. Morrison, Thomas J. Walsh, Paul R. Cohen. Challenges to Decoding the Intention Behind Natural Instruction In Proceedings of the IEEE International Symposium on Robots and Human Interactive Communications (RO-MAN-2011), Atlanta, GA, 2011.

    Derek T. Green, Thomas J. Walsh, Paul R. Cohen, Carole R. Beal and Yu-han Chang. "Gender Differences and the Value of Choice in Intelligent Tutoring Systems". In Proceedings of User Modeling, Adaptation and Personalization (UMAP-2011), Girona, Spain, 2011.

    Tasneem Kaochar, Raquel Torres Peralta, Ian R. Fasel, Clayton T. Morrison, Thomas J. Walsh, Paul R. Cohen Towards Understanding How Humans Teach Robots In Proceedings of User Modeling, Adaptation and Personalization (UMAP-2011), Girona, Spain, 2011.
  • Versions also appeared at the AAAI Spring Symposium on Bridging the Gaps in Human-Agent Collaboration and the 2011 Workshop on Agents Learning Interactively from Human Teachers (ALIHT) at IJCAI-2011.

    Thomas J. Walsh, Sergiu Goshin, and Michael L. Littman Integrating Sample-based Planning and Model-based Reinforcement Learning In Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence (AAAI-10), Atlanta, GA, 2010.

    Thomas J. Walsh, Kaushik Subramanian, Michael L. Littman, and Carlos Diuk Generalizing Apprenticeship Learning across Hypothesis Classes In Proceedings of the Twenty-Seventh International Conference on Machine Learning (ICML-10), Haifa, Israel, 2010.
  • Also appeared at the 2010 Workshop on Agents Learning Interactively from Human Teachers (ALIHT) at AAMAS-10.

    Thomas J. Walsh, István Szita, Carlos Diuk, and Michael L. Littman Exploring Compact Reinforcement-Learning Representations with Linear Regression In Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence (UAI-09), Montreal, Quebec, 2009.
  • A Tech Report is available for this paper that corrects the bounds reported in the conference version (with full proofs).

    Thomas J. Walsh and Michael L. Littman Efficient Learning of Action Schemas and Web-Service Descriptions In Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence (AAAI-08), Chicago, IL, 2008.
  • Expanded version available as a Technical Report

    Lihong Li, Michael L. Littman, Thomas J. Walsh Knows What It Knows: A Framework for Self-Aware Learning In Proceedings of the Twenty-Fifth International Conference on Machine Learning (ICML-08), Helsinki, Finland, 2008.
  • Co-winner of the ICML 2008 Best Student Paper Award

    Fusun Yaman, Thomas J. Walsh, Michael L. Littman, Marie desJardins Democratic Approximation of Lexicographic Preference Models In Proceedings of the Twenty-Fifth International Conference on Machine Learning (ICML-08), Helsinki, Finland, 2008.
  • Also appeared at the 4th Multidisciplinary Workshop on Advances in Preference Handling at AAAI-08.

    Thommas J. Walsh, Ali Nouri, Lihong Li, and Michael L. Littman Planning and Learning in Environments with Delayed Feedback In Proceedings of the 18th European Conference on Machine Learning (ECML-07), Warsaw, Poland, 2007.

    Lihong Li, Thomas J. Walsh, and Michael L. Littman Towards a Unified Theory of State Abstraction for MDPs Proceedings of the Ninth International Symposium on Artificial Intelligence and Mathematics (AIMA06), Ft. Lauderdale, FL, 2006.

    Bethany R. Leffler, Michael L. Littman, Alexander L. Strehl, Thomas J. Walsh. Efficient Exploration With Latent Structure In Proceedings of Robotics: Science and Systems. Cambridge, Massachusetts, 2005.

    Other Publications (Workshops, Magazines, Special Publications)

    Thomas J. Walsh, Javad Taheri, Jeremy B. Wright and Paul R. Cohen. Leadership Games and their Application in Super-Peer Networks AAAI Workshop on Applied Adversarial Reasoning and Risk Modeling, San Francisco, CA, 2011.

    Thomas J. Walsh and Michael L. Littman Planning with Conceptual Models Mined from User Behavior In Proceedings of the AAAI-07 Workshop on Acquiring Planning Knowledge via Demonstration, Vancouver, BC, 2007.

    Thomas J. Walsh, Lihong Li, and Michael L. Littman Transferring State Abstractions Between MDPs In Proceeding of the ICML-06 Workshop on Structural Knowledge Transfer for Machine Learning, Pittsburgh, PA, 2006.

    Alex Borgida, Thomas J. Walsh, and Haym Hirsh. Towards Measuring Similarity in Description Logics In Proceedings of the 2005 International Workshop on Description Logics (DL2005), Edinburgh, Scotland, 2005.

    Thomas J. Walsh and D. Richard Kuhn. Challenges in Securing Voice over IP IEEE Security & Privacy. Vol 3(3) 2005 (May/June) : 44-49.

    D. Richard Kuhn, Thomas J. Walsh, Steffen Fries. Security Considerations for Voice Over IP Systems Special Publication from the National Institute of Standards and Technology, 2005 [final version!] (slash-dotted on 5/6/2005 here).



    Video
    Here's a video of RL in action (implemented with the help of many others in the RL^3 Lab back in 2005) on a Sony Aibo trying to escape from a darkened room. Video
    A webpage with more info on the task is here: Explanation