Thomas Walsh Department of Computer Science formaer PhD Student at Rutgers University

Since I am constantly on the move, the best way to contact me is via EMAIL: thomaswa@cs.rutgers.edu

News
I Have graduated from Rutgers and am now working as a post-doc at the University of Arizona. My new webpage is HERE I am currently looking for a full time research or faculty position-- my CV and other materials can be found on my new web page.

Thesis
Here it is, my thesis (also available without hyperlinks).

CV and Research
A copy of my CV .


Job du Jour:
I am currently a post-doc at the University of Arizona. Previously, I was a Research Assistant in the Rutgers Laboratory for Real Life Reinforcement Learning (RL^3). My current research focusses on representation issues in planning and reinforcement learning.

Research:
I am an active member of the Rutgers Labratory for Real-Life reinforcement Learning (RL3) under the direction of Dr. Michael Littman. My research interests focus on representation issues for reinforcement learning problems, including state abstractions, factored and relational descriptions, and instance based approaches. In short, I play with robot puppies

PAPERS:
Lihong Li, Michael L. Littman, Thomas J. Walsh, and Alexander L. Strehl: Knows what it knows: A framework for self-aware learning. To appear in Machine Learning.

Fusun Yaman, Thomas J. Walsh, Michael L. Littman, Marie desJardins Democratic Approximation of Lexicographic Preference Models To appear in Artificial Intelligence, Special Issue on Representing, Processing, and Learning Preferences.

Thomas J. Walsh, Sergiu Goshin, and Michael L. Littman Integrating Sample-based Planning and Model-based Reinforcement Learning In Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence (AAAI-10), Atlanta, GA, 2010.

Thomas J. Walsh, Kaushik Subramanian, Michael L. Littman, and Carlos Diuk Generalizing Apprenticeship Learning across Hypothesis Classes In Proceedings of the Twenty-Seventh International Conference on Machine Learning (ICML-10), Haifa, Israel, 2010.

Also appeared at the 2010 Workshop on Agents Learning Interactively from Human Teachers (ALIHT) at AAMAS-10.

Thomas J. Walsh, István Szita, Carlos Diuk, and Michael L. Littman Exploring Compact Reinforcement-Learning Representations with Linear Regression In Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence (UAI-09), Montreal, Quebec, 2009.

A Tech Report is available for this paper that corrects the bounds reported in the conference version (with full proofs).

Thomas J. Walsh, Ali Nouri, Lihong Li, and Michael L. Littman Learning and Planning in Environments with Delayed Feedback In the Journal of Autonomous Agents and Multi-Agent Systems, Volume 18, Issue1, 83-101, February, 2009.

Thomas J. Walsh and Michael L. Littman Efficient Learning of Action Schemas and Web-Service Descriptions In Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence (AAAI-08), Chicago, IL, 2008.

Expanded version available as a Technical Report

Lihong Li, Michael L. Littman, Thomas J. Walsh Knows What It Knows: A Framework for Self-Aware Learning In Proceedings of the Twenty-Fifth International Conference on Machine Learning (ICML-08), Helsinki, Finland, 2008.

Co-winner of the ICML 2008 Best Student Paper Award

Fusun Yaman, Thomas J. Walsh, Michael L. Littman, Marie desJardins Democratic Approximation of Lexicographic Preference Models In Proceedings of the Twenty-Fifth International Conference on Machine Learning (ICML-08), Helsinki, Finland, 2008.

Also appeared at the 4th Multidisciplinary Workshop on Advances in Preference Handling at AAAI-08.

Thomas J. Walsh, Ali Nouri, Lihong Li, and Michael L. Littman Planning and Learning in Environments with Delayed Feedback In Proceedings of the 18th European Conference on Machine Learning (ECML-07), Warsaw, Poland, 2007.

Thomas J. Walsh and Michael L. Littman Planning with Conceptual Models Mined from User Behavior In Proceedings of the AAAI-07 Workshop on Acquiring Planning Knowledge via Demonstration, Vancouver, BC, 2007.
SLIDES: here

Thomas J. Walsh, Lihong Li, and Michael L. Littman Transferring State Abstractions Between MDPs In Proceeding of the ICML-06 Workshop on Structural Knowledge Transfer for Machine Learning, Pittsburgh, PA, 2006.
SLIDES: here

Lihong Li, Thomas J. Walsh, and Michael L. Littman Towards a Unified Theory of State Abstraction for MDPs Proceedings of the Ninth International Symposium on Artificial Intelligence and Mathematics (AIMA06), Ft. Lauderdale, FL, 2006.

Alex Borgida, Thomas J. Walsh, and Haym Hirsh. Towards Measuring Similarity in Description Logics In Proceedings of the 2005 International Workshop on Description Logics (DL2005), Edinburgh, Scotland, 2005.

Bethany R. Leffler, Michael L. Littman, Alexander L. Strehl, Thomas J. Walsh. Efficient Exploration With Latent Structure In Proceedings of Robotics: Science and Systems. Cambridge, Massachusetts, 2005.

Thomas J. Walsh and D. Richard Kuhn. Challenges in Securing Voice over IP IEEE Security & Privacy. Vol 3(3) 2005 (May/June) : 44-49.

D. Richard Kuhn, Thomas J. Walsh, Steffen Fries. Security Considerations for Voice Over IP Systems Special Publication from the National Institute of Standards and Technology, 2005 [final version!] (slash-dotted on 5/6/2005 here).

Dennis D.Y. Kim, Thomas T.Y. Kim, Thomas Walsh, Yoshifumi Kobayashi, Tara C. Matise, Steven Buyske, and Abram Gabriel Widespread RNA Editing of Embedded Alu Elements in the Human Transcriptome Genome Res. 2004 14 (September): 1719-1725.

Video
Here's a video of RL in action (implemented with the help of many others in the RL^3 Lab back in 2005) on a Sony Aibo trying to escape from a darkened room. Video
A webpage with more info on the task is here: Explanation