Invited Talk Abstracts.-Invited Talk: UCRL and Autonomous Exploration.-Invited Talk: Increasing Representational Power and Scaling Inference in Reinforcement Learning.-Invited Talk: PRISM – Practical RL: Representation, Interaction, Synthesis, and Mortality.-Invited Talk: Towards Robust Reinforcement Learning Algorithms.-Online Reinforcement Learning Automatic Discovery of Ranking Formulas for Playing with Multi-armed Bandits.-Goal-Directed Online Learning of Predictive Models.-Gradient Based Algorithms with Loss Functions and Kernels for Improved On-Policy Control.-Learning and Exploring MDPs -Active Learning of MDP Models.-Handling Ambiguous Effects in Action Learning.-Feature Reinforcement Learning in Practice.-Function Approximation Methods for Reinforcement Learning Reinforcement Learning with a Bilinear Q Function.-1-Penalized Projected Bellman Residual.-Regularized Least Squares Temporal Difference Learning with Nested 2 and 1 Penalization.-Recursive Least-Squares Learning with Eligibility Traces.-Value Function Approximation through Sparse Bayesian Modeling.-Automatic Construction of Temporally Extended Actions for MDPs Using Bisimulation Metrics.-Unified Inter and Intra Options Learning Using Policy Gradient Methods.-Options with Exceptions.-Policy Search and Bounds.-Robust Bayesian Reinforcement Learning through Tight Lower Bounds.-Optimized Look-ahead Tree Search Policies.-A Framework for Computing Bounds for the Return of a Policy.-Multi-Task and Transfer Reinforcement Learning.-Transferring Evolved Reservoir Features in Reinforcement Learning Task.-Transfer Learning via Multiple Inter-task Mappings.-Multi-Task Reinforcement Learning: Shaping and Feature Selection.-Multi-Agent Reinforcement Learning.-Transfer Learning in Multi-Agent Reinforcement Learning Domains.-An Extension of a Hierarchical Reinforcement Learning Algorithm for Multiagent Settings.-Apprenticeship and Inverse Reinforcement Learning Bayesian Multitask Inverse Reinforcement Learning.-Batch, Off-Policy and Model-Free Apprenticeship Learning.-Real-World Reinforcement Learning Introduction of Fixed Mode States into Online Profit Sharing and Its Application to Waist Trajectory Generation of Biped Robot.-MapReduce for Parallel Reinforcement Learning.-Compound Reinforcement Learning: Theory and an Application to Finance.-Proposal and Evaluation of the Active Course Classification Support System with Exploitation-Oriented Learning.