Gibson's theory of affordance, in its adherence to bottom-up direct perception, is antithetical to the top-down inferential models often proposed by modern robotics research purporting to tackle it. Such research assumes internal representation to be sacrosanct, but given current developments, to what extent can this assumption now be reexamined? The recently proposed sensorimotor contingency theory furthers the theoretical argument that internal representation is unnecessary, and its proof-of-concept application in robotics as well as the subsequent explosion in deep learning methodology sheds new light on the possibility of equipping robots with the capacity for directly perceiving their environments by exploiting correlated changes in their sensory inputs triggered by executing specific motor programs. This reexamination of direct perception is only one of several issues warranting scrutiny in current robotic affordance research. The aim of this workshop is therefore twofold.
Firstly, we will provide an overview of the state-of-the-art in affordance research and dissect open research challenges yielded thereof. Our speakers from ecological and perceptual psychology will be encouraged to ground these issues in human perceptual development and elaborate on their importance for robotics.
Secondly, we will encourage our speakers to debate whether computational models of affordance can potentially be advanced by adopting approaches that are more congruent with Gibson's original conception of direct perception. The question of whether the deep hierarchical models of the zeitgeist constitute a bridge between direct perception and internal representation provides a new vista in this interpretation of visual perception, and one which we aim to thoroughly explore.
The workshop organizers recently conducted a systematic literature survey on computational models of
affordance in robotics [1]. All papers considered in the survey (146) were classified under a
detailed taxonomy that considers a comprehensive array of criteria relating to the study of affordances
in the literature. The full results of the study
have been published online
in a database that is continuously updated with new publications.
It is hoped that some of the results that have emerged from this survey can form the inception point of the
thematic basis of the workshop. Have certain aspects of the computational modeling of affordance been
neglected in favour of others? Are certain types of models superior to others in the study of affordances
and, if so, under what circumstances? These are questions that the survey asks and hints at tentative
answers to, but that the workshop can field the detailed and fruitful discussion of amongst world-leading
cross-disciplinary experts,
with the goal of fostering focused future research directions for ultimately solving the
affordance learning problem in robotics.
All submissions will be peer-reviewed. Accepted papers will be presented during the workshop
in a poster session. Outstanding papers will be presented as oral spotlight talks and invited
to submit an extended version to a
special issue of Adaptive Behavior on "Computational Models of Affordance for Robotics",
a journal published by SAGE.
The review process for the journal is independent from the review for this workshop.
We will do our best to answer your request as soon as possible.
CONFIRMED: Prof. Justus Piater
Topic: Objects, Parts, Functions, Affordances, Actions, and Tasks
Abstract:
Human-made objects are constructed with functional parts. These are closely related to affordances; e.g., a handle is a part whose function is to be grasped and that affords grasping. It is these affordances that robots need to detect to perform tasks flexibly using unknown objects. Since affordances are often associated with parts, it makes sense for the segmentation of unknown objects into parts for affordance detection to be driven by known affordances. Then, the suitability of an object for a given task is often characterized by the combination of the actions its various parts afford. For example, to drive a nail into the wall you would like an object that possesses a part that affords pounding and another that affords grasping. I will discuss methods that allow robots to learn to segment objects in terms of their affordances, and to learn the suitability of objects for specific tasks as a function of their combination of affordances.
Biography:
Justus Piater is a professor of computer science at the University of Innsbruck, Austria, where he leads the Intelligent and Interactive Systems group. He holds a M.Sc. degree from the University of Magdeburg, Germany, and M.Sc. and Ph.D. degrees from the University of Massachusetts Amherst, USA, all in computer science. Before joining the University of Innsbruck in 2010, he was a visiting researcher at the Max Planck Institute for Biological Cybernetics in Tübingen, Germany, a professor of computer science at the University of Liège, Belgium, and a Marie-Curie research fellow at GRAVIR-IMAG, INRIA Rhône-Alpes, France. His research interests focus on visual perception, learning and inference in sensorimotor systems. He has published more than 160 papers in international journals and conferences, several of which have received best-paper awards, and currently serves as Associate Editor of the IEEE Transactions on Robotics.
CONFIRMED: Prof. Roderic A. Grupen
Topic: Affordances on a Landscape of Attractors
Abstract:
The concepts of affordance and direct perception imply a parsimonious integration of control and perception that is both efficient and inexpensive (direct) while it captures the potential for controllable interaction with the environment. This presentation considers how Gibson’s proposition can create a layer of perceptual abstraction on top of a compelling architecture for control—the landscape of attractors. Such an approach meets our desire for parsimony, but it also leads naturally to hierarchical perceptual abstractions that, once acquired, can be nearly “direct" as well. Moreover, this model can serve as the basis for interesting new perspectives on prediction, planning, and cognitive representation.
Biography:
Grupen studies how embodiment, computing, control theory, and machine learning contribute to an integrated model of intelligent systems—natural and artificial. A central focus of this work is computationally tractable controllers for complex, highly redundant systems. This framework forms a computational model of machines that interact with humans in a variety of settings. For example, it forms a computational model of early childhood development in control theoretic terms using techniques developed for adaptive optimal learning. Recent applications focus on personal robots and telemedicine in residential eldercare and stroke rehabilitation. Grupen has been the Principal Investigator on grants from the NSF, DARPA, ONR, NASA, AFOSR, and Microsoft for work at the intersection of engineering, AI, robotics, and developmental psychology. In 1996, he co-founded the Embedded Systems instructional laboratory at UMass in which undergraduate and graduate students from many disciplines learn about building integrated sensorimotor systems. Grupen is the recipient of an Outstanding Teacher Award from the School of Natural Sciences and Mathematics at UMass, is a Distinguished Faculty Lecturer, and has received a Chancellor's Medal for exemplary service to the UMass Amherst campus.
CONFIRMED: Prof. Yukie Nagai
Topic: Impact of Social Interaction on Affordance Learning
Abstract:
Humans do not always learn affordance of objects on their own but learn it from social partners. When facing a novel object, people observe other individuals capable of using the object and imitate them to efficiently acquire a new skill. Especially infants, who have limited knowledge and/or capability to manipulate objects, receive a large benefit from such social interaction. We investigated how social interaction facilitates affordance learning. Our first study analyzed scaffolding behavior given by partners. People do not simply demonstrate object manipulation but exaggerate their body movement and emphasize the usage of an object especially when interacting with immature agents (e.g., infants and robots). Our analysis revealed that infant-directed action presented by caregivers tend to highlight the initial and final states of an object, segment an action, and underline the action to be imitated (e.g., Nagai and Rohlfing, 2009; Fukuyama, …, Nagai et al., 2015). On the basis of these findings, we designed a robot that can take advantage of such scaffolding in learning (Ugur, Nagai, Sahin, and Oztop, 2015; Ugur, Nagai, Celikkanat, and Oztop, 2015). Our robot replicated the staged development of affordance learning observed in infants. Given reflex behaviors, the robot first discovered action primitives and then learned affordance of objects (e.g., pushability and graspability) by applying the primitives to different size and shape of objects. Acquired affordance enabled the robot to share the representation of novel actions with human users. Users demonstrated a desired action by combining affordance of objects so that the robot could interpret and imitate the affordance using its own repertoires. The shared representation further enabled the robot to predict the goal of an action presented by users and help them achieve the unfulfilled goal (Baraglia, Nagai, and Asada, 2016; Baraglia, Cakmak, Nagai et al., 2017). These results suggest that social scaffolding in affordance learning enriches the subsequent interaction with shared representation.
Biography:
Dr. Yukie Nagai has been investigating underlying mechanisms for social cognitive development by means of constructive approach. She designs computational neural network models for robots to learn to acquire cognitive abilities such as self-other cognition, estimation of others’ intention and emotion, altruistic behaviors, and so on based on the theory of predictive learning of sensorimotor signals. The simulator reproducing atypical perception in autism spectrum disorder (ASD), which has been developed by her group, greatly impacts on the society as it enables people with and without ASD to better understand potential causes for social difficulties. She is the research director of JST CREST Cognitive Mirroring since December 2016.
CONFIRMED: Prof. Stefanie Tellex
Topic: Learning Models of Language, Action and Perception for Human-Robot Collaboration
Abstract:
Robots can act as a force multiplier for people, whether a robot assisting an astronaut with a repair on the International Space station, a UAV taking flight over our cities, or an autonomous vehicle driving through our streets. To achieve complex tasks, it is essential for robots to move beyond merely interacting with people and toward collaboration, so that one person can easily and flexibly work with many autonomous robots. The aim of my research program is to create autonomous robots that collaborate with people to meet their needs by learning decision-theoretic models for communication, action, and perception. Communication for collaboration requires models of language that map between sentences and aspects of the external world. My work enables a robot to learn compositional models for word meanings that allow a robot to explicitly reason and communicate about its own uncertainty, increasing the speed and accuracy of human-robot communication. Action for collaboration requires models that match how people think and talk, because people communicate about all aspects of a robot's behavior, from low-level motion preferences (e.g., "Please fly up a few feet") to high-level requests (e.g., "Please inspect the building"). I am creating new methods for learning how to plan in very large, uncertain state-action spaces by using hierarchical abstraction. Perception for collaboration requires the robot to detect, localize, and manipulate the objects in its environment that are most important to its human collaborator. I am creating new methods for autonomously acquiring perceptual models in situ so the robot can perceive the objects most relevant to the human's goals. My unified decision-theoretic framework supports data-driven training and robust, feedback-driven human-robot collaboration.
Biography:
Stefanie Tellex is the Joukowsky Family Assistant Professor of Computer Science and Assistant Professor of Engineering at Brown University. Her group, the Humans To Robots Lab, creates robots that seamlessly collaborate with people to meet their needs using language, gesture, and probabilistic inference, aiming to empower every person with a collaborative robot. She completed her Ph.D. at the MIT Media Lab in 2010, where she developed models for the meanings of spatial prepositions and motion verbs. Her postdoctoral work at MIT CSAIL focused on creating robots that understand natural language. She has published at SIGIR, HRI, RSS, AAAI, IROS, ICAPs and ICMI, winning Best Student Paper at SIGIR and ICMI, Best Paper at RSS, and an award from the CCC Blue Sky Ideas Initiative. Her awards include being named one of IEEE Spectrum's AI's 10 to Watch in 2013, the Richard B. Salomon Faculty Research Award at Brown University, a DARPA Young Faculty Award in 2015, a NASA Early Career Award in 2016, a 2016 Sloan Research Fellowship, and an NSF Career Award in 2017. Her work has been featured in the press on National Public Radio, BBC, MIT Technology Review, Wired and Wired UK, as well as the New Yorker. She was named one of Wired UK's Women Who Changed Science In 2015 and listed as one of MIT Technology Review's Ten Breakthrough Technologies in 2016.
CONFIRMED: Prof. Robert E. Shaw
Topic: Three Affordances Types Exhibited by Physical Therapy Patients Wearing Assistive Robotics and an Action Measure of Therapeutic Success
Abstract:
The normal degrees of freedom for intended ambulatory and manipulatory actions are reduced by lesions occurring in physical therapy patients due to injury or disease. The ecological physics of tool-use is briefly reviewed in the context of the developing theory of intentional dynamics. More specifically, review of the Physical Therapy field shows a progression of technological advances with respect to assistive prosthetic devices that may be worn by the patient. In this regard, we present an array of all wearable assistive devices that have evolved over time, including both passive and static (e.g., a peg-leg) or active contributors to the task dynamics required for helping the patient restore lost degrees of freedom (e.g. a wearable robot with "learning" capabilities). This array shows a progression from tool to symbiote to full socially cooperative partner (e.g. a PT), wherein the therapy intention is not simply materially supported but actually shared by patient, patient's assistive device, and the therapist who must assure the device is properly programmed. In this discussion we introduce three kinds of affordance of interest: environmental affordances, actor affordances, and social affordances. Here the social dyad of patient and therapist is now broadened to be a social triad that includes the patient, the programmable assistive device that interfaces patient with the therapist. Finally, we present a genral method for calculating Total Generalized Action as a measure of relative success for all such patient-assistive device combinations in a variety of goal-directed real-world daily activities. This action measure is designed to aid the therapist in diagnosis, prognosis, treatment planning, treatment application, and treatment evaluation. The new measure provides a more meaningful and specific index of improvement than does either metabolic cost or information (bit-count) measures used heretofore.
Description:
Talk based on work by Robert Shaw and Jeffrey Kinsella-Shaw, Center for the Ecological Study of Perception and Action, University of Connecticut.
Biography:
1960 Mathematics/philosophy B.A. Birmingham-Southern College
1965 Philosophy/psychology Ph.D. Vanderbilt University
1965-1967 Human Learning/abstract machine theory Post Doc University of Minnesota
1968-1975 Assistant, Associate, and Professor at University of Minnesota
1969-1970 Visiting Professor Cornell University
1975-2002 Professor University of Connecticut
Background: Psychology—perception, action, cognitive psychology, ecological physics, goal-directed systems. (Approx. 100 journal articles and book chapters and four books). Invited presentations at over fifty national and international conferences and symposia (e.g., USA, Canada, India, Japan, Korea, China, France, Germany, England, Italy). Visiting scientist at Air Force and Navy laboratories. HONORS: Career Development Award (1970-1975); Fellow, Center for Advanced Studies in the Behavioral Sciences (1974-1975). Fellow, Japan Society for the Advancement of Science (2000).
First Phase (1968-1975): My early research was on psycholingustics: the application of mathematical methods of formal languages, generative grammars, and abstract machines to theories of learning and concept acquisition. In addition, this involved cybernetic and computer models, connection between logic and the Shannon information measure, application of symmetry group theory to perception and cognition.
Second Phase (1975-2000): During this period my research and theory development focused on the role of learning in tasks requiring perceptual information in the control of action. Also, I explored methods for the ergonomic design of various vehicle simulators for air force and navy (e.g., driving automobiles, piloting aircraft, landing helicopters, submarine navigation and attack/defense tactics). Independent of this we developed smart displays for computer-assisted medical training (e.g., training clinicians for safe and efficient ICU drug administration, explored a tensor calculus model for vector defined diagnostics, prognostics, and treatment of orthopedic anomalies (e.g., orthodontics).
Third Phase 2000-present: Until retirement from University of Connecticut, I was the Director of the Laboratory for the Study of Intentional Dynamics in the Center for the Ecological Study of Perception and Action, where we investigated problems of ecological psychology and ecological physics as they pertain to goal-directed actions. Here we began exploring the possible usefulness of applying principles of special relativity and quantum theory (esp. Feynman path integrals) to these problems.
Contributions to Science: My general scientific contributions include being founding president of the International Society of Ecological Psychology (ISEP), instrumental in establishing the Center for Ecological Study of Perception and Action, a book series, Resources for Ecological Psychology, a journal, Ecological Psychology, as well as being organizer in 1981 of the first in a series of continuing international bi-annual conferences entitled the International Conference on Event Perception and Action (ICPA).
CONFIRMED: Prof. Alberto Rodriguez
Topic: Learning affordances for picking, pushing and their synergies
Abstract:
Skilled manipulation benefits from synergies between non-prehensile and prehensile primitive motions, for example when pushing clutter out of the way to make space for fingers to grasp an object. The discretization of manipulation into primitive behaviors allows a reduction of perception into affordances, where the question is where to execute a behavior, rather than what is the state of the world. This yields the opportunity to focus exploration for systems that learn from experience. I'll start the talk by briefing on recent results from the Amazon Robotics Challenge. The need for robust perception has lead us, and several other teams, to show that it is possible to pick and place novel objects in severe clutter by learning affordances for grasping and suction using deep convolutional neural networks. This has demonstrated performance superior to more classical approaches that sequence state estimation and grasp planning. We are currently investigating the use of deep reinforcement learning to model affordances of different primitive manipulative motions and capture their synergies, and that way enable complex behaviors like pushing clutter out of the way before grasping an object.
Biography:
Alberto Rodriguez is the Walter Henry Gale (1929) Career Development Professor at the Mechanical Engineering Department at MIT. Alberto graduated in Mathematics ('05) and Telecommunication Engineering ('06) from the Universitat Politecnica de Catalunya (UPC) in Barcelona, and earned his PhD in Robotics (’13) from the Robotics Institute at Carnegie Mellon University. He spent a year in the Locomotion group at MIT, and joined the faculty at MIT in 2014, where he leads the Manipulation and Mechanisms Lab (MCube). Alberto is the recipient of the Best Student Paper Awards at conferences RSS 2011 and ICRA 2013 and Best Paper finalist at IROS 2016. He has lead Team MIT-Princeton in the Amazon Robotics Challenge between 2015 and 2017, and is recipient of the Amazon Research Award. His main research interests are in autonomous robotic manipulation, mechanical design, and automation.
