From vision for action to action for vision

Seeing agents use vision for a purpose, a purpose that is tightly connected to its embodiment and the ways with which the agent interacts with its environment. Without interaction, vision serves no real purpose. It has also been recognized that action itself is a prerequisite for perception. We observe the world as a consequence of the actions that we apply to the world. Perception and action are thus tightly intertwined, where our representation of the world is contained in the couplings between the two, not simply as an abstraction of just the perceptual data.

Given that biological systems can be seen as proofs of systems that actually work in practice, they could serve as an inspiration for the design of robotic systems. The question is what biology tells us, if we were to design an artificial seeing system? How should the sensory system and embodiment of the agent constrain the design? As an engineer you are often tempted to base your assumptions on yourself, forgetting that the embodiment of a robot can be totally different from your own.

How are relations between perception and action supposed to be represented, and what kind of information can be derived from such relations once they are learned? In this talk I will address these issues and present some recent studies on sensorimotor couplings in terms of robotic system.