How to teach a fish to swim [virtual fish tank]

We have developed a virtual fish tank in which computer users are represented by animated fish. The actions and interactions of the fish in the tank are meant to reflect the actions of users in the real world. Our first attempt at creating a programming environment that allowed people to customize their own fish did not work very well because users did not want to explicitly write programs to control their fish. Maintaining the fish tank metaphor, we attempted to solve this problem by having users teach fish rather than write code. We borrowed ideas from the literature on programming by demonstration and developed a method of programming by conditioning in which users. demonstrate behaviors and also reward (or feed) fish that are behaving appropriately. Rewards give users the ability to define highlevel behaviors (sets of specific movements) and complex relationships between situations and responses.