eTui prototype field trial

Abstract  

Initial trials with the prototype eTui robot are described.  These were held from October 2000 to January 2001.  The trials were structured accoding to the eTui methodology review, and this document sets out conclusions from analysis of the video recordings made.  Reference is also made to preparatory sessions discussing learning with children in May - June of 2000.  A more detailed description of the trials is available in the Annex to this document.

1. Introduction

This document reports on the final trials of the éTui project. In the first phase of the project, the main results obtained related to the appearance of the toy, with the objectives that it should be educationally sound, engaging for the children and not gender-biased.  It was developed using a methodology of co-design with children, and explored the behaviours that would be likely to evoke children’s reflection on learning.  This work is described in the document "eTui robot prototype trials".

The second phase of the project was planned when we had sufficient éTui prototypes working robustly enough for extensive trials with the schools. For this phase a document was developed which explicitly described the methodology to be used. The main point of investigation was the autonomous behavaviours of éTui related to the reflection on learning educational goal. The paradigm is that the changes of the toy motivated by sensing the environment for pursuing its goal are perceived as a candidate for the “robot” appearing to “learn” something,  would motivate reflections on learning.

In addressing the research questions set out in the eTui methodology review, we looked for evidence of 

- how the design of éTui is received by children in terms of attractiveness, extended engagement and comparable acceptance by both girls and boys,

- the level and what kinds of philosophical reflection and curiosity about learning, autonomy and sensory perception éTui stimulates when used in a classroom or home activity with other children, teacher, parent and facilitator involvement and intervention.

We had planned to get an extensive view of teachers views on these issues, but we only performed it with our school teachers.

In order to get a more structured approach, Units of Practice (in Apple’s terminology) which we shall denote as UoPs, were designed and used for the school trials.  The UoPs which we developed were: Reflecting on Perception; Reflecting on Autonomy; Reflecting on Learning and Design. The wandering insect behaviour is used for the first one; the éTui in a simple maze, for the second one; while the possibility of teaching movements to the toy by children’s direct manipulation is used in the last unit.

2. Summary of the trials

There have been three types of activities with the children: preparatory activities that were giving clues about their abilities of reflecting about learning not using éTuis; and, from the end of the summer, and more intensely, during October-November-December, educational activities with éTuis in the schools. In this phase of the project there were 31 schools visits, with a total of 39 sessions being conducted.  In addition .  Two visits to schools were abandoned due to hardware problems and had to be rescheduled.  Of these 18 visits were to Robert Graves School in Deià, Majorca with a total of 25 sessions; 6 visits were to Hollyfields School in Brentwood, England, with a total of 11 sessions, and three days of intensive work were carried out in Birralee School in Norway. A class from a school also visited the Ultralab to participate in the project.

We recall some of the characteristics of the schools. In the rural area of Deià the school is very small, and the class was a mix of children from 5 - 8 years old.  Of the 13 children in the class only 2 were boys. In the Norway school the children were between 8 - 10 years old.  These were a little older than our target group, but this is because they had been working with the project since its start two years ago.  The gender distribution was the opposite of Deià, with 11 boys and three girls. In Hollytrees school, Chelmsford, the

The usual two facilitators: Dai Griffiths and Richard Millwood helped the schools, and an extra facilitator was hired for supporting the Norway trials. Two extra people also helped in the Chelmsford trials. The facilitators were in frequent contact with the teachers to prepare the activities.  The children were working in small groups, from 2 to a maximum of 6 members. The activities usually took place outside of the normal classroom.

The trials in Norway were organised differently from the other trial centres, as they had to be concentrated in three days, while at the other schools they took place over a number of months.

3. Content of the trials

The sessions followed the pattern of the Units of Practice that were developed as part of the methodological work, and these are included in the eTui Methodology Review.  As had been intended, the facilitators used the UoPs as a framework for their interactions, as it was important to make space for the creative responses of the children.  The trials were carried out in accordance with the eTui Methodology Review, included in this documentation. A substantial part of the trials was the discussion carried out by the facilitators with the children about perception and learning.

The main blocks of school activities were as follows: 

3.1 Introductory activities.

While the eTui robot was still under development, in the summer of 2000, a number of introductory activities were carried out.  These involved the children in talking about learning, and in a number of different situations.  These ranged from free flowing interviews (five interviews in Hollyfields School, England), to a focused interview on Pokemon and learning ( Deià School Majorca), and the use of computer games as facilitators of childrens undestanding of learning (five sessions in Deià School).  The eTui prototype was also presented to the classes in England and Majorca which had been involved in the design work, to the great satisfaction an in interest of the children.

3.2 UoP 1

The first UoP, focusing on perception, used the wandering insect behaviour of the éTui. After introducing it to the children they observed the éTui moving around and interacted with it. The children often blocked its path, with their hands or with obstacles, and their engagement with the toy also led them to construct homes for the éTui.  There was extensive discussion of the nature of the eTui's perception.

3.4 UoP 2

The second UoP, focusing on autonomy orientation, used the "bounded insect" behaviour, which is a very simple case of a maze.  The toy moves around the bounded area rather like a fly stuck in a bottle, and eventually finds its way out.  The children were engaged in discussion about the whether the toy wanted to find its way out, the problems which it faced in doing so, and the way in which they would have approached the situation, had they a similarly limited sensory perception of the world.

3.5 UoP 3

The third UoP, focusing on learning, used the "repeating a path" behaviour.  The facilitator and children pushed the toy along a path, and the toy then repeated this movement.  The children were engaged in discussion of whether the toy was learning when it performed this action, and whether they learnt in the same way.

It is worth remarking that we cannot separate completely these UoPs: a progress was taking place in the children’s understanding of the toy and consequently they were more able to discuss and reflect on its behaviour.

4. Results of the trials

During the activities the facilitator discussed with the children their opinions about the toy’s actions, perception, ability to learn, with the aim of relating these to the children's own capabilities. After the sessions, written analysis was performed.

Nearly all the sessions were recorded on video in order to facilitate careful analysis of the experiences, and most of this has been transcribed and incorporated into a database, together with the comments of the facilitators, which has been used in the preparation of this report.  Having a searchable corpus of the work done greatly enhances the usefulness of the data generated in the trials. This database contains around 20,000 words of text, and represents a significant resource for further use in further analysis, and as the basis for subsequent research building on this project.

A detailed report on the trials is available in an annex to this report, the main findings of which are as follows:

4.1 Confirmation of working hypotheses with the prototype

How the design was percieved

-        Engagement was achieved both in appearance and behaviours

-        The prototype fulfilled the design team's pragmatic goal of producing a design which was appealing to girls without alienating boys by resorting to stereotypes of feminimity.  It is a positive indicator of acceptance that  many girls identified the toy as having female gender.

-        Children have a strong tendency to project animate intelligence onto the toy. 

Children's reflection using the toy

-        Childrens attempts at understanding the gulf between their projections and the toys actual performance provide rich learning opportunities.

-        The decision to avoid features on the toy which would mislead the children as to its abilities (such as eyes, a voice, or references to intelligent animals) increased the potential for learning.

-        Metalevel relection can be achieved using the eTui, but it takes time.

-        Reflective learning using robotic toys is not dependent on a high level of robotic intelligence.

-        eTui is most effective with a collaborative pedagogy based on dialogue, and can be a tool for shifting towards this pedagogy.

-        The constraints imposed by the UOPs led to more fruitful discussion.

4.2 New insights

-        Childrens use of the toy falls into a three stage pattern: fascination, problem solving and   reflection. Necessary improvements for future itterations

4.3 Evaluation of the interface

-        The doh re mi interface which controls the selection of programs could be controlled by most children of seven or eight, but was difficult for younger children.

-        The delicate control of the "do it" button, and the coordination with the changing colour of the LED indicator, are challenging for younger children.

-        More feedback should be provided, indicating which mode the eTui is in at any time.

-        Elementary context sensitivity of the toy is essential: it should “know” its being handled.

4.4 Hardware evaluation

-        The hardware needs to be more robust, and the "repeating a path" behaviour should be much more reliable.

-        More memory is required, so as to be able to program more complex behaviours, and in particular behaviours which develop over a period of time.  Some form of persitent memory would be desirable, so that children can have a relationship with the toy which builds over a number of sessions.

-        The problems identified in the evaluation of the interface should be addressed

-        The exploitation of sound should be greatly extended, both in terms of making the sound more varied and also giving it more meaning in terms of the toys behaviour.

5. Pointers for Further research

-        In all subsequent work, continued investigation into gender attribution of toys, and prefered toys of boys and girls.

-        Investigation of the balance between increasing the attractiveness of the toy through more sophisticated behaviours, and a possible reduction in reflection because of increased distraction and reduced contrast with the childrens own cognition.

New eTui actions

-        Autonomous behaviours v. control

-        Interaction between eTuis

-        Repeated paths and cognitive maps.

-        Creating a home for the eTui

-        Voice training

Hardware development

-        Addition of a camera

-        Touch awareness.

-        More varied sounds with a more explicit relationship to the toys internal state.

5. Methodological reflections (on the design and test)

The use of the Unit of Practice methodology was successful in unifying the work carried out at the three trial centres, and in ensuring that educational and technological goals were fully integrated.  Comparison with more open ended activities and interviews carried out in the initial activities of this phase of the project also suggest that the constraints imposed by the UOPs led to more fruitful discussion of perception and learning, and indeed there may be a case for adding more constrained activities.

The facilitators noted that with this age group written activities often proved less productive than dialogue.  This is not to say, of course, that there is no place for written work. The teacher may of course decide that she wishes to use the eTui as a stimulus for written work, or to ask the children to report on what they have learnt.  As a means of reflection, however, facilitated dialogue seems to be more effective. 

The UoPs are intended to be flexible, and one important factor is the amount of time which a teacher or facilitator can dedicate to activities with the eTui.

It proved difficult in practice to maintain a rigid distinction between the subject matter of the three UoPs. In the first sessions the children were fascinated and enchanted by the toy, and their projections of capabilities onto the toy were strong.  These factors made it difficult for them to focus clearly on the problem of the nature of the toys perception.  As time went on they gradually formed a clearly picture of the toys actions, and were better able to discuss it.  Consequently some of the more interesting discussions of perception occurred in UoP 2, which was intended to focus on orientation and autonomy, rather than in UoP 1.  Similarly the performance of the eTui in the bounded area raised issues related to learning.

We hypothesise that this is a pattern which will emerge in any similar use of interactive toys consisting of three stages:

1. Initial enchantment and a tendency to project intelligence and animate qualities on the toy.  This response occurs without any adult intervention.

2. A problem solving stage, where the behaviour of the unknown artefact presents an implicit challenge to the children, to describe and understand.  At the present state of robotic technology this problem is increased by the projection of abilities onto the toy by the children (and adult observers).  This is a complex task, and in order to make it achievable it is important not to complicate matters further by introducing misleading interface elements (such as eyes which cannot "see" in any everyday sense of the word, or an intelligent or emotionally appealing design which implies consciousness).

Children sometimes engage in this problem solving activity in the course of their play with the toy, but the presence of a facilitator greatly helps in drawing out their reflection.

3. An opportunity for reflection on the differences between the toys ability and that of the children themselves.  Children do not usually enter into this stage spontaneously.

We further suggest that children need time to work through these stages (depending on their previous experiences). This suggests that it would be wise to have an introductory stage, in which the objective is simply that the children can explore the nature of the artefact, without any specific learning goals.

The facilitator notes included in the trials database are taken from the eTui Project server in Ultralab, and they were intended to keep the other facilitators informed about progress in Deià.  The prime data source for the trials is the video recording and transcription, but the facilitator notes have also proved to be useful in providing a record of the facilitators immediate impressions of the work done.  Had this been recognised earlier then then a form could have been prepared for facilitators to fill in at the end of each session. This would have provided interesting auxiliary information.

6. Principal conclusions

The eTui is engaging and attractive for children.

The eTui, and similar autonomous robotic devices, constitute a problem solving activity: describing and understanding the toys interaction with the world and the child.  To make use of this learning opportunity the design should not include misleading features implying capablilities which the toy does not possess. 

The eTui provides opportunities for childrens reflection on perception and learning.  Such reflection requires facilitation.

The eTui is best used with small groups of children with the facilitation of the teacher.

The childrens repsonse to the eTui went through the stages of initial enchantment, problem solving, and an opportunity for reflection.  We hypothesise that this may be generalisable to similar activities with other autonomous robotic devices.