Law in Education Project (1984–1987): Evaluation report
In 1984 the Law Society was persuaded to fund an experimental project in association with the National Curriculum Council. This, the Law in Education Project, developed into the Citizenship Foundation which was formally founded in 1989 (see also Background to the Citizenship Foundation). What follows is the evaluation report of that initial project.
Organisation and Preparation
- Background to the Project
- Development Groups
- Aims of the Evaluation
- Organisation of the Trials
- Evaluation Activities
Putting theory into practice
- Schools and LEAs Involved in the Trials
- Ensuring Even Coverage
- Teachers Involved in the Trials
- The Problem of Getting Feedback
The main findings
- What did Teachers Think about the Value of Law-Related Education?
- What did Teachers Feel about the Selection of Legal Issues?
- What did Teachers Think about the Educational Value of the Materials?
- Did Teachers Thinks that Their Students Found the Materials Interesting?
- Did Teachers Find the Materials Suitable for Mixed-Ability Classes?
- In What Ways Were the Materials Criticised?
- What did the Students Think of the Materials and LRE Generally?
- How did Non-Expert Teachers Cope with Teaching Law?
- How Successful was the Scheme Linking Each School with a Solicitor?
- In What Courses Were the Materials Used?
Reflections and conclusions
The development phase of the Law in Education Project ran from 1984�1987.
The following is the report of that period.
This report reflects on the experiences of the period when the project's trial materials were evaluated. Data collected was used to refine the materials prior to publication in autumn 1988. It also shed light on the reactions of teachers and students to the project's approach to the law.
The project had no budget for a full-scale evaluation. David Clemson, Principal Lecturer in Education, Liverpool Polytechnic, acted as consultant evaluator and provided much helpful guidance during and after the evaluation period. It was decided not to attempt a large-scale summative evaluation of the whole project. Nevertheless, the team thought that a description of the way in which our evaluation exercise was set up and the reasons for doing what we did might be of value to others. The main findings concerning the content and approach of the materials are also summarised.
The Law in Education project, funded largely by The Law Society, had from the outset a clear set of aims�to develop an approach to law-related education in the last two years of compulsory schooling�and yet at the same time it was experimental in that we had no idea, in the first place, what form the outcomes would take nor how teachers would react to the introduction of yet another cross-curricular theme. We were mentally prepared for the fact that the experiment would fail, that the many pressures in the system would conspire against us or that teachers would throw up their hands at being asked to get involved with teaching law at any level. The reactions of teachers, many of which are recorded in this report, genuinely surprised us in their almost universal support for raising legal awareness amongst young people.
There is another reason why a report of this kind is necessary. A project team, however funded, is answerable to those who placed the responsibility on it to undertake the project. This report is an attempt to provide an account of a vital part of the project's work. Its main aim is to summarise the results of the trials and evaluation of the materials. The document also seeks to describe the process by which the evaluation itself was attempted and offers some reflections on it from the viewpoint of the project team.
Thanks must go to all those teachers who put in so much time and effort into the trials and whose comments are quoted throughout this report. Every effort has been taken to represent honestly what was said during the evaluation period. Apologies are offered to any whose views have inadvertently been misrepresented.
Don Rowe, September 1988
� SCDC/The Law Society 1988
Part one: Organisation and Preparation
Background to the Project
The Law in Education Project began work in 1984 as a joint initiative of The Law Society and the School Curriculum Development Committee (SCDC). Its aims were as follows:
- to develop and extend law-related education (LRE) in England and Wales for young people of 14-16 years and of differing abilities;
- to develop a rationale for LRE;
- to produce in-service training and classroom materials for teachers engaging in LRE;
- to provide a focus for the discussion of LRE, both within the profession and with interested outside bodies;
- to enable young people to gain a better understanding of law, it nature and function;
- to enable young people to become better informed about relevant legal aspects of their lives;
- to enable young people to develop linguistic, decision-making and problem-solving sills which would enhance their ability to behave appropriately in law-related matters;
- to encourage an attitude of independence and self-confidence in law-related matters whilst remaining open-minded and respectful of the reasonably held views of others.
Development groups of teachers, each supported by one or two solicitors, were created in eight LEAs, Calderdale, Knowsley, Leicestershire, Northumberland, Salford, Shropshire, Waltham Forest and Wiltshire. The group leaders were appointed during the autumn term, appropriate experience would be possessed by heads of department in one of the humanities subject, who had demonstrated the ability to lead a group of teachers and had some experience in curriculum development. Most group leaders were nominated by the LEA. In two cases they were volunteers from within the group. The group leaders were, typically, given one day per week release from school, though this was not adequate to fulfil all the was required of them. Again, authorities varied: one teacher was given two terms' secondment, whilst another was given no release at all.
The central team, based at SCDC, comprised Don Rowe (director), Tony Thorpe (assistant director), Lesley Aird (administrator from March 1986) and Barrie Goldstone (legal researcher during 1985-87).
The group leaders met the central team for two days at the end of December 1984 and considered a discussion document prepared by the team. This established broad agreement concerning the project's aims, its methods of working and the major areas of law to be considered. After some discussion and negotiation, it was agreed that the groups would work on the areas indicated below:
Implementing the Law
Health, Welfare and the Environment
The Individual and the Law
School, Work and Non-work
The Law and Society
Family and Home
Communications and the Law
The original time scale was divided into four main sections and was as follows:
Development of rationale; establishment of development groups.
Development period: teacher/solicitor groups working on materials, testing them and preparing them for the main evaluation.
Completion of materials development and evaluation period. Trial materials would be tested in the original eight LEAs plus some associate LEAs. (Associate LEAs were those which had originally expressed interest in supporting a development group but, for a variety of reasons, had not done so).
Revision of materials and preparation for publication.
This simple time scale proved difficult to sustain for a number of reasons. It proved impossible to edit and print the trial materials by April 1986 because the teachers' groups did not complete the development phase until then (see below). The printing and distribution processes imposed further delays, with the result that schools were receiving materials during the summer term piecemeal, and not as a coherent package, making it difficult for them to incorporate the new topics into their teaching plans. The main trials period was thus reduced from the envisaged three terms to one.
The Development Groups
The groups initially had a year (1985) in which to complete the development programme. The groups normally met during the school day, or after school, according to the account of supply cover available. One group met throughout the development period in the teachers' own time. Another group was more generously supported by its authority and was able to hold an initial two-day meeting over a weekend at a local hotel. Teachers normally met fortnightly with the group leaders coming together with the central team every four to six weeks. However, most groups found that their rate of progress was slower than they anticipated, due, in part, to the fact that it took a considerable time to develop their own understanding of the law and then to identify appropriate topics for development. The industrial action which was in progress throughout most of 1985 also had its effect. Few members left and no groups stopped working, but morale generally was low and meetings in school time became increasingly a problem. Supply cover, already scarce, was further eroded by the demands of GCSE training. More time was therefore needed and all eight LEAs agreed that the development groups could continue to meet until Easter 1996.
Despite the regular contacts which group leaders had with the central team and with each other, the materials finally produced varied considerably in length and style. In selecting and preparing materials for trial, we had several concerns in mind. In particular, we were striving to achieve a satisfactory balance between process and content. Each topic needed to focus on a small number of key ideas and/or a clearly defined and manageable area of legal knowledge. In addition, we had to find methods which provided access to these ideas for students of a wide ability range, as well as presenting them with lively material that would stimulate their interest and curiosity. Some material was rejected because it lacked clarity of purpose; other pieces were found to incorporate lively teaching methods whilst barely advancing the students' legal understanding.
An additional consideration for the central team was that of coverage. If the final outcome were to be a book or series of books to support law-related courses, then it was likely that certain key subject areas, for example the work of the courts, would need to be included. In such instances, where material coming from the groups was partial or non-existent, we felt the need to draft material ourselves.
The material developed by each group was designated a unit, and each of the eight units contained several self-standing sections covering different aspects of the law (41 in total). These were known as topics, each included student material and teacher's notes.
We originally asked the development groups to take responsibility for the design and printing of the trial materials themselves. It soon became clear, however, that this was unreasonable. This work would have placed a further demand on the teachers' time, particularly that of the group leader. The centralisation of the editing, design and layout may have been one contributory factor in the delay in printing the trial materials. It was necessary to employ the services of a free-lance editor and a designer to cope with the volume of material.
Although unplanned, this pre-evaluation editorial process was, in fact, advantageous for a number of reasons. The chief of these was that despite the common aims of the groups, the materials created were diverse in style and emphasis. The production process enabled the central team to take the first steps toward creating a unified approach and to become more familiar with the editorial and pedagogical problems that would be addressed during the main phase of the evaluation. Without the early establishment of this basic homogeneity the post-evaluation editing process, which was intended to be largely a refining one, would have been exceptionally difficult.
Aims of the Evaluation
The purpose of the evaluation was primarily formative. We wished to find out the strengths and weaknesses of the materials and the kinds of changes that would lead to their improvement. We were also interested to learn the degree to which law-related education could be incorporated into the existing curriculum framework of schools, and of the reactions of teachers and students to this curriculum initiative.
We were particularly aware of the novelty of the subject matter to many teachers, and wished also to assess their levels of confidence and need for support.
Organisation of the Trials
We had originally hoped that the materials would be thoroughly tested as they began to take shape, within and between development groups. We envisaged that more extensive trials would then take place in approximately eight new authorities, drawn from those which had expressed initial interest in the project but which had not been able to support a development group.
Industrial action made it difficult, however, to negotiate arrangements with new authorities. The trials were, therefore, carried out mainly in the development authorities by both group members and teachers new to the project. However it did prove possible to include South Glamorgan as an additional trialling authority. (It had earlier proved impossible to establish one of the development groups in Wales, despite a sustained effort.)
During the development period a number of schools outside the project LEAs had heard of the project's work and had written to the team, expressing interest and volunteering help. I was agreed that these individual schools could provide a useful test of the usability of the materials in the absence of any close contact with the team (there would be no introductory meetings and no regular contacts). This strategy proved useful, since some of the teachers in these ten schools (dubbed 'postal trialling schools') turned out to be very willing to provide detailed criticisms of the materials.
In all a total of 55 schools and about 100 teachers in the nine LEAs were involved, plus ten postal trialling schools.
In each LEA teachers in six schools were invited to participate in the trials, most schools being offered a choice of one of two units. The exception to this was that one school in each LEA was offered all the material. From this the school was asked to select topics to create a term's course in law. The team called this 'core course trialling. Nine LEA schools were, therefore, invited to create their own longer course. We hoped in this way to learn more about the kind of courses that teachers would wish to offer in a six or ten week module when they had the chance of selecting material from any of the eight units.
We had a strong wish to minimise the disruption that the trials would cause to the timetable. Accordingly, we suggested that teachers should choose for themselves when the trials would take place and in what curriculum setting. In addition, teachers were encouraged to select only those topics which they felt would be worth using, and to comment later on why certain items had been omitted.
We believed that, since most teachers using the materials would be coming to the law with no specialist knowledge, they would probably need support and legal advice beyond that which the Teacher's Notes could provide. Accordingly, with the help of The Law Society, each of the trial schools was linked with a local solicitor, except in two areas where the solicitor who had worked with the development group agreed to provide advice to all the schools and to find a local solicitor if required.
As we have said, the purpose of the evaluation was primarily formative, and it was anticipated that both quantitative and qualitative data would be required. That is to say we decided to use closed questions to elicit responses that could be quantified, eg: 'yes/no' answers, and open-ended questions in which teachers were invited to offer their own responses or ideas. During the preliminary meetings with David Clemson, the consultant evaluator, a range of techniques was discussed.
Pencil and Paper Responses
A series of seven questionnaires was developed (see Appendix A) to gather information about the nature of the groups, and student and teacher responses to particular aspects of the materials. These were designed to indicate the general level of acceptability of each topic and not provide us with detailed ideas for improvements. In October 1986, a further general questionnaire was sent out to all teachers concerning their evaluation of law-related education generally and, more specifically, of the overall value of the project's materials. This was prompted by the need to obtain this information promptly for a submission to The Law Society for further funding.
The responses of teachers to the detailed aspects of the materials needed to be gathered rather differently. For this we used several techniques, namely: blank sheets for teachers' comments; a request for each item used to be annotated with teachers' comments; observation and discussion with teachers.
At the onset of the evaluation we had aimed to visit a minimum of one third of the schools and, in the event, 31 of the 55 schools were visited. In most cases the pattern was to observe a lesson and then to discuss the materials in general with all staff involved in trials within the school. Neither the observation nor the interviews were rigidly structured. In each case a checklist of important questions was uses and reports were written up and circulated within the central team as soon as possible after each visit. We used every opportunity to share and discuss our reactions to the lessons we had seen. This proved to be valuable way of analysing the strengths and weaknesses of the materials and allowed us to keep under review the methods with which we were conducting the evaluation.
With schools in nine different parts of England and Wales the logistics, time and expense involved in undertaking these visits were considerable. We were not able to see all the topics taught, nor did we necessarily see the lesson taught in the form that we had expected. On one occasion the teacher was absent from school and on others the class size had been altered, or the groups rearranged. Despite these uncertainties, however, every visit was worthwhile either in demonstrating the shortcomings or strengths of a particular approach, or in providing useful indications of teaching strategies in general. We learnt a good deal about group work and discussion techniques during this stage.
Reflecting upon the evaluation, now that the final editing and revision have been completed, it is clear that the classroom observation and meetings with teachers influenced our assessment above anything else.
Three meetings�at the beginning, during the middle and at the end of the evaluation period�were also planned in each of the nine authorities. The purpose of these meeting was to prepare teachers for the evaluation, to provide support, heighten awareness and obtain preliminary feedback. Members of the original development groups were invited to the final meetings in order to provide them with an indication of how the materials they had developed had been received and how they might be revised.
A specific purpose of the mid-trial meeting, besides gathering some initial responses and encouraging the sharing of problems and solutions, was to develop teachers' critical awareness of the materials. An evaluation exercise was conducted during which members noticeably became more aware of the shortcomings of the text under scrutiny.
We sought help over the accuracy and realism of the materials from a number of specialist organisations which gave the team considerable help. These were:
- British Institute of Human Rights
- Children's Legal Centre
- Committee for the Reform of Animal Experimentation
- Equal Opportunities Commission
- National Consumer Council
- Police officers from the Metropolitan, Thames Valley, Nottinghamshire and South Yorkshire forces.
Part two: Putting theory into practice
Part one of this report set out the broad lines along which the trials were organised. This information may prove useful to those considering a similar exercise. However, in an exercise of this sort, circumstances may conspire to thwart the original intention. Awareness of some of the possible problems may be of benefit in reducing their impact or eliminating them altogether.
Schools and LEAs involved in the trials
We had planned to involve about eight new LEAs in trialling the materials produce by the development LEAs. Establishing working relationships with new authorities can be a time-consuming business. Industrial action during 1985 made this almost impossible and new arrangements were made with only one authority. Six schools per LEA were invited to take part in the evaluation. By and large this was achieved with minor variations either way.
Ensuring Even Coverage
It was recognised that to some extent the trialling would have to fit into each school's existing programme and so the more choice the schools were offered the better the likelihood of take-up. However, this had to be balanced against our wish to ensure as even a coverage as possible.
A compromise formula was developed�each authority would be offered two units from which schools were asked to choose one. In some authorities this strategy worked whilst in others it was less successful. In one LEA all schools trialled the same unit, leaving the alternative unit under-represented overall. The character of courses into which the materials were to be inserted also seemed to play some part in unit selection. The most popular choice of unit was implementing the law, which covers aspects contained in many existing PSE and humanities courses.
In this respect the 'postal trialling schools' proved very useful as a means of achieving more even coverage. These outside schools were offered only units which were under-trialled. Without this 'fall-back' the evaluation process could have been more seriously impaired.
Teachers Involved in the Trials
Each school was asked to nominate two teachers to try out the materials, since two teachers would be able to support and motivate each other, bearing in mind that the process of evaluating new material involves a considerable amount of extra work. The involvement of more than two teachers in each school would have:
- been difficult to achieve;
- placed too great a burden on the central team in processing all the additional information.
Many schools were able to provide two teachers as requested. However, circumstances in individual schools varied considerably. In one LEA the beginning of the trial period coincided with the complete reorganisation of the authority's secondary schools. This meant that many teachers were working in entirely new schools, getting to know new classes and, for most of these, trialling the project's materials was a low priority. Indeed, it was very difficult for the central team to know which schools had what material and who was supposed to be evaluating them.
In twenty of the fifty-five schools, only one teacher trialled the materials, although this was generally with more than one class.
Despite the fact that the trials were being conducted within the development authorities, a variety of circumstances seemed to prevent many members of the development groups from being involved in the classroom trials. In fact, out of 105 teachers (not including the 'postal school' teachers) only 18 had been members of the original groups. This meant that 82% of trial teachers were new to the project. One of the reasons for this appears to be the fact that members of the original groups were drawn from a wide range of subjects, including, in one case, French, whilst the trials were mainly confined to PSE and humanities classes.
The Problem of Getting Feedback
This was recognised to be the major problem of the evaluation. Experience of previous projects suggested that it would be unrealistic to expect anything approaching 100% feedback. The questionnaires were, therefore, designed to be completed with the minimum of effort. We tried not to ask for information that we would be unlikely to use and tried to find ways of increasing the teachers' sense of involvement through the meetings held at the beginning, in the middle and at the end of the trialling period. It was also felt that visits to the schools would be valuable in increasing the teachers' sense of involvement as well as being one of the most valuable sources of information on the materials.
In the end 84% of schools trialling the single units returned evaluation forms. Seven of the 'core-course' schools returned evaluations. About 60% of schools were visited. For obvious reasons, it is not possible to know why teachers did not return the questionnaires despite reminders and the provision of 'freepost' addressed envelopes. We know that one authority was in turmoil at this time and can assume that timetable requirements and the ordinary stresses and strains of the job will have prevented some other colleagues from trialling. The introduction of GCSE also increased the burden on teachers at this time.
Part three: The main findings
What did Teachers Think about the Value of Law-Related Education?
There was widespread acceptance of the view that law had a valid place in the curriculum of 14-16 year olds.
Two teachers put it this way:
"We think that this is a necessary part of preparing our pupils for the transition from school to work and from adolescence to adulthood."
"16 year olds should have a basic knowledge of law and their rights."
Some saw the need for legal knowledge to be built in to the curriculum to ensure that all students were given this information:
"I think an element of law-related education should exist in the curriculum of all children in the 14-16 age range on any pre-vocational course if not, in fact, as a core element of all students' curriculum. LRE should be seen as an important area of the curriculum and therefore be properly recognised, funded and staffed."
Most teachers saw the social value of raising the legal awareness of young people:
"The materials have helped students form a new perspective of the law, a positive one, in which they see the law as being reflective of social need."
"It poses them real life situations that they are able to identify with and the tasks provide opportunities to explore solutions. Above all, it helps foster positive attitudes to social responsibility! As education is encouraged to be relevant, an awareness of the laws which govern us all is crucial if young people are to assume their social duties."
"Using this material has shown me how essential a basic understanding of the law is for responsible citizenship. I am amazed that schools and parents have not seen the need for this before."
"Much of the content is valuable in that is shows the law in action rather than the law as a dormant set of rules to be obeyed. It allows pupils to see that the law can be used by them and not merely used against them! Pupils have a right to know how they are affected by the law in a passive sense and, perhaps more importantly, how they can 'empower' themselves and actively enlist the help of the law."
Individual topics were also seen as being of similar value, as in the following comment on the topic Parents and Children:
"The discussions as to how far the law should interfere or impose on family relationships were particularly useful. Protection of individuals, and public policy were particularly discussed."
What did Teachers Feel about the Selection of Legal Issues?
Some teachers were coming to law-related education for the first time but there were many involved in the trials who already included elements of law either in personal and social education (PSE) or in other courses. Commonly, the emphasis has been on deviancy and the criminal law. Teachers frequently commented on the value of the project's broader approach:
"My school has taught a law module of six weeks to all 4th year pupils several years. In the past we have concentrated on criminal law, enforcement and the court system. However, I would be very interested in extending this to include other areas covered by the Law in Education Project."
"The material has brought new ideas to the work we do in social studies and helped to increase pupils' awareness of the law in their everyday lives."
"I found the material to be useful in that if fills a gap that has existed in educational literature. The content is interesting and important in that it helps to make the pupils aware of situations that previously they had not imagined existed."
At no time did the team encounter the view that the law had no place in the curriculum or that the project's broad approach, of including aspects of civil and criminal law, was misjudged.
What did Teachers Think about the Educational Value of the Materials?
This covers not only the factual content of the topics but the teaching strategies by which the ideas were conveyed:
"I have been particularly grateful for the practical classroom help the Law in Education Project trial materials have given me. It is well designed and usable."
"The material is excellent�detailed and very thoroughly prepared. It is particularly valuable because of the stimulating variety of approaches suggested."
Many teachers appreciated the methodologies adopted by the project in that they encouraged the development of a range of skills. This they saw as being in line with developments in other curriculum areas:
"The material is good, and provides stimulus for discussion on a wide range of topics. It also provides the opportunity to acquire and practice a range of skills, eg: problem-solving, data analysis, discussion, oratory and many others."
"I have used the materials The Law and the Road User. The materials are excellent because:
- they encourage students to think and research;
- they promote experience of concepts;
- they lead to active class discussion."
"I was especially pleased with the promotion of discussion and tolerance of other people's opinions."
The final question on Questionnaire 2 (Appendix A) asked teachers if they would teach the topic again given the opportunity. 85% (n=152) replied Yes.
Did Teachers Think that Their Students Found the Materials Interesting?
A head of social studies wrote:
"The main thing that teachers in my department want is interesting, 'teachable' material that shows the points in an exciting way. What we have used so far from this project (Family and Consumer) does exactly that."
Another teacher commented:
"It offers a different approach to a difficult subject. My pupils have shown great interest."
"Because it was a subject that was so relevant to their own lives, it was of great interest to them; they had a great deal of fact and opinion to offer to the discussion."
On Questionnaire 2 teachers were also asked to respond to the statement "My pupils found the subject of this topic interesting." On 181 questionnaires, 91 of teachers replied 'Yes', whilst on 9% they replied 'No'. One topic out of the 41 trialled was widely criticised as long and boring (How laws are made and changed) and another was very confusing (Consequences). Together these accounted for the majority of the 'no' replies to the questionnaire.
Did Teachers Find the Materials Suitable for Mixed-Ability Classes?
Teachers confirmed that students of a wide range of ability were able to handle the materials at their own level. Successful trials were reported amongst grammar and public school students and high ability groups in comprehensive schools. At the same time, positive responses were reported from teachers of less able groups. One teacher, of a 'difficult' group of CPVE students, found that the level of debate stimulated by the materials on the family, during which students became very involved in drafting their own law, was 'quite remarkable'. In another school, two teachers used the same material with a sixth-form group and a low-achieving fifth year class. The comparison was interesting:
"The unit that I have used I have found to be excellent. It has been used with 6th form as a stimulus for further self-directed study and with a low ability 5th year group where attendance was notably higher for these lessons than some others."
Another teacher reported the 'best lessons all year' with a difficult 4th year group. He wrote:
"Parents and Children stimulated a lively response and interest with previously poorly motivated fourth year RE classes. It linked well with work on moral responsibilities."
We carried out reading tests, on a sample of materials, revealing the reading age of the materials to be around 13 years on average, according to Fry's readability graph. However, when observing the lessons the team repeatedly asked whether the reading level of the materials was suitable for students. Overall teachers confirmed that the materials were usable in mixed-ability groups. Nevertheless, one or two topics were discovered to be more difficult than others. Several factors were important in determining the difficulty than others. Several factors were important in determining the difficulty of a piece besides the reading age. Of particular importance was the quantity of reading that students were asked to do. In addition many of the topics included lists of cases for discussion. Several of these lists were too long. By the time students had studied three or four cases, boredom was setting in. The discussion sheet Work and the Sexes was criticised on similar grounds. One observation report noted:
"It was criticised for being rather long-winded, repeating the same ideas. Could have used fewer photographs. Felt some of the issues lacked relevance. Text hard to follow."
The result of these criticisms was the complete re-writing of How Laws are Made and Changed and the modification or deletion of other topics as appropriate.
In What Ways Were the Materials Criticised?
Qualitative and Quantitative Data
As outlined above the main purpose of the evaluation was to produce the best possible materials. To do this critical comments were required from teachers and observers. They came in written form or were given to us orally on an observation visit. The trials teachers were being asked to criticise relatively unfamiliar materials, usually after teaching them only once. This requires an unusual degree of objectivity since teachers' concerns generally appear to focus on questions of classroom management and pupil interest rather than on matters such as clarity of text, sequencing of information and so on. In the normal course of lessons, teachers are compensating for weaknesses in teaching materials so much that they appear to be relatively unaware of the process.
On the whole, the trials teachers seemed much less aware of the shortcomings within the materials and offered fewer suggestions for improvements than we had hoped. Several reasons for this can be advanced. The subject matter of the materials was not one with which many were familiar. Perhaps their involvement with the project predisposed teachers to give a favourable response. If they had thought the materials were poor, would they have imagined the fault lay with them rather than with the materials? Criticism of colleagues' work is not a normal feature of teachers' work and there appeared to be a general reluctance to be openly critical.
Such considerations should be borne in mind when setting up classroom evaluations and when processing data. They would seem to suggest that considerable training of the trials teachers (for, say, two days) would be beneficial in raising awareness and developing the motivation to become involved with a process which can place very heavy demands on busy people and call for skills which may be relatively undeveloped.
It follows from the above considerations that quantitative evaluation data is unlikely to provide a sufficient basis for those refining the materials. If a topic were regarded as good by the majority (say 85%) of teachers would that mean it should not be improved? It might, as suggested, say more about those evaluating than what is being evaluated.
We felt from an early stage that the kind of evaluation data likely to be most useful would not be found in majority comments but in the particular insights of individual teachers. Indeed, it appeared that the most helpful comments would almost by definition be minority elements.
For example, only one teacher made the following point about the material on judge-made laws which came as part three of How Laws are Made and Changed:
"At this stage in the lesson the introduction of the Gillick case seemed a 'throwaway'. What was a good, interesting piece of work did not stimulate the pupils towards the end of the hour. The piece is good enough to stand as a lesson in its own right. Because it was seen to be tagged on the original piece, it unfortunately did not succeed in achieving its aim of contrasting judge-made and parliamentary law."
This perceptive comment was heeded and the piece in question was moved to the Family unit where the main interest, ie the content of the law not the legal process, would be emphasised.
Different Kinds of Expert
Naturally, teachers varied enormously in the style of evaluation comment they offered. Some were brief, others elaborated points at much greater length. One striking difference was the difference between the comments of the non-expert teacher and those with a legal background. For example:
"The first half went well, but 'Patient Rights' was less successful as they did not wish to give patients many rights and they did not find the answers relevant. Shortened section on admission to hospital."
As compared with:
"I would much preferred the explanation of negligence to have been co-ordinated here, in The Accident and in Consequences. If the principles had been explained in similar terms with only their application differing, the pupils would get an overall picture of the tort. I already use the three cases in my teaching of negligence. I was hoping for the exploration of an area of hotter legal debate like, eg, the extent to which a surgeon must inform the patient of possible risks of an operation."
Comments from both kinds of teacher were valued. The comments from law teachers were particularly valuable to the team who themselves had not appreciated all of the points made. In retrospect, it might have been helpful to have had at least one law teacher as a consultant to each of the development groups to provide this kind of critical evaluation, although it is unclear what effect this would have had on the confidence of the other teachers.
Initially, we were hoping for the trial materials to spark off many alternative and better ideas and blank sheets of paper were provided for this purpose, as described above. In fact the number suggestions was rather low, the following comment on A Case of Dismissal being an exception rather than the rule:
"Good idea, but did not work. Unless you have a very bright group, taking on roles in the way suggested is surely going to be difficult. Better perhaps to act out a scripted play and get the rest of the group to act as a jury."
Comments on Layout and Presentation
Teachers were also asked about the presentation of the trial materials and a wide range of responses were received. Many teachers expressed the view that the materials could have been more varied in their presentation, employing a range of media to avoid what one teacher called 'bookletitis'. In particular, video material would have been welcomed. (It had been originally hoped that each development group would have been able to develop some audio-visual material. Although two groups produced outlines for video programmes, there was not time to bring these ideas to fruition. The team also put some time into a tape/slide programme on health and safety at work which was never completed.) Many appeared quite satisfied with the general standard of design and layout of the materials whilst others criticised them severely as having a visual monotony:
"The basic concept is sound, has interest but the presentation is appalling. Had pupils produced this they would quickly have been told to improve presentation and layout!"
This criticism, though harsh, highlights a problem for evaluators. To what standard should the trial materials be produced? Clearly they cannot achieve the sophistication of published materials and yet the quality of the design and layout will have a material effect on their reception. It appears that some teachers were expecting materials of a higher standard and were reacting accordingly. It was impossible to unravel from observations the effect of the layout and design and yet it undoubtedly is a hidden factor.
On the other hand, whilst the materials obviously should not appear to have been written the night before, there appear to be disadvantages in creating too sophisticated a product for evaluation, quite apart from the immense of time and expense involved. The more polished and authoritative the materials appear, the less confident teachers may be in making criticisms. Trial materials that have the appearance of a final product might also convey the impression that criticism is to late and that only minor amendments can be incorporated.
Comments on Structure and Sequence
Several criticisms were made of the position of the answers in relation to questions and problems. These were frequently on the following page. Trial booklets had been stapled together for convenience and most teachers used them in this state instead of breaking them up and distributing them to classes as required. One teacher wrote:
"On the whole, very good, but present structure of Growing Up booklet useless�whoever put the answers in with the questions?"
Comments like these convinced the team that the most appropriate format for the materials was one in which the teacher could delay the distribution of certain sheets until certain tasks had been accomplished. This suggested a photocopy master format rather than a textbook. This realisation was consistently confirmed during many discussions with teachers on the subject. The cross-curricular nature of the materials and the low budgets of some of the target departments also pointed to the same conclusion.
Another question that faced the writers was how to structure the legal information in a way that would provide the necessary reference material, without pre-digesting the information into potted answers which would require no research skills on the part of students. Many of these 'legal panels' appeared to have been pitched at the right level but more than one teacher criticised the legal sections of Growing Up because of the mass of information conveyed and the difficulties faced by students in extracting the relevant information:
"The low-ability 5th year I used this with found sorting out the answers from the mass of information given difficult! More pictorial presentation to perhaps do away with the mass of complicated reading�comic strip type presentation or simple diagrams or simpler language would help."
As already mentioned, the value of lesson observation impressed itself on the team greatly. In watching a lesson, not only was it possible to appreciate the shortcomings of the materials as drafted, it often seemed to be easier to visualise the solution to the problem. This is illustrated in the following extract from an observation report:
"In the first lesson, Pedestrians Only was tackled by the students copying the original bylaw onto their page and then individually rewriting and changing it in the light of each of the complaints. This seemed to me to be laborious and some of the weaker students found it confusing. With less time available in the second lesson, the class considered the exercise as a whole. The teacher tried to draw out ideas and opinions from each section of the class. There were some excellent ideas and again the students' attention was retained. As a third Alternative I suggest that students could be given the complaints as if they had all arrived by letter at the same time. They then have to sift through these, and decide what they will do. Their revised bylaw is produced in the light of all these, and is written out only once." (TT)
What did the Students Themselves Think of the Materials and LRE Generally?
A central aim of the materials is to raise the students' legal awareness and yet it was often not easy to discover whether the materials succeeded in this. Occasionally the teacher, being new to the subject, was not able to comment on this evaluation question with any clarity:
"I found the materials developed my students' factual knowledge of the law, but I'm not sure to what extent, because my knowledge of consumer law is limited and this was my first experience of teaching law. I found the material developed my students' understanding of the nature of law but again inexperience on my part meant that I was not sure to what depth I had to cover the legal aspect."
In contrast there were occasions when clearly the lesson had been very effective in developing students' understanding of the law�particularly in areas they saw as relevant, such as part-time employment and consumer law. This is demonstrated in this observation report:
"She used Out of Hours which went very well, nearly all her group had jobs and were very interested to find out that some were not in accordance with the law. One girl had an invalid work permit." (DR)
Another teacher made this comment:
"Very worthwhile in informing students of their rights as consumers. Almost all had experienced or known someone who had experienced an infringement of consumer rights. At least two students informed me that they had been able to quote some of the law involved when returning unsatisfactory goods."
Testing the students themselves might have been a way of gaining more insight into this important question. The possibility of pre- and post-tests was discussed by the team, but was rejected as being constrictive and encouraging teachers to teach to the test. The team required a more open-ended approach to the materials and furthermore were not primarily concerned with teaching legal facts. Legal awareness, if assessable at all, is clearly a much more complex matter to evaluate, incorporating, as it does, a number of strands including attitudes, understanding of broad concepts, legal research skills and so on.
Of the eight core-course schools, four returned 167 questionnaires completed by their students. No two schools had taught the same combination of topics. Apart from commenting on the specific content of what they had studied, students were also asked:
- "Did you enjoy this course on law?"
- "Do you think learning about the law is worthwhile?"
Responses were graded on a six-point scale (YES!; Yes; yes; no; No; NO!). 82% of students said they had enjoyed the course to some extent, the largest response (45%) being in the 'Yes' category.
Asked if they felt that learning about the law was worthwhile, 165 students (99%) replied in the three 'yes' categories. Of these 80% felt either sure ('Yes') or very sure ('YES!') that the law was worth studying. Students gave a wide variety of reasons why they thought learning about the law was worthwhile. Amongst the most frequently quoted reasons was that it was 'useful to know' or 'helpful'.
How Did Non-Expert Teachers Cope with Teaching Law?
Only four of the trials teachers were law teachers which suggested that we should look carefully at how well the materials supported the non-expert teacher. Several teachers expressed a certain amount of anxiety concerning their own legal knowledge and their ability to assess their students' learning; Two comments received were:
"We both feel we could have been better informed on the legal intricacies and our limitations could have been a restraining facto in the pupils' learning."
"�inexperience on my part meant that I was not sure to what depth I had to cover the legal aspect".
Much of this uncertainty was due no doubt to the newness of the materials. Teachers would be more familiar with the lessons when they taught them again and would be more aware of the areas of difficulty. The Teacher's Notes had tried to anticipate the main areas where extra information might be required. The questionnaires revealed general satisfaction with these.
How Successful was the Scheme Linking Each School with a Solicitor?
We had expected teachers to welcome the chance to discuss legal problems raised by the materials with someone locally and so each school was linked with a local solicitor who agreed to act as a 'legal friend' and, if appropriate, to take part in classroom activities.
To ease the establishment of links between teachers and solicitors, we wrote to each solicitor personally thanking him or her for agreeing to cooperate and sending copies of the material being taught by the school with the name and address of the local lawyer and was urged to make contact.
In all 50 solicitors were recruited to the scheme. In one authority, the development group solicitors agreed to answer any local queries and, if asked, to link local schools with a solicitor of their own. In the event many teachers found that their materials raised no great legal problems and therefore they made no contact with their solicitor. In other cases, the newness of the materials meant that teachers were uncertain of themselves and to some extent reluctant to involve an outsider in a situation where they felt less than fully confident. One teacher put it this way:
"The failure to contact our solicitor earlier has been totally the fault of the school. We have been reluctant to involve a solicitor in an experimental situation. In retrospect general discussion of topics before teaching would be most advisable�but there was no time available. There are many questions both general and specific which I should like to discuss before repeating the course".
Other comments suggested that some teachers were aware that solicitors would have to come into school during the working day, which would mean a loss of earnings. It was not possible to determine whether the solicitors' professional aura deterred any teachers from establishing contact.
Ten schools (roughly one in five) made contact with their solicitor, either by phone or in person. Not all contacts were followed by school visits. For most classes these did not appear appropriate. Teachers in one school found that their solicitor could not come into a morning lesson because of court work and in two other cases, appointments made could not be kept. The number of contacts was lower than anticipated but this was an experimental situation and there was no way of anticipating what the level of take-up would be. The low level of contact may be partly explained by the short time available for many of the trials and the uncertainties involved.
In What Courses Were the Materials Used?
The MORI poll carried out for the Law Society in 1980 had identified 12 different curriculum area said by teachers to include elements of law. These included history, home economics, religious and moral studies, business studies, social studies, humanities and, of course, law, but the majority of respondents felt that the obvious place for such material was personal and social education which, despite a variety of names, is common to the curriculum of the majority of schools. Would this picture be confirmed six years later?
Of the 80 classes for which we have data, by far the largest course category was personal and social education (PSE, 28). If we add to this number those classes which were clearly similar in character (non-examination and with a 'life-skills' orientation) the total increases to 40. These other classes were described as:
Teachers' own comments throw light on the kind of courses which were able to incorporate the trial materials into their skills-oriented programmes.
Teachers' own comments throw light on the kind of courses which were able to incorporate legal elements:
"The content of the course includes a whole range of topics vital to an effective and relevant curriculum for Personal Development / Social / Skills / Preparation for Living courses and seems well suited to the developing pre-vocational courses in year 4/5 and the CPVE course in year 6."
"The material on The Law and the Individual has proved very valuable in the context of Integrated Humanities GCSE teaching, as part of the 'Freedom and Constraint theme. Although the individual element has been stressed during trialling, I would envisage extending the legal theme to include consumer education in a new modular approach to the curriculum."
"We have used several units of the project in a GCSE Social Science course. They have been of great value in making the area of the family both interesting and relevant. Teachers of a City and Guilds course have also shown interest in some of the units."
"It has value in courses on law, history, PSE and allied courses and has been used by us in O level / A level General Studies. A set (or more) could be used and made available to all departments in the school, including child care courses an careers departments".
The question whether to establish a discrete module or 'infuse' the material into existing courses was faced by many teachers. Opinions varied, as this remark indicates:
"I wouldn't wish to use the law-related materials as an end in themselves, but rather teach law-related education as an integral part of other topics, eg business studies / consumer law, as I feel that taken separately some of the topics are viewed as being rather dry and irrelevant".
The evaluation also provided evidence from a number of sources that, both inside and outside of the examination system, teachers are able to adapt and develop courses to incorporate new topics or themes:
"We have always had a law component in our personal and social development course, so the Law in Education Project has been a very useful resource to back up this course. Next year the materials will be introduced in TVEI, and the City and Guilds 365 course. We are about to develop the Adults Other Than Teachers Project with links with solicitors and Hants police:
"We are already including extra provision for law education in next year's curriculum as part of TVEI type modular courses. It is considered to be so important that every pupil in their final two years will have the opportunity of following such a course".
Part four: Reflections and conclusions
- Questionnaire responses provided general indications of teachers' responses to the materials under evaluation. They produced, however, relatively few detailed and developed suggestions for improvement or alternative strategies. Similar responses to certain items were a useful indication of the strengths or weaknesses of a particular topic, but even general praise for an item did not necessarily preclude the need for detailed improvements.
- The observation of lessons was very beneficial. Not only did it help us judge the effectiveness of the materials from the points of view of both teachers and students, but also provided us with a valuable insight into many classroom contexts in which law-related issues are handled. Further, it enabled evaluation data to be set in the context of a particular school and class.
- Teachers do not evaluate materials in a vacuum�but in terms of the ways in which they serve their own needs. Their comments tend to be shaped by such things as pupil response, the subject setting, and the nature of the class. For these reasons it is not possible to avoid the necessity of evaluating the evaluators.
- Trialling materials can be extremely time-consuming and immensely complicated logistically. Adequate time for teachers to be inducted into the processes and for them to prepare is essential.
- The evaluation showed that the idea of LRE found a high level of support amongst teachers.
- The numbers of teachers who were intending to "teach the topic again" indicated that despite their faults the materials fulfilled a need and were regarded as appropriate to the target age group.
- Both students and teachers saw the materials as motivating and high in interest. Students were almost unanimous in regarding learning about the law as worthwhile.
- Despite some initial uncertainty because of the novelty of the content area, a large majority of teachers were satisfied with the structure of the material and their own ability to handle it, as non-experts.
- There was widespread acceptance of the value of teaching law at both a personal and a social level. Many teachers endorsed the project's view that the curriculum has in the past placed undue emphasis on the criminal law.
- The evaluation clearly demonstrated that, given a carefully structured approach, elements of law can be taught to students of all abilities and some schools expressed the view that law should be offered to all students as an entitlement.