Introduction by James R. Vivian

As the movement nationally for university-school collaboration continues to gain momentum; as this movement becomes more sharply focused on excellence in teaching in public schools; and as numerous collaborative programs, some with the Institute’ s assistance and following the Institute’s model, are being established in other communities, it is vitally important that we deepen our understanding of the ways in which such programs can strengthen teaching and learning in public schools. If, in fact, this movement is to be sustained, in New Haven and elsewhere, collaborative programs must present persuasive evidence of their results. The Institute therefore has a responsibility as one of the most visible collaborative programs, as well as an opportu nity as one of the programs of longest duration, to make a significant contribution to educators and policy makers working in this vein.

Since its inception, the Institute has acted on the belief that ongoing evaluation by participants and others is indispensable to the continuing development of the Institute and to ensuring that the program remains responsive to the needs of New Haven teachers and their students. From our perspective in New Haven, the results of the numerous evaluations of the Teachers Institute conducted thus far offer real encouragement that such collaborative programs can assist schools in specific ways.

The present report reflects the analysis of data from teacher surveys which the Institute developed and administered throughout the New Haven Public Schools during a nine-year period. These surveys include questionnaires administered in 1982 and in 1987 to teachers who at any time had been Institute Fellows as well as many who had not participated, and questionnaires completed by teachers participating in the Institute in each year between l986 and l990.

This “Progress Report” was prepared for discussion initially at a national conference on “School–College Collaboration: Preparing Teachers and Curricula for Public Schools,” which the Institute organized and held at Yale University in December 1 991. The conference was attended by representatives of the several main audiences for the results of the Institute’s evaluation efforts: that is, teachers and administrators from New Haven and Yale, and from numerous other schools and colleges across th e country which are planning or conducting similar programs; as well as other educators, policy makers, and funders interested in the Institute’s type of collaboration. Responses from these different contituencies—whose needs for information may differ—w ill contribute very directly to our ongoing consideration of the value of the preliminary results presented here, and to the plans we will make for the further evaluation of the efficacy of the Institute’s approach. This document is therefore correctly t itled as a report on our progress, and not on our definitive conclusions.

James R. Vivian

Introduction to the Teacher Surveys

A brief chronology of the teacher surveys will help to establish the findings described in this report in the larger context of evaluation of the Institute. During the first few years of the Institute, its staff developed a series of open-ended questions to elicit information about how Fellows assessed their Institute experiences. Thus, a review of their Institute experience by Fellows has always been an important feature of the overall evaluation strategy of the Institute. Between 1978 and 19 81, Fellows’ comments were used mainly for internal purposes. Responses to the open-ended questions contributed to ongoing self-analysis and review of the program and to the revision and refinement of program elements. During the same period, seminar le aders, independent evaluators, and the director of the program also reviewed the program in written form. The reactions of all of these individuals appeared to cluster around a set of themes which formed the basis of the first “formal” teacher questionna ires, developed for administration in 1982. The themes were augmented by topics in the questionnaire that emerged from reviews of the literature in areas of education that seemed particularly germane to the Institute.

The purpose of the teacher questionnaires was to provide a comprehensive examination of the influence of the Institute on teaching and learning in New Haven middle and high schools. The questionnaires gathered more systematic data than was possi ble from the open-ended questions alone (although the annual evaluation by Fellows continued until 1985 in mainly open-ended form). Moreover, the questionnaires were not administered only to Institute Fellows. Separate questionnaires were developed for administration in 1982 to teachers who had participated as Fellows of the Institute and to teachers who may have used curriculum units created by the Institute but had not participated as Fellows. The questionnaires were reviewed by 12 Institute Coordina tors and a school administrator, revised to reflect the concerns of the reviewers, and administered by Institute Coordinators in the schools they represented.

The 1982 questionnaires were revised for a second administration in all New Haven schools in 1987, five years after the initial system-wide survey. Three separate but parallel questionnaires were created for the survey: one for Fellows of the I nstitute in 1986 (and, by extension, Fellows in future years); one for all middle and high school teachers still in the system who had been Fellows at least once during the years between 1978 and 1986; and one for all teachers of academic subjects encompa ssed by the Institute, who had never been Fellows. The focus of the questionnaires was expanded once again with this survey. In addition to the questions that had been asked in previous surveys, questions were added to reflect issues of national concern and to enable comparisons with sources of national data about teachers. Thus, questions were included from surveys that had been conducted by the Gallup Organization, the National Assessment of Educational Progress, the National Education Association, a nd Educational Testing Service.

There are, therefore, three strands of evidence represented in these pages. One comes from the annual evaluations by Fellows of their Institute experience. Between 1978 and 1985, these evaluations took the form of responses to open-ended questi ons. Beginning in 1986, the evaluations were recorded in questionnaires that included both fixed-choice and free-response questions. At that time, many of the free-response questions were re-formulated as fixed-choice items. A second strand involves da ta from questionnaires administered in 1982 to Institute Fellows and to middle- and high-school teachers in the New Haven system who used Institute-developed curriculum units. The third strand is from an even broader system-wide survey conducted in 1987 t hat included former Fellows and non-Fellows.

Structure of the Questionnaires

The 1987 questionnaires consisted of questions grouped into sections according to topic. The first section asked about the educational background and teaching experience of the respondents, both Fellows and other teachers: their undergraduate a nd graduate degrees and certificates, their major fields of study, the kinds of classroom materials they were using, the subjects in which they were certified to teach and which they felt best prepared to teach, and the types of professional growth activi ties in which they had participated in the years immediately preceding the survey. A second section asked respondents about their attitudes toward the teaching profession and toward their own teaching experience, including their reactions to various reco mmendations for improving public education, their morale, and the degree of influence they perceived that they had over what they taught.

A section of the questionnaire completed only by former Fellows asked about their experience with the Institute and for their retrospective evaluation of selected features of the program. A series of questions was based on statements that Fellow s had made in previous years about the changes that may have resulted from their participation in the Institute. Former Fellows were also asked about the units they had prepared when they were Fellows, and about their subsequent use, revision, and dissem ination of the units.

Both Fellows and non-Fellows were asked about their use of and reaction to units prepared by Fellows (teachers) other than themselves, and about their perceptions of student reactions to the units they had taught. Both Fellows and non-Fellows we re asked to express their attitudes toward Yale University and its faculty, and to indicate their interest in participating in the Institute in the future.

Finally, all respondents were asked a series of demographic questions intended to characterize their social and economic background. The purpose of these questions was to enable us to compare teachers who have been Fellows with those who have no t.

The questionnaires just described were intended for use throughout the New Haven school system. They were developed concurrently with a new questionnaire for Fellows that would be administered each year. Developing the two sets of questionnaire s simultaneously enabled us to ask many of the same questions of the several groups of respondents (current Fellows, former Fellows, and teachers who had never been Fellows), but also maintained the tradition established in 1978 of recording the testimony of participants at the conclusion of each year’s seminars. This annual census of participants is one of the most valuable forms of evaluation that we have undertaken. The goals of the new set of questionnaires were to transform some of the formerly ope n-ended questions to multiple-choice questions and to replace some of the earlier questions to which answers had become predictable.

The new questionnaires included, in addition to many of the same questions (described above) asked of former Fellows and non-Fellows, questions about Fellows’ teaching, including the grade levels and subjects they had taught, the instructional ap proaches they employed in their classrooms, and their inclusion of various competencies and skills. Many of these questions were suggested by the literature on teaching (Adler, 1982, 1983; Sizer, 1984; and the College Board, 1983-1986). We also asked Fe llows for their reactions to particular approaches to educational reform derived from a review by Institute staff of major education reports issued between 1982 and 1986.

A major section of the Fellows’ questionnaire asked about Fellows’ experience in the Institute: reasons for their participation, responses to specific features of the Institute program, and their use of University facilities and resources. Anot her major section was devoted to fairly specific questions about the curriculum units they developed.

Finally, we retained three open-ended questions to elicit Fellows’ comments about their Institute seminar, the potential influence of their participation on their teaching, and their perceptions of the strengths and weaknesses of the program.

Reviews and Revisions

The 1982 questionnaires were pre-tested by 12 Institute Coordinators and reviewed by the Supervisor of Staff and Organizational Development for the New Haven Public Schools. The questionnaires were revised to reflect the comments and suggestions of these individuals.

The 1987 questionnaires were developed through a process of reviewing the responses to the 1982 questionnaires which included re-working earlier open-ended questions into multiple-choice form. Existing multiple-choice questions were retained fro m 1982 to 1987 to allow for comparisons between the results of the two surveys.

The draft questionnaires were reviewed by Institute staff and Coordinators, and a survey developer from Educational Testing Service.

Administration and Response Rates

The 1987 system-wide questionnaires, introduced by a cover letter that explained the purposes of the survey and assured the confidentiality of respondents, were administered by Institute Coordinators within the schools that they represented. Onl y teachers who were potentially eligible to become Fellows completed the system-wide questionnaires. Teachers in fields not covered by the Institute (e.g., physical education) were not surveyed. Completed questionnaires were received by 75 percent of th e Fellows (83 individuals) and 57 percent of the non-Fellows (183 teachers).

Because Fellows do not receive their stipends until they have completed their questionnaires, response rates to the Fellows questionnaire were 100 percent in each of the years between 1986 and 1990. The numbers of respondents in each year were 5 0, 54, 44, 63, and 70 respectively. Some of these respondents appear more than once in the data, since they completed questionnaires for each year of their participation in the Institute.

Between 1978 and 1988 the Institute served middle and high school teachers exclusively. Only in 1989 on a pilot basis--and then in 1990 in a regular way--did the Institute include elementary school teachers as Fellows. Most of the present repor t, therefore, concerns the experience of middle and high school teachers in the Institute; references to elementary school teacher participants reflect data only from the last two years reported here. Because of their recent inclusion, the proportion of elementary school teachers who have taken part in the Institute is therefore much smaller than the proportion of middle and high school teachers who have participated.