University of Massachusetts at Boston
Graduate College of Education
Critical & Creative Thinking Program
Seminar on
Evaluation of Educational Change
CrCrTh693
Spring 2007
Syllabus
Instructor: Peter Taylor, Critical & Creative Thinking Program
Email: peter.taylor@umb.edu
Phone: 617-287-7636
Office: Wheatley 2nd flr 143.09 (near Counseling & School Psychology)
Class: M 4-6.30, in McC 2-628C (facing out into the main corridor)
Office/phone call hours: M 2-3.40 & 6.40-7 by sign up,
or by arrangement.
Course Website: http://www.faculty.umb.edu/pjt/693-07.html
Course wiki: http://cct.wikispaces.com/693
General email: Emails sent to cct693@lists.umb.edu go to everyone in the course.
E-clippings: Clippings from the internet sent to cct693clips@yahoogroups.com
will be archived for all to read at http://www.yahoogroups.com/group/cct693clips
CATALOG DESCRIPTION
This course covers techniques for and critical thinking about the evaluation
of changes in educational practices and policies in schools, organizations,
and informal contexts. Topics include quantitative and qualitative methods for
design and analysis, participatory design of practices and policies, institutional
learning, the wider reception or discounting of evaluations, and selected case
studies, including those arising from semester-long student projects.
COURSE DESCRIPTION for Spring 2007
Theme: Cycles and epicycles of Action Research
First note that in this course "educational change" is construed broadly
to include organizational change, training, and personal development, as well
as curricular and school change.
This course builds on the question: "If you have good ideas how do you
get others to adopt and/or adapt them?" (in other words, how do you build
a "constituency" around your idea). This concern can lead you into
evaluating how good the ideas actually are (with respect to objectives you formulate)
so you can demonstrate this to others. It can also lead you to work with others
to develop the idea so it becomes theirs as well and thus something they're
invested in. In any case, the first person in your constituency is yourself!
To prepare you for such work, class activities introduce tools for group facilitation,
participatory planning, reflective practice, and systematic evaluation. We do
all this within a framework of "Cycles and epicycles of Action Research."
PREREQUISITES: For CCT students, this course is best taken after the
Practicum: Processes of Research and Engagement.
ACCOMMODATIONS: Sections 504 and the Americans with Disabilities Act
of 1990 offer guidelines for curriculum modifications and adaptations for students
with documented disabilities. If applicable, students may obtain adaptation
recommendations from the Ross Center (287-7430). The student must present these
recommendations to each professor within a reasonable period, preferably by
the end of the Drop/Add period.
Students are advised to retain a copy of this syllabus in personal files
for use when applying for certification, licensure, or transfer credit.
This syllabus is subject to change, but workload expectations will not be
increased after the semester starts. (Version 5 April '07, with incomplete links)
SECTIONS TO FOLLOW IN SYLLABUS
Texts and Materials
Requirements
Schedule of Classes and Preparation
Reports from Students' Projects
Bibliography
TEXTS and MATERIALS
Calhoun, E. F. (1994). How to Use Action Research in the Self-Renewing School.
Alexandria, VA: ASCD.
Schmuck, R. (either 1997 or 2006). Practical Action Research for Change. Arlington Heights,
IL: Skylight. (Used copies of old editions available via amazon.com)
Readings for the course consist primarily of individual articles and book chapters.
Most of these can be downloaded from the Healey Library's Electronic Reserves
(marked ERes on the syllabus, docutek.lib.umb.edu/
(path: Electronic reserves and Course Materials | select cct693, enter password
provided by instructor) or e-journals).
Recommended: portable storage (e.g., flash drives), synchronization & bibliographic
software. (For more info see http://www.cct.umb.edu/competencies.html)
Additional materials linked to the course website include:
Notes on Teaching/Learning Interactions (including
guidelines for assignments and links to previous students' work)
Rubrics (for course as a whole & for individual
assignments), including Objectives for Thoughtful and Responsive Educators
Handouts, some non-copyrighted Readings and other Resources, and examples of
student reports from previous classes. (If link doesn't work for a handout that
you need, notify the instructor.)
REQUIREMENTS
More detail about the assignments, expectations, and rationale is provided in
the Notes on Teaching/Learning Interactions and Rubrics (see links above) and
in handouts that will be linked to this website. (Alert the instructor if a
link does not work to a handout you need.)
Written assignments and presentations (2/3 of grade)
A. Action Research written assignments (eight)
B. Action Research work-in-progress presentations (two)
Design an Action Research Process related to a change or intervention in a specific
classroom, workplace or personal teaching/learning practice, an educational
policy, an educational institution, or a social policy. Your design should include
how you will evaluate the existing situation, how you would facilitate the reflective
and/or collaborative process in which a constituency shapes a change or intervention,
and how you would evaluate the outcome with a view to expanding further the
constituency for adopting/adapting the change or intervention. (If you actually
carry out some of the design, that will deepen the project, but it is not required.)
A sequence of 10 assignments is required-reflection on class demonstration action
research, initial description (based on strategic personal planning), KAQ, evaluation clock, work-in-progress presentation, notes on research and planning, narrative outline, second work-in-progress presentation, complete draft report, and final (1500-2500 words) report.
Participation and contribution to the class process (1/3 of grade)
C. Building learning community through prepared participation and attendance
at class meetings (=13 items)
D. Personal/Professional Development (PD) Workbook submitted for perusal before
end of week 6 (with worksheet in week 6) & at the end of the semester (=2
items)
E. Minimum of two in-office or phone conferences on your assignments, PD workbook,
and project -- one before mid-semester break; the other before week 10 (=2 items)
F. Peer commentary on another student's draft report (with copy submitted to
PT or included in PD workbook)
G. Assignment Check-list maintained by student
and submitted week 12
H. Process Review on the development of your work, included with your PD Workbook
at end-of-semester perusal.
SCHEDULE OF CLASSES
Class 1 (1/29) Introduction to Action Research Cycles and Epicycles, I
The framework of Action Research Cycles and Epicycles
is introduced through a compressed example during
this class and the next.
Critical Incident Questionnaire
Homework tasks include: review the syllabus and overview, get set-up to use the internet and computers,
download Notes on Teaching/Learning Interactions (including
guidelines for assignments and links to previous students' work) and
Rubrics (for course as a whole & for individual
assignments),
peruse vision charts and evaluations from previous semesters, begin your
PD workbook, and sign up for first conference (to which you should bring your
PD workbook).
Class 2 (2/5) Introduction to Action Research Cycles and Epicycles, II
Reading: Schmuck, 1997, p. vii-29; 2006, p. ix-29
Pre- or post-class reading on Focused Conversations: Stanfield, 6-29.
Feedback on Critical Incident Questionnaire
Questions on Syllabus and course mechanics
Continuation of activity from class 1
Focused Conversation on compressed Action Research experience to date (handout)
Class 3 (2/12) Strategic Participatory Planning, applied to personal course
and life projects
Reading: Final project by alum of the course (Jan Coe; from password protected site - see below), CEDAC, Our Economy, Taylor, "Epilogue," 204-210,
Schmuck on "cooperative" action research, Spencer, chaps. 5 &
7, Weissglass, "Constructivist Listening," also Review Project reports
from previous semesters (online using password protected
site.)
Interview an alum of the course about their experience developing an Action Research process.
Supportive Listening (a variant of constructivist
listening) on one's hopes/fears re: educational change
Strategic personal planning workshop (about the educational/organizational change
you want to facilitate/promote) (handout,pdf)
In-class drafting of initial description of AR design project
Post-class reading: (for those interested in Strategic Participatory Planning) Materials from ICA Facilitators Manual (PPR)
*A* Asmt. 1: 1st Reflection on your Experience as Novice Action Researchers
in relation to the Considered Formulations from Other Sources (Schmuck)
2/19 No class (Presidents' Day)
Class 4 (2/26) Opening up problems and focusing in on inquiry
Reading: Greenwald, "Learning from Problems."
Use of KAQ framework.
*A* Asmt. 2: Initial Project Description (revised in response to PT's
comments by email on in-class draft)
Class 5 (3/5) Formulating informative comparisons as a basis for evaluations,
I
Comparison steps (2-4) in the evaluation clock, used to analyze a clipping on
the effects of a smoking ban, then to design evaluation as part of students'
projects
Reading: Clipping (PPR)
Guide to the Evaluation clock (handout (before and/or after class)
*A* Asmt. 3: KAQ assignment
Class 6 (3/12) Formulating informative comparisons as a basis for evaluations,
II
Introduction to statistical formulations of comparisons and background assumptions
Peer coaching on Evaluation clock assignment and its extension to students'
projects
*A* Asmt. 4a. Use the comparison steps (2-4) in the evaluation clock
to design evaluation as part of your project.
*A* First conference must be completed by 3/15 to discuss your Action
Research ideas, the course thus far, and your PD workbook (bring to conference)
*A* Submit worksheet on PD workbook
and research organization (as part of participation item on PD workbooks)
*A* Schedule second conference before 4/25 to discuss your projects and
use of evaluation clock
3/19 No class (Spring break)
Class 7 (3/26) Work-in-progress presentations, I
*A* Asmt 5: Work-in-progress Presentation I on Project
*A* Asmt 6: Notes on Research and Planning for Student Projects (to be pasted onto the wiki by 3/25 at latest so everyone can see how other students are approaching their projects)
Class 8 (4/2) Reflection on your Experience as Novice Action Researchers
with the Considered Formulations from Other Sources
Readings: Schmuck, pages 29-146, Calhoun, How to Use Action Research
(especially chapters 1-3), Weiss, chapter 1, and (optional) Weiss, chapters 2 &4.
Dialogue Process session
Critical Incident Questionnaire II on course to date
*A* Asmt. 4b due: Use the full evaluation clock to design the evaluation
part of your project.
Class 9 (4/9) Reflection on your Experience as Novice Action Researchers
with the Considered Formulations from Other Sources, II
Readings: At least three from Hitchcock & Hughes, Chap. 3, "Access,
ethics, and objectivity," Chapter 5, "Designing, planning and evaluating
Research"; Greenwood & Levin, Chaps. 8 & 11, "Action research
cases," & "Action science and organizational learning"; Rokovich,
et al., "Implementing change"; Jenkins, "Action learning";
CEDAC, Our Economy; Greenwald, Science in Progress; Madison Metropolitan
School District, "Classroom action research" (and linked pages),
study of CIT
Feedback on Critical Incident Questionnaire II
Small group work on guidelines
for a) small group work and b) Action Researchers
Video segment on Myles Horton and the Highlander Center, a longterm source of
educational and social change
4/16 No class (Patriots' Day)
Class 10 (4/23) Politics and Theories of Evaluation and Educational Research,
I: Case studies
(handout)
Readings--at least one of:
Woodhead, "When psychology," Hunt, "The dilemma," Metcalf,
"Reading between the lines."
"Jig-saw" digestion and discussion of readings
*A* Asmt 7: Narrative Outline for Project Report
Class 11 (4/30) Work-in-progress presentations, II
*A* Asmt 8: Work-in-progress Presentation II on Project
Titles of Projects
Julie Barrett, "Schedule Migration and Problem Based Learning at Massachusetts College of Art"
Elizabeth Burch & Renee Plouffe, "To foster a more inclusive school environment for children with special needs"
Virginia De La Garza, "Improving The Effectiveness of a Faculty Training Program at a University in Mexico"
Elliot Frank, "Evaluation of Metacognition in Biology"
David Frias, "How Can Lower Level Employees Affect Upper Level Management Decision Making?"
Scott Goddess, "An inquiry into establishing productive, sustainable, coaching partnerships through making connections "
Marnie Jain, "Separating Myself from My Art and Becoming the Artist I Want to Be"
Jane Lachance, "My Use of Story as a Facilitating Tool in a Small Heatlh Care Workers Group "
Kathleen Leavitt, "Supporting Staff who are without a Nurse in Charge"
Michael Lihon, "Action research to continuously motivate participation, of production workers, on changes to support process improvement "
Elizabeth Naylor, "Action Research to develop curriculum for teachers to become Ordinary Inspirers and encourage their
students to be Moral Learners"
John Quirk, "Teachable Leadership and the Junior Class: A Plan to Develop Effective Student Leadership through Regenerative, Reflective Action Research Practices"
Joshua Reyes, " Electricians Online Review: A Community of Learners"
Class 12 (5/7) Politics and Theories of Evaluation and Educational Research,
II: Action Research as an alternative to Positivist and Interpretivist approaches
Readings--at least one of:
Carr & Kemmis, Becoming Critical, CEDAC, Our Economy, Greenwood,
"Action science and organizational learning," Taylor, "Epilogue"
Additional reading: McLeod, et al., "Changing how we work," Senge
et al., "Fostering communities"
Dialogue Process session on theory and practice
*A* Assignment Check-list maintained
by student, with incomplete contract if needed
*A* Asmt due: Complete Draft of Design Project (2 copies)
*A* Make comments on another student's draft, and email them to the person
by 5/12.
Class 13 (5/14) Taking stock of course & of change: Where have we come
& where do we go from here?
(handout)
Reading: Tuecke, "Creating a wall of wonder," Cashin, "Student
ratings of teaching"
Historical/Future Scan of Educational change in its wider world context
GCOE & CCT course evaluations (see previous semesters' evaluations)
Additional reading: Stanfield, Courage to Learn, Stanfield, The Workshop
Book (selections TBA)
5/21
Due before 5pm *A* PD workbook submitted for perusal, to be picked
up after 5/28 from Department of Curriculum & Instruction office, W-2-93
*A* Final Project report (printed copy as well as by email attachment
or on disk) and Process Review 5pm, Initial meeting for fall synthesis students
(tentative)
BIBLIOGRAPHY
ERes indicates reading on electronic reserve.
# indicates additional texts on evaluation, action research, or facilitating
group process (to be borrowed from the library, interlibrary loan, or instructor.
## indicates usefull readings to help in writing and revising.
Backer, T., J. Chang, A. Crawford, T. Ferraguto, D. Tioseco and N. Woodson (2002).
"Case study and analysis: The Center for the Improvement of Teaching, University
of Massachusetts, Boston."(PPR)
Brookfield, S. D. (1995). Becoming a Critically Reflective Teacher. San
Francisco: Jossey-Bass Publishers (eRes)
Calhoun, E. F. (1994). How to Use Action Research in the Self-Renewing School.
Alexandria, VA: ASCD.
Carr, W. and S. Kemmis (1986). Becoming Critical: Education, Knowledge and
Action Research. Geelong: Deakin University Press., chapters 6 & 7 (up
to p. 200 on eRes)
Cashin, W. E. (1990). "Student ratings of teaching: A summary of the research."
Management Newsletter 4(1): 2-7. (eRes)
CEDAC (Community Economic Development Advisory Committee) (1995). Our Economy:
Our Future, Final Report. York, Ontario: City of York. (eRes)
Conlin, M. L. (2002). "The basics of writing: Process and strategies,"
in Patterns Plus: A Short Prose Reader with Argumentation. Boston: Houghton
Mifflin, 1-11. (eRes)##
Couto, R. A. (2001). "The promise of a scholarship of engagement."
The Academic Workplace 12(2): 4, 6.
Daniel, D., C. Fauske, P. Galeno and D. Mael (2001). Take Charge of Your
Writing: Dicovering Writing Through Self-Assessment. Boston: Houghton Mifflin.##
Elbow, P. (1981). Writing with Power. New York: Oxford Univ. Press##
Entin, D. (2001). "Review of The Reflective Practitioner: How Professionals
Think in Action." The Academic Workplace 12(2): 13, 18.
Greenwald, N. (2000). "Learning from Problems." The Science Teacher
67(April): 28-32. (eRes)
Greenwald, N. (2000). Science in Progress: Challenges in Problem-based Learning
for Secondary Schools
Greenwood, D. J. and M. Levin (1998). Introduction To Action Research: Social
Research For Social Change. Thousand Oaks, CA: Sage. (pp. 187-202 on eRes)
Hitchcock, G. and D. Hughes (1995). Research and the Teacher: A Qualitative
Introduction to School-based Research. New York: Routledge.(pp. 77-112 on
eRes)
Hunt, M. (1985). "The dilemma in the classroom: A cross-sectional survey
measures the effects of segregated schooling," in Profiles of Social
Research: The Scientific Study of Human Interactions. New York: Russell
Sage,51-97. (eRes)
Institute of Cultural Affairs, n.d., Facilitators Manual (excerpts on
Strategic Participatory Planning). Toronto: Canadian Institute of Cultural Affairs.(PPR)
Isaacs W. (1999) Dialogue and the Art of Thinking Together. New York:
Currency.#
Jenkins, M. (2000). "Action learning: Taking the time it takes." Paper
presented to the International Association of Facilitators, Toronto, April 27
2000.
Kanar, C. (2002). "Improving your paragraph skills," in The Confident
Writer. Boston: Houghton Mifflin, 60-88.##
Madison Metropolitan School District (2001). "Classroom action research."
http://www.madison.k12.wi.us/sod/car/carhomepage.html viewed 25 Jan.
03.
Madison Metropolitan School District (2001). "Classroom action research
starting points." http://www.madison.k12.wi.us/sod/car/carstartingpoints.html
viewed 25 Jan. 03.
McLeod, M., P. Senge and M. Wheatley (2001). "Changing how we work."
Shambhala Sun(January): 29-33. (eRes)
Metcalf, S. (2002). "Reading between the lines." The Nation(Jan.
28): 18-22. (eRes)
Nelson, J. (2001). The Art of Focused Conversation for Schools. Toronto:
Canadian Institute of Cultural Affairs.
Patton, M. Q. (1982). Practical Evaluation. Newbury Park, CA: Sage.#
Perelman, L., J. Paradis, E. Barrett (n.d.) The Mayfield Handbook of Technical
and Scientific Writing. https://mit.imoat.net/handbook/home.htm##
Pietro, D. S. (Ed.) (1983). Evaluation Sourcebook. New York: American
Council of Voluntary Agencies for Foreign Service.#
Rokovich, M. A., M. Stevens and J. Stallman (2000). "Implementing change
at SJUSD: An unfinished case study." Presented to the International Association
of Facilitators, Toronto, April 27 2000. (PPR)
Schmuck, R. (1997). Practical Action Research for Change. Arlington Heights,
IL: Skylight.
Senge, P., N. Cambron-McCabe, T. Lucas, B. Smith, J. Dutton and A. Kleiner (2000).
"Fostering communities that learn," in Schools That Learn.
New York: Currency,459-465. (eRes)
Spencer, L. J. (1989). Winning Through Participation. Dubuque, Iowa:
Kendall/Hunt (eRes)
Spina, S. U. (2002). "Six key principles of action research." (handout)
Stanfield, B. (Ed.) (1997). The Art of Focused Conversation. Toronto:
Canadian Institute of Cultural Affairs. (pp.30-37 on eRes)
Stanfield, B. (2000). The Courage To Lead: Transform Self, Transform Society.
Gabriola Island BC: New Society Publishers.#
Stanfield, R. B. (2002). The Workshop Book: From Individual Creativity to
Group Action. Toronto: Canadian Institute of Cultural Affairs.#
Stark, J. S. and A. Thomas (Eds.) (1994). Assessment and Program Evaluation.
Needham Heights, MA: Simon & Schuster. (#, in Healey stacks)
Taylor, P. J. (2005). "Epilogue," in Unruly Complexity: Ecology,
Interpretation, Engagement. Chicago, University of Chicago Press: 203-213.
(PPR)
Tuecke, P. (2000). "Creating a wall of wonder with the TOP environmental
scan." International Association of Facilitators, Toronto, Canada, April
27 - 30 (iaf-world.org/iaf2000/Tuecke.PDF).
Turabian, K. L. (1996). A Manual For Writers of Term papers, Theses, and
Disertations. Chicago: Univ. of Chicago Press (in Healey reference section)##
Weiss, C. H. (1998). Evaluation. Upper Saddle River, NJ: Prentice Hall.#
Weissglass, J. (1990). "Constructivist listening for empowerment and change."
The Educational Forum 54(4): 351-370. (eRes)
Winter, R. (1989). Learning from Experience: Principles and Practice in Action
Research London: Falmer.#
Woodhead, M. (1988). "When psychology informs public policy." American
Psychologist 43(6): 443-454. (eRes)