CONF 660: Conflict Assessment and Program Evaluation

Institute for Conflict Analysis and Resolution

George Mason University

Class Time:       10:00am – 4:00pm

Sat. 1/24, Sat. 1/31, Sat. 2/7, Sat. and Sun. 2/21—2/22, Sat. 3/7

Location:          Arlington Campus – Original Building 303 (and 250)      web:  courses.gmu.edu

Instructors:       Carolyne Ashton, Ph.D.                                                Mara Schoeny, Ph.D.

                        703-993-1300 (ICAR front desk)                                703 993-9191 (ICAR)

                        cashton@gmu.edu                                                        mschoeny@gmu.edu

Office Hours:    TBA and by appointment

Introduction

Prerequisites or corequisites: CONF 501 or 502, and acceptance in the graduate certificate program or graduate program in Conflict Analysis and Resolution, or permission of instructor.

The course examines the monitoring and evaluation of conflict resolution programs and initiatives and will teach evaluation strategies relevant for work in conflict or post-conflict contexts. Course readings will emphasize conflict assessment and evaluation methods for conflict resolution initiatives generally and specifically as these topics relate to the certificate programs, including: community planning and collaborative initiatives; conflict prevention, reconstruction and stabilization; intrastate and international conflict resolution and peacebuilding; and evaluating the use of specific conflict resolution skills and processes.

Practical difficulties and things to consider for evaluation will include: balancing multiple purposes of evaluation; defining success; power and empowerment; funding; confidentiality; impartiality; ethics; cultural issues; conflict sensitivity; and working in conflict environments.  Students will develop an appreciation and understanding of the emergent approaches for evaluating conflict interventions. Students will complete an evaluation design for a conflict intervention initiative as the major course requirement.

Objectives and Core Competencies

The course will focus on the following objectives:

v     To provide an overview of the role and importance of program evaluation and the variety of approaches and tools available.

v     To provide an overview of the phases and steps in designing and implementing a monitoring and evaluation plan.

v     To explore the constraints, challenges and other considerations an evaluator working in conflict environments might face, and how these influence the design and implementation of assessments and evaluations.

By the end of the course, students will be expected to have the following core competencies:

Ø      Know how to select the best approaches and tools for monitoring and evaluation given the goals of the evaluation and constraints and challenges to design and implementation.

Ø      Know how to design a monitoring and evaluation plan that links conflict assessment to intervention goals, objectives, design and implementation.

Ø      Know how to design specific data collection methods, measures, and indicators for use in monitoring and evaluation.

Ø      Know how to conduct an evaluation, write a  report and encourage utilization of findings.

Course Expectations

1. Consistent attendance. Barring exceptional circumstances, you are expected to attend all weekends for the full time scheduled.
2. Effective preparation. Class discussions and activities depend on your preparation.
3. Appropriate participation. Engage actively in the course in whichever of the formats you are most comfortable with: large group discussions, small group discussions, class exercises, on-line homework assignments between classes, etc.
4. Course completion. In keeping with departmental policy, incomplete grades will be given only in cases of personal or immediate family illness.

Course Requirements

Students are responsible for completing individual and group assignments on time.  You will be penalized the equivalent of a full letter grade for each day the assignment is late.  Some class assignments and readings will be on the course homepage or sent via e-mail. Students are responsible for checking GMU email and keeping up-to-date with these. 

1.      Preparation and Active Participation (10%)  Students are expected to actively participate in class and online and to demonstrate completion of assigned readings. Given the intense schedule of this course, absences will be excused only in exceptional circumstances (for example, death in family and medical emergencies). Unexcused absences will negatively affect participation grades. An alternate assignment may be required for any excused absences (to be negotiated with the instructor).

2.      Proposal Critique and Evaluation Design (25%)  Several proposals are available on the course homepage.  Please choose one that matches your interests in the conflict resolution field.  Your assignment is to review the proposal and then develop a draft evaluation plan using the readings and lecture materials from the first two classes.  You may use one of the logframe formats provided in the “Resources” section at the end of the syllabus.  Provide a critique of the proposal by determining if the program design and description provide the necessary elements from which to develop an evaluation plan.  Develop a draft evaluation plan based upon the proposal content.  If you find the proposal lacking in direction for an evaluation plan, you are to add the missing elements to your evaluation design and explain the changes needed in the original program design description that would have indicated the new evaluation elements.   Due: (before 3^rd weekend) Thursday, February 5^th (midnight)

3.      Group Bibliographic Essay and Presentation (25%)  The class will be divided into groups of 4-5 students (by certificate program of study or other method TBD) for the purpose of investigating, analyzing and organizing existing resources for evaluation in specific conflict arenas (such as interfaith dialogue, public participation processes or in-house ADR programs). Each group will present their results and provide an online resource guide available to all class members. The bibliographic essay is a narrative discussion and review of the literature, issues, indicators, evaluation approaches, organizations and resources relevant to your particular program type, presented to help orient those new to the subject. The presentation will be a maximum of 20 minutes, including 5 minutes for discussion and Q&A. Further details to be provided in class.  Due: Weekend Four, Saturday, February 21

4.      Final Paper (40%) The final paper is an evaluation design demonstrating mastery of the course material. All papers should be 15-18 pages and be well supported through research. The evaluation proposal should be as specific and practical as possible since the assignment is designed for you to demonstrate your ability to design a monitoring and evaluation plan that can be utilized in real world settings.  Begin by selecting a conflict and conflict resolution initiative as the basis for the paper; you should have personal access to the program or be able to do research via available program documentation. Briefly describe the conflict and the assessment that the intervention was designed to impact. Identify the goals and objectives of the intervention.

Now, design a monitoring and evaluation plan to measure progress towards achieving the initiative/program’s goals and objectives. Identify the type of evaluation you have selected and discuss the alternative approaches considered. Also include the purpose of the evaluation, hypothesis (if any), intended audience, specific data collection and analysis plan, and how the results of the evaluation will be distributed and utilized. The paper should be written in a format consistent with typical evaluation plans/reports (minus the results and conclusions section assuming there isn’t time/access to conduct an actual evaluation). Further details to be provided in class.  Due: Sunday, March 15^th (midnight)

University Resources and Assistance

Writing Center: The Writing Center provides tutors and online services to help you develop ideas and revise papers at no charge.  It can sometimes accommodate walk-ins, but generally it is best to call for an appointment.  Location: ARL 334C (in the main SPP suite; call for office hours) Contact: (703) 993-4491  http://writingcenter.gmu.edu

Disability Resource Center: The Disability Resource Center assists students with learning or physical conditions affecting learning.  Students with learning differences that require special conditions for exams or other writing assignments should provide documentation provided by the Disability Resource Center. Please see one of the instructors the first week of classes.

Location: SUB I, Room 222. Contact: 703-993-2474 www.gmu.edu/student/drc/

Honor Code: You are responsible for knowing, understanding and following Mason’s Honor Code, found at http://www.gmu.edu/catalog/apolicies/#Anchor12. Be sure that all work submitted is your own and that you use sources appropriately. It is recommended that you review requirements regarding use and citation of sources prior to submitting your final work.

Readings

Required Text (available at the GMU Arlington Bookstore or from online booksellers)

Bamberger et al. RealWorld Evaluation. Sage Publications, 2006.

Required Books/Articles/Manuals (Available On-line)

Church, Cheyanne and Mark Rogers. Designing for Results: Integrating Monitoring & Evaluation in Conflict Transformation Programmes. Washington, DC: Search for Common Ground, 2005. http://www.sfcg.org/programmes/ilr/ilt_manualpage.html

Elliot, Michael, Pearson d’Estree, Tamra, and Kaufman. "Evaluation as a Tool for Reflection," Beyond Intractability. Ed. Guy Burgess and Heidi Burgess. Conflict Research Consortium, University of Colorado, Boulder, Colorado, USA. September 2003.
http://www2.beyondintractability.org/m/Evaluation_Reflection.jsp

Lederach, John Paul, et al., Reflective Peacebuilding:  A Planning, Monitoring and Learning Toolkit.  The Joan B. Kroc Institute for International Peace, University of Notre Dame, Indiana and Catholic Relief Services Southeast Asia, 2007.

http://crs.org/publications/showpdf.cfm?pdf_id=80

Search for Common Ground. “Evaluation Guidelines.” July 2006. http://www.sfcg.org/sfcg/sfcg_evaluations.html

Natalie J. Grove a; Anthony B. Zwi.  Beyond the log frame: a new tool for examining health and peacebuilding initiatives (Critiques use of logframes within the context of health promotion as a peacebuilding component.) http://pdfserve.informaworld.com/816811__789773310.pdf       OR

Gasper, Des (2001) ‘Logical Frameworks: Problems and Potentials. The Logical Framework Approach: A Critical Assessment’ (a general critique of logframes used in a variety of settings.) http://winelands.sun.ac.za/2001/Papers/Gasper,%20Des.htm

Required Additional Reading—Available on course homepage

D’Estree, Tamra Pearson, Larissa Fast, Joshua Weiss, and Monica Jakobsen. (2001) “Changing the Debate about ‘Success’ in Conflict Resolution Efforts.”  Negotiation Journal, vol. 17., no. 2.  pp. 101-113.

Patton, Michael Quinn, Utilization Focused Evaluation, 4^th Ed. Thousand Oaks:  Sage Publications, 2008.  (selected chapters)

Ross, Marc Howard. (2000) “’Good-Enough’ Isn’t So Bad: Thinking about success and failure in ethnic conflict management” Peace and conflict: Journal of peace psychology 6(1).

pp. 27–47.

Assorted conflict resolution program proposals (for first assignment)

CLASS PLANS AND ASSIGNMENTS

Weekend  1                                                                Saturday: January 24^th, 10 am-4 pm

Introductions

• Class members
• Introduction to the Course, Syllabus Overview

Overview

History and purpose of evaluation

Types of evaluation and evaluation roles

Monitoring and Evaluation (M&E) for Conflict Interventions

• Evaluation cycle and stages
• Assessment/analysis, design & implementation, monitoring & evaluation

Evaluation Exercise: developing criteria, measures

Required Reading for Weekend One

Bamberger et al. RealWorld Evaluation. Chapter 1

            Church and Rogers. Designing for Results, Chapters 1, 7   http://www.sfcg.org/programmes/ilr/ilt_manualpage.html 

Elliot, Pearson d’Estree, and Kaufman. "Evaluation as a Tool for Reflection," Beyond Intractability.  http://www2.beyondintractability.org/m/Evaluation_Reflection.jsp

Lederach, John Paul, et al., Reflective Peacebuilding:  A Planning, Monitoring and Learning Toolkit.  http://crs.org/publications/showpdf.cfm?pdf_id=80

Weekend 2                                                                 Saturday: January 31^st, 10 am-4 pm

Evaluation cycle and Logic Models

• Connecting evaluation to intervention design & program planning
• Theories of change

Applications

• Case Studies

• Sign up/Select Group for Group Presentation in Weekend 4

Required Reading for Weekend Two

Bamberger et al. RealWorld Evaluation. Chapter 2

            Church and Rogers. Designing for Results: Chapter 2, 3, 4 http://www.sfcg.org/programmes/ilr/ilt_manualpage.html 

         Grove and Zwi. “Beyond the log frame…” http://pdfserve.informaworld.com/816811__789773310.pdf       OR

            Gasper. “Logical Frameworks: Problems and Potentials….” http://winelands.sun.ac.za/2001/Papers/Gasper,%20Des.htm

         Patton, Utilization Focused Evaluation, Chapter 10 (available on course homepage)

Between Weekends 2 & 3   (February 1—6 )

First paper due (Proposal Critique and Evaluation Design): Thursday, February 5^th

Weekend 3                                                                 Saturday, February 7, 10 am-4 pm

Theories of Change and Evaluation Reports

• Paper feedback and debrief 

Overview of Evaluation Models

Evaluation Planning

• Steps in planning an evaluation
• Case Study examples

Panel of Guest Speakers

Required Readings for Weekend Three

Bamberger et al. RealWorld Evaluation. Chapters 3, 4, 5, 16.

           Church and Rogers. Designing for Results.  Ch. 5, 8.  

            http://www.sfcg.org/programmes/ilr/ilt_manualpage.html

D’Estree, Tamra Pearson, Larissa Fast, Joshua Weiss, and Monica Jakobsen. “Changing the Debate about ‘Success’ in Conflict Resolution Efforts.”  Negotiation Journal, vol. 17. no. 2.  pp. 101-113.

Between Weekends 3 and 4 (February 8-20)

Group project research and preparation

Weekend 4                                                     Saturday and Sunday, February 21 & 22

Saturday

Group Presentations

Evaluation Obstacles

·        Time, budget, data

Sunday

Selecting M&E Criteria and Setting Standards

·        Developing indicators

·        Defining success

Methods and Measures

·        Collecting data

·        Developing measures

·        Writing questions for interviews, surveys

·        Conducting focus groups

Required Readings for Weekend Four

Bamberger et al. RealWorld Evaluation. Chapters 7, 14.

            Church and Rogers. Designing for Results.  Ch. 9 and 12.  

            http://www.sfcg.org/programmes/ilr/ilt_manualpage.html

D’Estree, Fast, Weiss, and Jakobsen. “Changing the Debate about ‘Success’ in Conflict Resolution Efforts.”  pp. 101-113.

Between Weekends 4 & 5   (February 22—March 1 )

Online Discussion: opens Feb. 22 and closes March 1. Read article below, see online for discussion prompts.

Ross, Marc Howard. (2000) “’Good-Enough’ Isn’t So Bad: Thinking about success and failure in ethnic conflict management” Peace and conflict: Journal of peace psychology 6(1), 27–47.

Weekend 5                                                                             Saturday, March 7   10am-4pm

Evaluation Obstacles Part II: Political Influences, Conflict Effects

·        Power and empowerment

·        Ethics

·        Conflict context

Evaluation plan drafts: Peer review

Course Wrap-Up

• Picture

• Evaluations

Required Readings for Weekend Five

Bamberger et al. RealWorld Evaluation. Chapters 6, 8 and review 16.

Church, Cheyanne and Mark Rogers. Designing for Results: Integrating Monitoring & Evaluation in Conflict Transformation Programmes.  Ch. 10 and 11.

            http://www.sfcg.org/programmes/ilr/ilt_manualpage.html

Patton, Utilization Focused Evaluation, Chapter 12 (available on course homepage)

** FINAL Evaluation Plan DUE: March 15^th, midnight**

Supplemental Resources

Sample Logframes:

Search for Common Ground Sample Logframe Matrix. (CONF 660 homepage)

Council of Europe/European Union:  YouthFrame II:  Framework Partnership Agreement in the field of Youth 2007-2009

http://www.jp.coe.int/CEAD/JP/Default.asp?PrintVersion=True&SpecificObjectiveID=5108&SA=1&SE=0

EDPRS Five-year Plan – Justice, Reconciliation, Law and Order Logframe

http://www.minijust.gov.rw/IMG/doc/Strategie_Ministry_of_Justice_COMPLETE_DOC.doc

Articles on Logframes

Cummings - Canadian Journal of Development Studies, (University of Ottawa) Special Issue, Vol. XVIII. 1997, Results Based Performance Reviews and Evaluations, edited by H. Cummings. Ashton, pp 587-596 and Sawadogo & Dunlop, pp 597-612.

Gasper, Des (2000) ‘Evaluating the “Logical framework approach”: towards learning-oriented development evaluation’, Public Administration and Development 20: 17–28.

WFP Report on Workshop in India 2000 - Project Cycle Management & LogFRAME Approach

http://www.wfp.org.in/events/workshops/logframe/logframe_report.htm

Sample Terms of Reference:

Search for Common Ground Sample TOR available at:

http://evaluation.francophonie.org/IMG/doc/Evaluators__GTFRFP_Final_3Dec08.doc

Evaluation Guides & Checklists:

Resource Guide on Evaluation

http://www.sfcg.org/programmes/ilr/resevaluation.doc

Resource Guide for Monitoring

http://www.sfcg.org/programmes/ilr/monitoring1.doc

Evaluation Checklist, Michael Quinn Patton, 2003

http://www.wmich.edu/evalctr/checklists/qec.pdf