Share
Embed
Evaluation as reflective practice
A presentation by Camilla Child and Dione Hills as part of our new series of lunchtime dialogues on The Dynamics of Evaluation, which contributes to the celebration of the International Year of Evaluation.
Tavistock Institute
June 29, 2015
Transcript
28 slides
-
Evaluation as Reflective practice A Dynamics of Evaluation Dialogue Dione
Hills Camilla Child -
What this event is about The series: • Bringing together
evaluation and consultancy practice • Exploring this interface with others working in the field • Considering the potential for a future professional development offer -
Today’s event How can we make best use of our
personal experience, alongside evaluation skills, to support learning and change? Reflective practice and evaluation Dione Hills The perspective from an organisational consultant Camilla Child An experience of reflective practice Feedback -
Reflective Practice is • Central to the evaluation process •
Implicit in guidelines on professional practice • At the heart of recent professional development activities -
Reflective practice involves bringing feelings and implicit experience to the
surface “Bridging the gap between the ‘high ground’ of academic rigour and the ‘lowland of messy practice’ requires ‘professional artistry’, an artistry which should be informed by ‘reflective practice’”. Donald Schön (1983): The Reflective Practitioner: How professionals think in action. -
Reflective practice as central to evaluation activities Rationale Objectives Appraisal
Monitoring Evaluation Feedback *ROAMEF cycle from Treasury Green Book Opportunity to reflect and learn conceptuali se test experience reflect -
Reflective Practice is generally part of the competency frameworks developed
by Evaluation societies *From UKES Evaluation capabilities framework 2013 http://www.evaluation.org.uk/assets/UKES%20Evaluation%20Capabilities%20Framework%20January%202013.pdf PROFESSIONAL PRACTICE 2.25 Displays impartiality in conducting and reporting evaluation 2.26 Manages conflicts of interests and values fairly 3. QUALITIES AND DISPOSITIONS 3.2 Exercises sound, rigorous and fair judgment 3.4 Displays independence of mind and integrity especially when evaluation challenged 3.6 Displays self-knowledge and pursues professional development -
Reflective practice is part of a process for enhancing professionalism
in evaluation being developed by European and UK evaluation societies (EES and UKES) Practice standards Practitioner skills Ethical guidelines Personal conduct Capability frameworks Knowledge base Profession al application Developing the individual 9 -
Voluntary Evaluator Peer Review (VEPR) is about reflection and development
rather than designation Reflective practice Peer review è Accountabilit y & professional development 10 -
Reflective practice bridges the gap between theory, skills, values and
methods 11 methods Interpersonal skills and values theory Bridges the gaps in evaluation “Bridging the gap between the ‘high ground’ of academic rigour and the ‘lowland of messy practice’ “If you can keep your head when all about you are losing theirs….you don’t understand the situation”. Evan’s law -
How do evaluators learn reflective practice? • On ‘research or
evaluation methods’ courses? • Reading (journal articles, methods text books)? • Professional development opportunities? • ‘On the job’? • Prior / outside work experience? -
Reflective practice uses subjective and objective data (four quadrants integral
map by Ken Wilber) Research training/text books often focus attention on the right hand side: objective knowledge and data (and away from the left side) -
But it can be difficult to be reflective when we…..
Are under time pressure – to produce, analyse and report on ‘data’ Have tensions in the evaluation team -
What is reflective practice? • Reflective practice is the capacity
to reflect on action so as to engage in a process of continuous learning. (Schon 1983) • It allows you to be a part of an unfolding drama as well as providing opportunities to be sitting in the audience watching it • Is about creating awareness of the self in relation to the system with which we are working so that we can work better with that system or set of relationships. Self as instrument/instrumentality • Offers the opportunity to think and take stock -
Different models of reflective practice • Kolb (1975) Experiential learning
• Argyris and Schon (1978) Single and Double loop learning • Schon (1983) – Reflection-in-Action, Reflection-on-Action • Brookfield (1998) Seeing through different lenses • Johns (1995) Looking out, looking in -
Carper (1978) – fundamental ways of knowing Many sources of
data for ‘knowing’ • Empirical - factual knowledge from science, or other external sources, that can be empirically verified. • Personal - knowledge and attitudes derived from personal self understanding and empathy, including imagining one's self in the patient's position. • Ethical - attitudes and knowledge derived from an ethical framework, including an awareness of moral questions and choices. • Aesthetic - awareness of the immediate situation, in this sense is used to mean "relating to the here and now", from the Greek meaning "I perceive, feel, sense"; -
Audience Fitz Actor Henry Benjamin Anna Stage Manager Actor Actor
Off stage/ Outside Humphrey Jane Rehearsal Rooms (Front) stage Neil Actor Back stage Audience Audience The wings Back stage Goffman (1959) – Stage behaviour -
Roles given to and/or taken up by evaluator – Expert:
client plays inactive role, key decisions and technical control lies with consultant (evaluator), two way communication limited, client’s role to ‘judge and evaluate’ after the fact – Pair of Hands: all key decisions taken by client who specifies actions required, consultant (evaluator) role is to apply specialist knowledge, two way communication limited – Collaborator : interdependency, data collection and analysis jointly undertaken, implementation by joint discussion and agreement Peter Block: Flawless consulting -
What can happen to an evaluator? • Crossing the Boundary
into the system Crossing boundaries induces mul1ple responses and feelings. These feelings may be heightened, some1mes difficult, but always useful to no1ce in oneself and in others • Being too far in • Too far out • On the boundary -
What can happen to evaluation teams This can be understood
to be a fractal of the system as the same patterns repeat through the system Parallel process – the dynamics within an organisation are reproduced between the organisation and a consultant [or evaluator], or within the consultant [evaluation] team -
Ways of supporting reflective practice • Reflective notes (as part
of the reporting on interviews or field observations) • One to one - with someone not directly involved in the evaluation/issue – supervision/coach/colleague • In teams – taking time to reflect on feelings/general observations as well as on evaluation ‘content’ • Recognising that members of the team might have a ‘valence’ to take on feelings, or identify closely with one part, of the system – Using this as data (rather than rejecting or scapegoating them) • Feedback reflections (appropriately) to clients to test out whether this resonates or is helpful. (May depend on role taken up) -
Practical exercise in reflection • Think of an evaluation in
which you have been involved (as evaluator, commissioner, or being evaluated) that you experienced as challenging – At what point did these challenges arise? – What feelings came up for you? • Can you see any link between the challenges and feelings you experienced and: – The nature of the programme/project/work situation itself (e.g. lack of clear boundaries, any conflicts or disagreements between different groups involved, denial of difficulties coming up) – The characteristics of the ‘issue’ or client group that the programme/ project/policy was concerned with – What your brought personally to the situation (e.g. personal or professional experience, professional identity, personal values) -
In pairs A • Briefly tell your partner about the
situation and what came up for you when reflecting on this and the questions we posed B Do: • Listen to both the content and ‘feeling tone’ of what A is telling you • Support A through empathising ( e.g. ‘that must have been difficult’) • Ask clarifying questions (e.g. ‘Can you say a bit more about….’or ‘how did you feel about that?) Don’t: • Give advice or try to ‘sort out’ the problem -
In groups of four • Discuss • Generally – how
did you find this exercise? Did you find it easy to talk about this situation? • Did any new insights come up? • Was there anything particular in the way your partner responded that helped you talk about the situation, or gain new insights or understanding? • How do you now feel about the ‘challenging situation’ you described