Policy design, impact assessment and evaluation

Capacity Building in Impact Evaluation: Counterfactual and Theory-Based Approaches

About this course

Does your organisation have qualified experts who are willing and able to properly manage evaluation activities? How can you ensure high-quality evaluations? Will evaluation results really provide the evidence required for decision-making, and help to improve policies and programmes?

Does your organisation have qualified experts who are willing and able to properly manage evaluation activities?

How can you ensure high-quality evaluations?

Will evaluation results really provide the evidence required for decision-making, and help to improve policies and programmes?

This online course will answer all the above questions, and specifically, will enhance your ability to design, manage and monitor the evaluations conducted by your organisation, irrespective of whether they are supported by EU funds.

What you will learn in this course:

  • The two most prominent approaches to the current evaluation system: Counterfactual (also known as experimental or quasi-experimental) and theory-based
  • What counterfactual approaches can and cannot do and the conditions in which they can be used
  • How theory-based approaches can provide a good alternative when such conditions are not met and discuss ways in which the two approaches can be combined

Course methodology:

The course will be led by experts who have decades of both research and practical experience in evaluation in an EU context.

There is no previous knowledge/experience required to attend this course, but if you do have some prior understanding, it could definitely help you get the maximum benefits out of this course.

By the end of the course, you will be able to:

  • Evaluate the pros and cons of different evaluation tools, and select the ones that are most relevant
  • Commission and manage evaluations and ensure better dissemination of the results
  • Design briefs for both internal and external evaluators, interact with them smoothly, and do an efficient assessment of their work quality
  • Integrate evaluation better into the policy and programming cycle so that it is actually used

After-course support:

You will get the slides as well as additional learning material to apply the new knowledge in your work.

  • Public officials from EU Member States, candidate countries, EU institutions and Agencies Managing authorities
  • Auditors, consultants, staff of NGOs, and other stakeholders involved in the management of evaluation procedures for policies, programmes, and projects
  • Professionals working with European Funds

Online Course
For this online course we make use of Zoom

Programme Organiser
Ms Belinda Vetter
Tel: + 31 43 3296382
b.vetter@eipa.eu

Discounts
EIPA member fee
EIPA offers a discount to all civil servants working for one of EIPA’s supporting countries, and civil servants working for an EU institution, body or agency

Who are the supporting countries?
Civil servants coming from the following EIPA supporting countries are entitled to get the reduced fee: Austria, Belgium, Bulgaria, Cyprus, Czech Republic, Denmark, Finland, France, Germany, Greece, Hungary, Ireland, Italy, Lithuania, Luxembourg, Malta, the Netherlands, Norway, Poland, Portugal, Spain, Sweden.

For all other participants, the regular fee applies

Early bird discount
The early bird discount is not cumulative with other discounts or promo codes, except for the EIPA member fee.

Who are the supporting countries?

Civil servants coming from the following EIPA supporting countries are entitled to get the reduced fee: Austria, Belgium, Bulgaria, Cyprus, Czech Republic, Denmark, Finland, France, Germany, Greece, Hungary, Ireland, Italy, Lithuania, Luxembourg, Malta, the Netherlands, Norway, Poland, Portugal, Spain, Sweden.
For all other participants, the regular fee applies.

Confirmation
Confirmation of registration will be forwarded to participants on receipt of the completed online registration form.

Payment
Prior payment is a condition for participation.

Cancellation policy
For administrative reasons you will be charged € 50 for cancellations received within 7 days before the activity begins. There is no charge for qualified substitute participants.

EIPA reserves the right to cancel the activity up to 1 week before the starting date. In that case, registration fees received will be fully reimbursed.

I have a question

Our experts

Gracia Vara Arribas
Project leader

Gracia Vara Arribas

EU Law and Public Procurement, EU Governance, Head of the European CAF Resource Centre

Programme

08.50Opening of on-line platform, Europe Summer Time (Amsterdam)
09.00Introduction to the course: presentation of the methodology to be followed
‘Tour de table’ – participants and experts present themselves. Needs assessment of the audience – evaluation on knowledge level
Gracia Vara Arribas and Nikolaos Sarris, Experts, EIPA
I. What is an evaluation and how to manage it
09.15The Policy Cycle in the European Commission
Presentation of policy cycle as implemented in the European Commission
Nikolaos Sarris, Expert, EIPA
09.45Strengthening the impact evaluation culture in the European Union – the lessons from the European Commission
The European Commission has institutionalised the evaluation culture, mainstreaming the different aspects of it – learn from the Commission experience the do’s and don’ts for your own organization
Gracia Vara Arribas, Expert, EIPA
10.30Coffee Break
II. Introduction to theory-based evaluation approaches and how to use them in practice
Bradford Rohmer, Senior Analyst, Oxford Research
10.45Different evaluation approaches and their practical use
All evaluations should be systematic, objective and evidence-based. But in other respects they differ depending on the issues and problems at hand. This session gives an overview of different evaluation approaches, their key features and how and why to use them. The discussion revolves around real-life examples to show the strengths and weaknesses of various evaluation approaches and ways to design and apply them in practice.
12.30Lunch break
13.30Theory-based evaluation: What is it, how does it work, and how can it be used to evaluate impact?
Theory-based evaluation (TBE) is versatile but is only useful insomuch as it is tailored to the intervention in question. This session runs through the different approaches to TBE (e.g. realistevaluation, contribution analysis, qualitative comparative analysis), the circumstances they are suited to and tools used to employ them. Again, the focus is on practical examples that participantscan apply as commissioners or practitioners of evaluations.
14.30Coffee break
14.45Practical Session 1: a practical example of a theory-based evaluation
16.30End of the first day
09.20Opening of on-line platform, Europe Summer Time (Amsterdam)
III. Introduction to counterfactual evaluation approaches and how to use them in practice
Stephen Morris, Professor of Evaluation, PERU, Manchester Metropolitan University
09.30Counterfactual approaches to causation
Early on in the process of designing an evaluation you will need to consider how a counterfactualwill be identified and estimated. This session will show you that once having identified a potential counterfactual you will then need to consider how you will measure counterfactual outcomes and possible collect control data from them. This will set ground for the next session of the programme.
10.30Randomised Controlled Trials (RCTs) as a way of identifying counterfactuals
A randomised controlled trial is an evaluation methodology which aims to draw an objectivepicture of the difference that a project, programme or policy makes by comparing its effect on separate groups. There is a real need to gain a clearer understanding of who requests and uses RCTs, and this session will provide that knowledge.
11.30Coffee break
11.45Practical Session 2: designing an experiment
Participants design an experiment and solve problems from a Commissioner´s perspective.
12.45Lunch break
13.45Quasi-experimental approaches
Overview of quasi-experimental methods and techniques to quantitatively evaluate the impactof an intervention. Using ample examples, the approaches covered include regression, difference-in-differences and matching. ITS and regression discontinuity will also be discussed.
14.45Coffee break
15.00Practical Session 3: using quasi-experimental approaches
Participants discuss the practical application of quasi-experimental approaches, also in cases where randomisation is not possible.
16.00Wrapping up and closing of the course
16.30End of the course

What former participants say

Leave a review

Review
Privacy

Capacity Building in Impact Evaluation: Counterfactual and Theory-Based Approaches

13 Dec 2021 - 14 Dec 2021
Online
Register before: 3 Dec 2021

Simultaneous interpretation can be offered. Contact the programme organizer for more information and conditions.

 900 per attendee
 810 for EIPA members
Registration closed

Discount policy

EIPA offers a discount to all civil servants working for one of EIPA’s supporting countries, and civil servants working for an EU institution, body or agency.

Who are the supporting countries?

Civil servants coming from the following EIPA supporting countries are entitled to get the reduced fee: Austria, Belgium, Bulgaria, Cyprus, Czech Republic, Denmark, Finland, France, Germany, Greece, Hungary, Ireland, Italy, Lithuania, Luxembourg, Malta, the Netherlands, Norway, Poland, Portugal, Spain, Sweden.

For all other participants, the regular fee applies.

Early bird discount

For a limited number of training courses, EIPA offers an early bird discount.

For more information please contact us.

Subscribe to our newsletter

Related courses

Register before: 3 Dec 2021
Registration closed