COURSE ID: I-SS10 LANGUAGE: RESIDENTIAL

VI SUMMER SCHOOL – PROGRAM EVALUATION IN ACTION

Providing effective evaluation of economic, social and public health programs has become an increasingly important requirement for both public and private institutions. Program evaluation allows us to assess the effectiveness of a specific program or intervention, and to use the subsequent results to identify how we can improve the future implementation of the program. As such program evaluation can be an extremely valuable tool for managers and evaluators looking to improve the quality of their programs and improve participation outcomes for the individuals involved. TStat’s residential summer school offers participants a unique opportunity to acquire the requisite toolset, both theoretical and applied, for the correct implementation of effective modern micro-econometric methods for implementing evidence-based program evaluation using Stata. As such, the program has been developed to encompass both: standard statistical methods of program evaluation: regression-adjustment, matching, selection-models and difference-in-differences methodologies; and the more advanced econometric techniques: for example, instrumental variables, synthetic control method, and regression discontinuity design.

 

The School opens with an optional introductory one data introduction to the statistical software Stata module to enable participants unfamiliar with the statistical software Stata to acquire the necessary introductory toolset to enable them to fully participant in the applied sessions during the course of the week.

 

In contrast to previous editions of the school, the 2020 edition includes a supplementary evening Case Study Group session during which participants have the opportunity to present and discuss their own research agendas related to applied program evaluation. Course leaders will be available to discuss with participants the appropriates of the methodologies adopted in their case study, the interpretation of the results obtained and also to indicate potential problems to be aware of given the characteristics of the underlying data, as well as providing feedback and guidance on possible future developments of individual research agendas. The school closes with a Q & A style session entirely dedicated to real world program evaluation case-studies, where the participants will have the opportunity to actively engage with the course tutor in a current program evaluation case study.

 

In common with TStat’s training philosophy, throughout the course of the week theoretical sessions are reinforced by both case study examples, in which the course tutors discuss current research issues, highlight the potential pitfalls and advantages of individual techniques, and an applied (hands-on) segment during which participants have the opportunity to implement these newly acquired techniques using real data under the watchful eye of the course tutor. Specific attention is given to both the intuition behind the choice and implementation of a specific technique, given the analysis in hand is of the utmost importance. In this manner, the course leader is able to bridge the “often difficult” gap between abstract theoretical methodologies, and the practical issues one encounters when trying to implement such techniques on real data.

 

At the end of the school participants are expected to be able to master complex evaluation design by: identifying the type of data required in their specific policy framework; evaluating which specific econometric method is more appropriate for the analysis in hand; and finally extracting policy recommendations from the obtained results. Participants should leave the course being in a position to autonomously implement, with the aid of the Stata routines utilized during the sessions, the techniques discussed during the course of the school.

Managers, researchers and professionals working in public and private institutions needing to undertake econometric program evaluation analysis using micro data. Although these methodologies are commonly used to evaluate economic policy interventions in, for example, the labour market, investment activities of enterprises, education policy, regional development, etc., they can in fact be used across a variety of studies, such as Public Health sector or interventions in the Banking sector or Financial markets, which aim to estimate the ex-post impact of a given intervention or project on specific targets.

Participants are expected to have a working knowledge of Stata and an intermediate knowledge of econometrics, and in particular knowledge of Ordinary Least Squares, Logit and Probit regression models and instrumental variable analysis.

MODULE A | DAY 1 – STATA IN JUST ONE DAY!

SESSION I: INTRODUCTION – GETTING STARTED

Stata’s GUI
File types in Stata
Working interactively in Stata
Saving output: the log file
Interrupting Stata
Loading Stata databases
The Log Output File
Saving databases in Stata
Exiting the software

 

SESSION II: PRELIMINARY DATA ANALYSIS

A preliminary look at the data: describe, summarize commands
Abbreviations in Stata
Stata’s syntax
Summary statistics
Statistical Tables: table, tabstat and tabulate commands

 

SESSION III: DATA MANAGEMENT

Renaming variables
Selecting or eliminating variables
The count command
sort command
Creating sub-groups: the prefix by
Creating new variables: generate
Operators in Stata
The command assert
program evaluation in action
Missing values in Stata
Modifying variables: replace, recode
Creating Labels: variable labels and value labels
Creating dummy variables

 

SESSION IV: IMPORTING DATA FROM SPREADSHEETS

Import Excel and Export Excel commands
The insheet and outsheet commands
Reading in Text Data Files
Issues to watch out for when importing data

Missing values
String variables
Date variables

Redefining missing values
destring command
tostring command
dealing wih “messy” strings

 

SESSION V: GRAPHICS – A BRIEF INTRODUCTION

Stata’s syntax for two way graphs
Saving and exporting graphs
Useful graph commands
Personalizing a graph
Stata’s Graph Editor

 

APPENDIX A

Merging data bases
do files

 

APPENDIX B: MORE ADVANCES ISSUE (time permitting)

do files
Merging data bases
e-class and r-class variables
collapse command
preserve command
restore command

 

MODULE B | DAY 2 – THE BASICS OF EVIDENCE-BASED PROGRAM EVALUATION

SESSION I: AN INTRODUCTION TO PROGRAM EVALUATION

Mastering evidence-based program evaluation: an overview
The concept of counterfactual causality
Experimental and quasi-experimental settings
The selection bias problem
Selection on observables and selection on unobservables
Definition of treatment effects: types of effects and potential outcome
Notation and working assumptions
Available econometric methods: limits and advantages
Stata for effective program evaluation: user-written commands and the teffects package

 

SESSION II: WORKING UNDER OBSERVABLE SELECTION USING REGRESSION ADJUSTMENT

Program evaluation under observable selection
The control-function regression approach
Linear and non-linear Regression-Adjustment (RA)
Stata implementation with the commands teffects ra and ivtreatreg

 

DAY 3 – GETTING A BIT MORE MODEL-FREE VIA MATCHING AND REWEIGHTING

SESSION I: MASTERING THE MATCHING APPROACH

Why the Matching Approach can be superior to the Regression Adjustment?
The rationale and usefulness of the Matching method
The propensity score: definition, properties, usefulness
Matching identification of treatment effects
Matching on the covariates versus Matching on the propensity score
Matching in practice: how to run tests and robustness checks
Implementation in Stata

 

SESSION II: EMPOWERING YOUR SKILLS BY REWEIGHTING

From Matching to Reweighting: a short but instructive journey
Reweighting as a general approach to program effect’s evaluation
Reweighting on the propensity score
Advantages and limits of using a Reweighting procedure
Implementation in Stata

 

DAY 4 – RELAXING THE ASSUMPTION OF OBSERVABLE SELECTION USING INSTRUMENTAL-VARIABLES (IV) AND SELECTION MODELS

SESSION I: THE “WHY” AND THE “HOW” OF INSTRUMENTAL-VARIABLES

The causal logic of instrumental variables: definition and simple examples
Treatment endogeneity and consistent estimation of program effects
Types of IV methods
Implementation in Stata

 

SESSION II: DIGGING INTO THE IV APPROACH WITH THE LOCAL AVERAGE TREATMENT EFFECT

The Local Average Treatment Effect (LATE) setting: an illustrative example
The relation between LATE and IV
A primer into Regression Discontinuity Design (RD)
Stata implementation

 

SESSION III: ALTERNATIVES TO IV AND THE SELECTION MODELS

Dealing with unobservable selection when an IV is not available
The Heckman selection model and its generalization
Implementation in Stata

 

DAY 5 – DIFFERENCE-IN-DIFFERENCE

SESSION I: UNDERSTANDING AND MASTERING THE DIFFERENCE-IN-DIFFERENCES

Difference-in-differences (DID) program evaluation setting
DID with longitudinal data
DID with repeated cross-section
The analysis of pre- and post-treatment effects
Implementation in Stata

 

SESSION II: SYNTHETIC CONTROL METHOD

The Synthetic Control Method (SCM): setting
Parametric and nonparametric SCM with Stata implementation
The placebo test and its graphing
Stata applications on real datasets

 

WINE MEETS PROGRAM EVALUATION: EVENING SUPPLEMENTARY WORKGROUP

During this informal early evening gathering, participants are given the opportunity to come together to discuss, via a poster or short presentation, their own research and/or program evaluation projects, whilst enjoying an offering of Tuscan Tapas and a glass of Tuscan wine.

 

DAY 6 – PROGRAM EVALUATION IN PRACTICE

SESSION I: A TEMPLATE FOR AN EFFECTIVE EVIDENCE-BASED PROGRAM EVALUATION

Designing an ex-post program evaluation: an overview
Description of the main steps of analysis from data collection to reporting
The use of the results for policy making
Reporting and communicating program evaluation results

 

SESSION II: PROGRAM EVALUATION CASE-STUDY

Implementing evidence-based program evaluation techniques: a case-study
The potential and limitation of program evaluation: a discussion
Summing-up and concluding remarks

 

PARTICIPANTS’ Q & A SLOT

In this concluding session participants are offered the opportunity to delve deeper into specific arguments of interest, when we open up the floor to questions and discussions of specific aspects of program evaluation relevant to their research work.

 

REFERENCES

A Gentle Introduction to Stata, 6th Ed., Alan Acock (2018) Stata Press
Data Analysis Using Stata, 3rd Ed., Ulrich Kohler, Frauke Kreuter (2012) Stata Press
Data Management Using Stata: A Practical Handbook, Michael N. Mitchell, (2010) Stata Press
The Workflow of Data Analysis Using Stata, J. Scott Long (2009) Stata Press
Mostly Harmless Econometrics: An Empiricist’s Companion, Joshua D. Angrist e Jorn-Steffen Pischke (2008) Princeton University Press
Microeconometrics Using Stata, Colin Cameron and Pravin K. Trivedi (2010) Stata Press
Econometric evaluation of socio-economic programs: theory and applications, Giovanni Cerulli (2015) Springer Verlag

The summer school is expected in Florence from the 24th to the 29th of August 2020.

 

CISL Studium Centre ♦ Via della Piazzola, 71 Florence (Italy)

Dr. Una-Louise BELL, TStat Training | TStat S.r.l.

Dr. Giovanni CERULLI, National Research Council of Italy

Dr. Roberto GABRIELE, University of Trento

ENTIRE WEEK (MODULES A plus B, 6 days)
Full-Time Students*: € 1620.00
Academic: € 2640.00
Commercial: € 3900.00

 

MODULE B (5 days)
Full-Time Students*: € 1350.00
Academic: € 2200.00
Commercial: € 3250.00

 

*To be eligible for student prices, participants must provide proof of their full-time student status for the current academic year. Residential costs for full time students are completely sponsored by TStat Training through our Investing in Young Researchers Programme. Participation is however restricted to a maximum of 3 students.

Fees are subject to VAT (applied at the current Italian rate of 22%). Under current EU fiscal regulations, VAT will not however applied to companies, Institutions or Universities providing a valid tax registration number.

Please note that a non-refundable deposit of €100.00 for full-time students and €250.00 for Academic and Commercial participants, is required to secure a place and is payable upon registration. The number of participants is limited to 15. Places will be allocated on a first come, first serve basis.

Course fees cover: i) teaching materials (copies of lecture slides, databases and Stata routines used during the summer school; ii) a temporary licence of Stata valid for 30 days from the day before the beginning of the school; iii) half board accommodation (breakfast, lunch and coffee breaks) in a single room at the CISL Studium Centre or equivalent (6 nights for entire school, 5 nights for Modules B). Participants requiring accommodation the night of the final day of the school, are requested to contact us as soon as possible.

To maximize the usefulness of this summer school, we strongly recommend that participants bring their own laptops with them, to enable them to actively participate in the empirical sessions.

Individuals interested in attending this summer school must return their completed registration forms by email (training@tstat.eu) to TStat by the 1st August 2020.


NAME

EMAIL

OBJECT

ADDITIONAL COMMENTS

[recaptcha]
I authorise the use of my personal data pursuant to Article 13 of L. Decree no 196 / 2003

TStat’s residential summer school offers participants a unique opportunity to acquire the requisite toolset, both theoretical and applied, for the correct implementation of effective modern micro-econometric methods for implementing evidence-based program evaluation using Stata. As such, the program has been developed to encompass both: standard statistical methods of program evaluation: regression-adjustment, matching, selection-models and difference-in-differences methodologies; and the more advanced econometric techniques: for example, instrumental variables, synthetic control method, and regression discontinuity design.