Using Quantitative Methods to Study Parliament PSA Parliaments & Legislatures Workshop, Uni. of Leeds Peter Allen p.allen@qmul.ac.uk http://www.peter-allen.co.uk School of Politics & International Relations Queen Mary University of London 25 June 2015 Peter Allen (QMUL) Quants & Parliament 25 June 2015 1 / 13
Overview 1 Introduction What are quantitative methods? What is the process? 2 Identifying variables & collecting data Thinking about variables Collecting data 3 Methods for analysing Parliamentary data Problems Solutions 4 Conclusion Peter Allen (QMUL) Quants & Parliament 25 June 2015 2 / 13
What are quantitative methods? Descriptive and analytic/inferential statistics Describing what we see in the data Helping us move towards establishing which things affect one another, how much, and when AKA - establishing causal relationships These might include: counting the numbers of something of interest (bills, MPs, speeches, etc.) Peter Allen (QMUL) Quants & Parliament 25 June 2015 3 / 13
What are quantitative methods? Descriptive and analytic/inferential statistics Describing what we see in the data Helping us move towards establishing which things affect one another, how much, and when AKA - establishing causal relationships These might include: counting the numbers of something of interest (bills, MPs, speeches, etc.) using bivariate analysis to separate the instances of this thing by some other thing (e.g. MPs by sex) Peter Allen (QMUL) Quants & Parliament 25 June 2015 3 / 13
What are quantitative methods? Descriptive and analytic/inferential statistics Describing what we see in the data Helping us move towards establishing which things affect one another, how much, and when AKA - establishing causal relationships These might include: counting the numbers of something of interest (bills, MPs, speeches, etc.) using bivariate analysis to separate the instances of this thing by some other thing (e.g. MPs by sex) using multivariate analysis to establish how some thing is the outcome / is affected by variation in other things (e.g. modelling voting behaviour in relation to MP age, sex, party, safeness of seat, etc.) Peter Allen (QMUL) Quants & Parliament 25 June 2015 3 / 13
What is the process? I see doing quantitative research on Parliament as a three-stage process: 1. Identify the variables of interest based on the research question Peter Allen (QMUL) Quants & Parliament 25 June 2015 4 / 13
What is the process? I see doing quantitative research on Parliament as a three-stage process: 1. Identify the variables of interest based on the research question 2. Collect the required data as well as possible Peter Allen (QMUL) Quants & Parliament 25 June 2015 4 / 13
What is the process? I see doing quantitative research on Parliament as a three-stage process: 1. Identify the variables of interest based on the research question 2. Collect the required data as well as possible 3. Analyse the data appropriately and write-up findings Peter Allen (QMUL) Quants & Parliament 25 June 2015 4 / 13
Variable identification and development What is the research question? Think about how I would design a study to find out the effect of my variable of interest ceteris paribus i.e. if I could run an experiment to find out the answer to my research question, what would the treatment be? Example What is the effect of having worked in Parliament before becoming an MP on subsequent career performance? Can I run an experiment and randomly assign this experience to different MPs? No. Instead, I can observe what has happened and collect relevant data on variables of interest i.e. on pre-parliamentary experience Peter Allen (QMUL) Quants & Parliament 25 June 2015 5 / 13
Another example of variable selection Think of another example I might want to know what the policy impact of Select Committees is. What are the variables I will be interested in? Policy outcomes; a measure of SC activity; a measure of contact between policymakers and SC, etc. Legislative studies is strong on this; there are a lot of potential variables to choose from... The challenge is picking the right one and measuring it well (more on that in a bit) Peter Allen (QMUL) Quants & Parliament 25 June 2015 6 / 13
Collecting data on Parliament There are a growing number of sources of parliamentary data Examples of data sources 1 TheyWorkForYou 2 Hansard 3 tapir (tools for accessing parliamentary information in R) 4 data.parliament.uk 5 Dod s Parliamentary Companion 6 Lexis and other media repositories for lobby stories 7 Other researchers parliamentarycandidates.org; Michael Rush parliamentary candidate dataset; Pippa Norris constituency dataset Peter Allen (QMUL) Quants & Parliament 25 June 2015 7 / 13
A warning... Quantitative methods are not (necessarily) a quicker or easier option that extensive qualitative fieldwork variable development takes time Peter Allen (QMUL) Quants & Parliament 25 June 2015 8 / 13
A warning... Quantitative methods are not (necessarily) a quicker or easier option that extensive qualitative fieldwork variable development takes time data collection takes time Peter Allen (QMUL) Quants & Parliament 25 June 2015 8 / 13
A warning... Quantitative methods are not (necessarily) a quicker or easier option that extensive qualitative fieldwork variable development takes time data collection takes time and analysis, done well, takes time Peter Allen (QMUL) Quants & Parliament 25 June 2015 8 / 13
Common problems when analysing legislative data Small sample sizes you are often dealing with small-n datasets e.g. you are interested in MPs from one party, or women MPs, or BME MPs your sample size can be small relative to working with the British Election Study data, for example Your wonderful dependent variable is confounded by party/parliamentary procedure often, the visible manifestation of the thing you are interested in will not really help you answer your question the real answer will be hidden (hence, qualitative methods... ) Peter Allen (QMUL) Quants & Parliament 25 June 2015 9 / 13
Dealing with these problems What can you do to deal with these issues? 1. Focus on descriptive statistics 2. Use methods designed to correct for small sample bias (next slide) 3. Mixed-methods! Peter Allen (QMUL) Quants & Parliament 25 June 2015 10 / 13
Useful methods for legislative studies Most descriptive techniques frequencies, crosstabulation, etc. For small samples exact tests Simple regression models ordinal logistic regression, etc. For small samples, penalised logistic regression models Peter Allen (QMUL) Quants & Parliament 25 June 2015 11 / 13
More exciting methods If you have larger datasets: latent modelling (latent class, factor analysis, etc.) Mediation and moderation modelling conditional process analysis huge opportunities for text analysis see Gary King s work, Bara, Bicquelet and Weale on abortion debates in Commons, other work from Eggers & Spirling Figure: Graph showing rates of parliamentary speech from Eggers & Spirling 2014 Peter Allen (QMUL) Quants & Parliament 25 June 2015 12 / 13
Concluding thoughts legislative studies offers interesting opportunities for quantitative methods there are limitations, but these can be dealt with increasing opportunities for data collection big data future? legislative studies still healthy, has incorporated advanced quantitaitve methods pretty well Questions, etc. p.allen@qmul.ac.uk Peter Allen (QMUL) Quants & Parliament 25 June 2015 13 / 13