Text as Data. Justin Grimmer. Associate Professor Department of Political Science Stanford University. November 20th, PDF Free Download

Text as Data Justin Grimmer Associate Professor Department of Political Science Stanford University November 20th, 2014 Justin Grimmer (Stanford University) Text as Data November 20th, 2014 1 / 24

Ideological Scaling 1) Task - Measure political actors position in policy space - Low dimensional representation of beliefs 2) Objective function - Linear Discriminant Analysis (ish) Wordscores (today) - Item Response Theory Wordfish - Item Response Theory + Roll Call Votes Issue-specific ideal points (12/2) 3) Optimization - Wordscores straightforward, based on training texts - Wordfish EM, MCMC methods 4) Validation - What is the goal of embedding? - What is the gold standard? Justin Grimmer (Stanford University) Text as Data November 20th, 2014 2 / 24

The Spatial Model - Suppose we have actor i (i = 1, 2, 3,..., N) Justin Grimmer (Stanford University) Text as Data November 20th, 2014 3 / 24

The Spatial Model - Suppose we have actor i (i = 1, 2, 3,..., N) - Actor has ideal point θ i R M Justin Grimmer (Stanford University) Text as Data November 20th, 2014 3 / 24

The Spatial Model - Suppose we have actor i (i = 1, 2, 3,..., N) - Actor has ideal point θ i R M - We describe actor i s utility from proposal p R M with utility function Justin Grimmer (Stanford University) Text as Data November 20th, 2014 3 / 24

The Spatial Model - Suppose we have actor i (i = 1, 2, 3,..., N) - Actor has ideal point θ i R M - We describe actor i s utility from proposal p R M with utility function u i (θ i, p) = d(θ i, p) Justin Grimmer (Stanford University) Text as Data November 20th, 2014 3 / 24

The Spatial Model - Suppose we have actor i (i = 1, 2, 3,..., N) - Actor has ideal point θ i R M - We describe actor i s utility from proposal p R M with utility function u i (θ i, p) = d(θ i, p) L = ( θ il p }{{} l ) 2 l=1 ideal policy Justin Grimmer (Stanford University) Text as Data November 20th, 2014 3 / 24

The Spatial Model - Suppose we have actor i (i = 1, 2, 3,..., N) - Actor has ideal point θ i R M - We describe actor i s utility from proposal p R M with utility function u i (θ i, p) = d(θ i, p) L = ( θ il p }{{} l ) 2 l=1 ideal policy Estimation goal: θ i Justin Grimmer (Stanford University) Text as Data November 20th, 2014 3 / 24

The Spatial Model - Suppose we have actor i (i = 1, 2, 3,..., N) - Actor has ideal point θ i R M - We describe actor i s utility from proposal p R M with utility function u i (θ i, p) = d(θ i, p) L = ( θ il p }{{} l ) 2 l=1 ideal policy Estimation goal: θ i Scaling placing actors in low-dimensional space (like principal components!) Justin Grimmer (Stanford University) Text as Data November 20th, 2014 3 / 24

Estimating Ideal Points: Roll Call Data and the US Congress US Congress and Roll Call Justin Grimmer (Stanford University) Text as Data November 20th, 2014 4 / 24

Estimating Ideal Points: Roll Call Data and the US Congress US Congress and Roll Call - Poole and Rosenthal voteview Justin Grimmer (Stanford University) Text as Data November 20th, 2014 4 / 24

Estimating Ideal Points: Roll Call Data and the US Congress US Congress and Roll Call - Poole and Rosenthal voteview - Roll Call Data 1789-Present Justin Grimmer (Stanford University) Text as Data November 20th, 2014 4 / 24

Estimating Ideal Points: Roll Call Data and the US Congress US Congress and Roll Call - Poole and Rosenthal voteview - Roll Call Data 1789-Present - NOMINATE methods place legislators on one dimension, estimate of ideology - Wildly successful: - Estimates are accurate: face validity Congressional scholars agree upon - Insightful unidimensional Congress Justin Grimmer (Stanford University) Text as Data November 20th, 2014 4 / 24

Estimating Ideal Points: Roll Call Data and the US Congress US Congress and Roll Call - Poole and Rosenthal voteview - Roll Call Data 1789-Present - NOMINATE methods place legislators on one dimension, estimate of ideology - Wildly successful: - Estimates are accurate: face validity Congressional scholars agree upon - Insightful unidimensional Congress - Extensible: insight of IRT allows model to be embedded in many forms Justin Grimmer (Stanford University) Text as Data November 20th, 2014 4 / 24

Estimating Ideal Points: Roll Call Data and the US Congress US Congress and Roll Call - Poole and Rosenthal voteview - Roll Call Data 1789-Present - NOMINATE methods place legislators on one dimension, estimate of ideology - Wildly successful: - Estimates are accurate: face validity Congressional scholars agree upon - Insightful unidimensional Congress - Extensible: insight of IRT allows model to be embedded in many forms - Widely used: hard to write a paper on American political institutions with ideal points Justin Grimmer (Stanford University) Text as Data November 20th, 2014 4 / 24

Estimating Ideal Points in General Two Limitations with the NOMINATE project: Justin Grimmer (Stanford University) Text as Data November 20th, 2014 5 / 24

Estimating Ideal Points in General Two Limitations with the NOMINATE project: 1) US Congress is distinct Justin Grimmer (Stanford University) Text as Data November 20th, 2014 5 / 24

Estimating Ideal Points in General Two Limitations with the NOMINATE project: 1) US Congress is distinct roll call votes fail to measure ideology in other settings Justin Grimmer (Stanford University) Text as Data November 20th, 2014 5 / 24

Estimating Ideal Points in General Two Limitations with the NOMINATE project: 1) US Congress is distinct roll call votes fail to measure ideology in other settings - Weak party pressure individual discretion on votes - Parliamentary systems no discretion, no variation. - Spirling and Quinn (2011) mixture model like models for blocs in UK Parliament 2) Not everyone votes! - Voters survey responses (but problems with that) Justin Grimmer (Stanford University) Text as Data November 20th, 2014 5 / 24

Estimating Ideal Points in General Two Limitations with the NOMINATE project: 1) US Congress is distinct roll call votes fail to measure ideology in other settings - Weak party pressure individual discretion on votes - Parliamentary systems no discretion, no variation. - Spirling and Quinn (2011) mixture model like models for blocs in UK Parliament 2) Not everyone votes! - Voters survey responses (but problems with that) - Challengers NPAT surveys (but they don t fill those out anymore) Justin Grimmer (Stanford University) Text as Data November 20th, 2014 5 / 24

Estimating Ideal Points in General Two Limitations with the NOMINATE project: 1) US Congress is distinct roll call votes fail to measure ideology in other settings - Weak party pressure individual discretion on votes - Parliamentary systems no discretion, no variation. - Spirling and Quinn (2011) mixture model like models for blocs in UK Parliament 2) Not everyone votes! - Voters survey responses (but problems with that) - Challengers NPAT surveys (but they don t fill those out anymore) - Bonica (2013, 2014) estimate ideology from donations (but not everyone donates) Justin Grimmer (Stanford University) Text as Data November 20th, 2014 5 / 24

Estimating Ideal Points in General But Everyone talks! Justin Grimmer (Stanford University) Text as Data November 20th, 2014 6 / 24

Estimating Ideal Points in General But Everyone talks! - If we could scale based on conversation, we can measure ideology anywhere Justin Grimmer (Stanford University) Text as Data November 20th, 2014 6 / 24

Estimating Ideal Points in General But Everyone talks! - If we could scale based on conversation, we can measure ideology anywhere - Much of the literature relies upon intuition from US Congress - Hard not to find ideology - Behavior that is primarily ideological - Reality: scaling is much more difficult than roll call voting examples Justin Grimmer (Stanford University) Text as Data November 20th, 2014 6 / 24

Estimating Ideal Points in General But Everyone talks! - If we could scale based on conversation, we can measure ideology anywhere - Much of the literature relies upon intuition from US Congress - Hard not to find ideology - Behavior that is primarily ideological - Reality: scaling is much more difficult than roll call voting examples - Hard to find ideology Justin Grimmer (Stanford University) Text as Data November 20th, 2014 6 / 24

Estimating Ideal Points in General But Everyone talks! - If we could scale based on conversation, we can measure ideology anywhere - Much of the literature relies upon intuition from US Congress - Hard not to find ideology - Behavior that is primarily ideological - Reality: scaling is much more difficult than roll call voting examples - Hard to find ideology - Much of political speech reveals little about position on ideological spectrum advertising, regional Justin Grimmer (Stanford University) Text as Data November 20th, 2014 6 / 24

Estimating Ideal Points in General But Everyone talks! - If we could scale based on conversation, we can measure ideology anywhere - Much of the literature relies upon intuition from US Congress - Hard not to find ideology - Behavior that is primarily ideological - Reality: scaling is much more difficult than roll call voting examples - Hard to find ideology - Much of political speech reveals little about position on ideological spectrum advertising, regional Healthy skepticism! Justin Grimmer (Stanford University) Text as Data November 20th, 2014 6 / 24

Our plan Justin Grimmer (Stanford University) Text as Data November 20th, 2014 7 / 24

Our plan 1) Wordscores supervised scaling Justin Grimmer (Stanford University) Text as Data November 20th, 2014 7 / 24

Our plan 1) Wordscores supervised scaling 2) Wordfish single dimension Justin Grimmer (Stanford University) Text as Data November 20th, 2014 7 / 24

Wordscores Big in Europe Justin Grimmer (Stanford University) Text as Data November 20th, 2014 8 / 24

Wordscores, Like the Hoff Big in Europe Justin Grimmer (Stanford University) Text as Data November 20th, 2014 8 / 24

Wordscores: Objective Function For each legislator i, suppose we observe D i documents. Justin Grimmer (Stanford University) Text as Data November 20th, 2014 9 / 24

Wordscores: Objective Function For each legislator i, suppose we observe D i documents. Define: Justin Grimmer (Stanford University) Text as Data November 20th, 2014 9 / 24

Wordscores: Objective Function For each legislator i, suppose we observe D i documents. Define: x i = D i l=1 x il Justin Grimmer (Stanford University) Text as Data November 20th, 2014 9 / 24

Wordscores: Objective Function For each legislator i, suppose we observe D i documents. Define: x i = = D i l=1 D i x il (x il1, x il2,..., x ilj ) l=1 Justin Grimmer (Stanford University) Text as Data November 20th, 2014 9 / 24

Wordscores: Objective Function For each legislator i, suppose we observe D i documents. Define: x i = = D i l=1 D i x il (x il1, x il2,..., x ilj ) l=1 x i aggregation across documents. Justin Grimmer (Stanford University) Text as Data November 20th, 2014 9 / 24

Wordscores: Objective Functions Choose two legislators as exemplars Justin Grimmer (Stanford University) Text as Data November 20th, 2014 10 / 24

Wordscores: Objective Functions Choose two legislators as exemplars - Legislator L {1, 2,..., N} is liberal. Y L = 1 Justin Grimmer (Stanford University) Text as Data November 20th, 2014 10 / 24

Wordscores: Objective Functions Choose two legislators as exemplars - Legislator L {1, 2,..., N} is liberal. Y L = 1 - For example, might select Elizabeth Warren Justin Grimmer (Stanford University) Text as Data November 20th, 2014 10 / 24

Wordscores: Objective Functions Choose two legislators as exemplars - Legislator L {1, 2,..., N} is liberal. Y L = 1 - For example, might select Elizabeth Warren - Legislator C {1, 2,..., N} is Conservative. Y C = 1 - For example, might select Ted Cruz Justin Grimmer (Stanford University) Text as Data November 20th, 2014 10 / 24

Wordscores: Objective Functions Choose two legislators as exemplars - Legislator L {1, 2,..., N} is liberal. Y L = 1 - For example, might select Elizabeth Warren - Legislator C {1, 2,..., N} is Conservative. Y C = 1 - For example, might select Ted Cruz For each word j we can define: Justin Grimmer (Stanford University) Text as Data November 20th, 2014 10 / 24

Wordscores: Objective Functions Choose two legislators as exemplars - Legislator L {1, 2,..., N} is liberal. Y L = 1 - For example, might select Elizabeth Warren - Legislator C {1, 2,..., N} is Conservative. Y C = 1 - For example, might select Ted Cruz For each word j we can define: P jl = Probability of word from Liberal Justin Grimmer (Stanford University) Text as Data November 20th, 2014 10 / 24

Wordscores: Objective Functions Choose two legislators as exemplars - Legislator L {1, 2,..., N} is liberal. Y L = 1 - For example, might select Elizabeth Warren - Legislator C {1, 2,..., N} is Conservative. Y C = 1 - For example, might select Ted Cruz For each word j we can define: P jl = Probability of word from Liberal P jc = Probability of word from Conservative Justin Grimmer (Stanford University) Text as Data November 20th, 2014 10 / 24

Wordscores: Objective Functions Choose two legislators as exemplars - Legislator L {1, 2,..., N} is liberal. Y L = 1 - For example, might select Elizabeth Warren - Legislator C {1, 2,..., N} is Conservative. Y C = 1 - For example, might select Ted Cruz For each word j we can define: P jl = Probability of word from Liberal P jc = Probability of word from Conservative Define the score for word j Justin Grimmer (Stanford University) Text as Data November 20th, 2014 10 / 24

Wordscores: Objective Functions Choose two legislators as exemplars - Legislator L {1, 2,..., N} is liberal. Y L = 1 - For example, might select Elizabeth Warren - Legislator C {1, 2,..., N} is Conservative. Y C = 1 - For example, might select Ted Cruz For each word j we can define: P jl = Probability of word from Liberal P jc = Probability of word from Conservative Define the score for word j S j = Y C P jc + Y L P jl Justin Grimmer (Stanford University) Text as Data November 20th, 2014 10 / 24

Wordscores: Objective Functions Choose two legislators as exemplars - Legislator L {1, 2,..., N} is liberal. Y L = 1 - For example, might select Elizabeth Warren - Legislator C {1, 2,..., N} is Conservative. Y C = 1 - For example, might select Ted Cruz For each word j we can define: P jl = Probability of word from Liberal P jc = Probability of word from Conservative Define the score for word j S j = Y C P jc + Y L P jl = P jc P jl Justin Grimmer (Stanford University) Text as Data November 20th, 2014 10 / 24

Wordscores: Objective Functions Scale other legislators: Justin Grimmer (Stanford University) Text as Data November 20th, 2014 11 / 24

Wordscores: Objective Functions Scale other legislators: N i = J j=1 x i Justin Grimmer (Stanford University) Text as Data November 20th, 2014 11 / 24

Wordscores: Objective Functions Scale other legislators: N i = J j=1 x i ˆθ i is Justin Grimmer (Stanford University) Text as Data November 20th, 2014 11 / 24

Wordscores: Objective Functions Scale other legislators: N i = J j=1 x i ˆθ i is ˆθ i = J j=1 ( xij N i ) S j Justin Grimmer (Stanford University) Text as Data November 20th, 2014 11 / 24

Wordscores: Objective Functions Scale other legislators: N i = J j=1 x i ˆθ i is ˆθ i = J j=1 ( xij N i ) S j = x i N i S Justin Grimmer (Stanford University) Text as Data November 20th, 2014 11 / 24

Wordscores: Optimization N L = N C = J m=1 J m=1 x ml x mc Justin Grimmer (Stanford University) Text as Data November 20th, 2014 12 / 24

Wordscores: Optimization N L = N C = J m=1 J m=1 x ml x mc Estimate P jl, P jc, and S j Justin Grimmer (Stanford University) Text as Data November 20th, 2014 12 / 24

Wordscores: Optimization N L = N C = J m=1 J m=1 x ml x mc Estimate P jl, P jc, and S j P jl = x jl N L x jl N L + x jc N C Justin Grimmer (Stanford University) Text as Data November 20th, 2014 12 / 24

Wordscores: Optimization N L = N C = J m=1 J m=1 x ml x mc Estimate P jl, P jc, and S j P jl = x jl N L x jl N L + x jc N C P jc = 1 P jl = x jc N C x jl N L + x jc N C Justin Grimmer (Stanford University) Text as Data November 20th, 2014 12 / 24

Wordscores: Optimization N L = N C = J m=1 J m=1 x ml x mc Estimate P jl, P jc, and S j P jl = x jl N L x jl N L + x jc N C P jc = 1 P jl = S j = P jc P jl x jc N C x jl N L + x jc N C Justin Grimmer (Stanford University) Text as Data November 20th, 2014 12 / 24

Applied to the Senate Press Releases L = Ted Kennedy C = Tom Coburn Apply to other senators. Justin Grimmer (Stanford University) Text as Data November 20th, 2014 13 / 24

Applying to Senate Press Releases Gold Standard Scaling from NOMINATE Density 0.0 0.2 0.4 0.6 0.8 1.0 3 2 1 0 1 2 3 Ideal Point Estimate Justin Grimmer (Stanford University) Text as Data November 20th, 2014 14 / 24

Applying to Senate Press Releases WordScores Density 0 5 10 15 20 25 0.2 0.1 0.0 0.1 0.2 Score Justin Grimmer (Stanford University) Text as Data November 20th, 2014 14 / 24

Supervised Scaling Wordscores is one example of supervised embedding: Justin Grimmer (Stanford University) Text as Data November 20th, 2014 15 / 24

Supervised Scaling Wordscores is one example of supervised embedding: 1) Linear Discriminant Analysis (LDA) the Federalist Papers Justin Grimmer (Stanford University) Text as Data November 20th, 2014 15 / 24

Supervised Scaling Wordscores is one example of supervised embedding: 1) Linear Discriminant Analysis (LDA) the Federalist Papers 2) Multinomial Inverse Regression Justin Grimmer (Stanford University) Text as Data November 20th, 2014 15 / 24

Supervised Scaling Wordscores is one example of supervised embedding: 1) Linear Discriminant Analysis (LDA) the Federalist Papers 2) Multinomial Inverse Regression 3) Any method that separates words The findings will depend heavily on supervision Justin Grimmer (Stanford University) Text as Data November 20th, 2014 15 / 24

Supervised Scaling Wordscores is one example of supervised embedding: 1) Linear Discriminant Analysis (LDA) the Federalist Papers 2) Multinomial Inverse Regression 3) Any method that separates words The findings will depend heavily on supervision - Sensitive to who is chosen Justin Grimmer (Stanford University) Text as Data November 20th, 2014 15 / 24

Supervised Scaling Wordscores is one example of supervised embedding: 1) Linear Discriminant Analysis (LDA) the Federalist Papers 2) Multinomial Inverse Regression 3) Any method that separates words The findings will depend heavily on supervision - Sensitive to who is chosen - False prediction problem speech accomplishes many goals, only some of which are ideological Justin Grimmer (Stanford University) Text as Data November 20th, 2014 15 / 24

What Can Go Wrong with Wordscores? Lowe (2008): Discusses potentially problematic wordscores properties Justin Grimmer (Stanford University) Text as Data November 20th, 2014 16 / 24

What Can Go Wrong with Wordscores? Lowe (2008): Discusses potentially problematic wordscores properties 1) Each word is weighted equally (fixable with different scoring procedure) 2) Unique words are conflated with centrist (fixable with MCQ fightin words style algorithm) 3) General problem: hard to interpret lack of a model Justin Grimmer (Stanford University) Text as Data November 20th, 2014 16 / 24

What Can Go Wrong with Wordscores? Lowe (2008): Discusses potentially problematic wordscores properties 1) Each word is weighted equally (fixable with different scoring procedure) 2) Unique words are conflated with centrist (fixable with MCQ fightin words style algorithm) 3) General problem: hard to interpret lack of a model To be fair: fast, nonparametric, and novel [trailblazing] method for scoring documents (starts conversation) Justin Grimmer (Stanford University) Text as Data November 20th, 2014 16 / 24

Unsupervised Embedding Basic idea: Justin Grimmer (Stanford University) Text as Data November 20th, 2014 17 / 24

Unsupervised Embedding Basic idea: - Actors have underlying latent position Justin Grimmer (Stanford University) Text as Data November 20th, 2014 17 / 24

Unsupervised Embedding Basic idea: - Actors have underlying latent position - Actors articulate that latent position in their speech Justin Grimmer (Stanford University) Text as Data November 20th, 2014 17 / 24

Unsupervised Embedding Basic idea: - Actors have underlying latent position - Actors articulate that latent position in their speech - This is associated with word usage, so high discriminating words correspond to ideological speech Justin Grimmer (Stanford University) Text as Data November 20th, 2014 17 / 24

Unsupervised Embedding Basic idea: - Actors have underlying latent position - Actors articulate that latent position in their speech - This is associated with word usage, so high discriminating words correspond to ideological speech - Some words discriminate better than others encode that in our model Justin Grimmer (Stanford University) Text as Data November 20th, 2014 17 / 24

Unsupervised Embedding Basic idea: - Actors have underlying latent position - Actors articulate that latent position in their speech - This is associated with word usage, so high discriminating words correspond to ideological speech - Some words discriminate better than others encode that in our model Simplest model: Principal Components Justin Grimmer (Stanford University) Text as Data November 20th, 2014 17 / 24

Application of Principal Components in R Consider press releases from 2005 US Senators Justin Grimmer (Stanford University) Text as Data November 20th, 2014 18 / 24

Application of Principal Components in R Consider press releases from 2005 US Senators Define x i = (x i1, x i2,..., x ij ) as the rate senator i uses J words. Justin Grimmer (Stanford University) Text as Data November 20th, 2014 18 / 24

Application of Principal Components in R Consider press releases from 2005 US Senators Define x i = (x i1, x i2,..., x ij ) as the rate senator i uses J words. x ij = No. Times i uses word j No. words i uses Justin Grimmer (Stanford University) Text as Data November 20th, 2014 18 / 24

Application of Principal Components in R Consider press releases from 2005 US Senators Define x i = (x i1, x i2,..., x ij ) as the rate senator i uses J words. x ij = No. Times i uses word j No. words i uses dtm: 100 2796 matrix containing word rates for senators Justin Grimmer (Stanford University) Text as Data November 20th, 2014 18 / 24

Application of Principal Components in R Consider press releases from 2005 US Senators Define x i = (x i1, x i2,..., x ij ) as the rate senator i uses J words. x ij = No. Times i uses word j No. words i uses dtm: 100 2796 matrix containing word rates for senators prcomp(dtm) applies principal components Justin Grimmer (Stanford University) Text as Data November 20th, 2014 18 / 24

Application of Principal Components in R Projection 0.02 0.00 0.02 0.04 0.06 Dewine2005 Alexander2005 Bunning2005 Shelby2005 Warner2005 Graham2005 Isakson2005 Chafee2005 Specter2005 Allen2005 Feingold2005 Kennedy2005 Hagel2005 Byrd2005 Bayh2005 Obama2005 Cantwell2005 Brownback2005 Stevens2005 Lautenberg2005 Kyl2005 Conrad2005 Feinstein2005 Grassley2005 Rockefeller2005 Thomas2005 Hutchison2005 Craig2005 Murkowski2005 Salazar2005 Landrieu2005 McConnell2005 Lott2005 Akaka2005 Coleman2005 Crapo2005 Jeffords2005 Martinez2005 Demint2005 Corzine2005 Boxer2005 Murray2005 Lieberman2005 Carper2005 Vitter2005 Talent2005 Baucus2005 Sessions2005 Enzi2005 Ensign2005 Roberts2005 Dorgan2005 Wyden2005 Allard2005 BenNelson2005 Pryor2005 Durbin2005 Inhofe2005 Voinovich2005 Domenici2005 Frist2005 Clinton2005 Dodd2005 Snowe2005 Dayton2005 Johnson2005 Burns2005 Smith2005 Harkin2005 Dole2005 Lugar2005 Thune2005 Sununu2005 Schumer2005 Sarbanes2005 Bingaman2005 Chambliss2005 Kohl2005 Burr2005 Lincoln2005 Mikulski2005 Santorum2005 Collins2005 Bennett2005 Bond2005 Gregg2005 Stabenow2005 Inouye2005 Reed2005 Levin2005 Leahy2005 Reid2005 BillNelson2005 Kerry2005 Cornyn2005 Biden2005 McCain2005 Coburn2005 Hatch2005 Cochran2005 0 20 40 60 80 100 Order Justin Grimmer (Stanford University) Text as Data November 20th, 2014 19 / 24

Application of Principal Components in R Shelby2005 Dewine20 Difference: Credit Claiming, Position Taking 0.4 0.2 0.0 0.2 0.4 0.6 Graham2005 Bond2005 Bunning2005 Sarbanes2005 Specter2005 Bennett2005 Allen2005 Warner2005 Isakson2005 Chafee2005 Allard2005 Dole2005 Gregg2005 Harkin2005 Chambliss2005 Inouye2005 Talent2005 Burr2005 Mikulski2005 Thune2005 Collins2005 Levin2005 Johnson2005 Dodd2005 Bingaman2005 Lugar2005 Lincoln2005 Santorum2005 Stabenow2005 Schumer2005 Durbin2005 Burns2005 Kohl2005 Hutchison2005 Domenici2005 Conrad2005 Roberts2005 Dorgan2005 Clinton2005 Dayton2005 Crapo2005 Lott2005 Carper2005 Enzi2005 Wyden2005 Pryor2005 Snowe2005 Sununu2005 Murray2005 Boxer2005 BenNelson2005 Inhofe2005 Reed2005 Obama2005 Landrieu2005 Sessions2005 Vitter2005 Voinovich2005 Smith2005 Brownback2005 Coleman2005 Hatch2005 Ensign2005 chran2005 Bayh2005 Grassley2005 Thomas2005 Martinez2005 Lieberman2005 Hagel2005 Byrd2005 McConnell2005 Stevens2005 Feinstein2005 Rockefeller2005 Akaka2005 Baucus2005 Murkowski2005 Salazar2005 Craig2005 BillNelson2005 Cornyn2005 Corzine2005 Demint2005 Frist2005 Leahy2005 Feingold2005 Lautenberg2005 Jeffords2005 Coburn2005 Biden2005 Kyl2005 McCain2005 Kerry2005 Kennedy2005 Cantwell2005 Reid2005 Alexander200 0.02 0.00 0.02 0.04 0.06 Principal Component Justin Grimmer (Stanford University) Text as Data November 20th, 2014 19 / 24

Probabilistic Unsupervised Embeddings Principal components is powerful Justin Grimmer (Stanford University) Text as Data November 20th, 2014 20 / 24

Probabilistic Unsupervised Embeddings Principal components is powerful statistical model for unsupervised scaling Justin Grimmer (Stanford University) Text as Data November 20th, 2014 20 / 24

Probabilistic Unsupervised Embeddings Principal components is powerful statistical model for unsupervised scaling Item Response Theory (IRT) Justin Grimmer (Stanford University) Text as Data November 20th, 2014 20 / 24

Probabilistic Unsupervised Embeddings Principal components is powerful statistical model for unsupervised scaling Item Response Theory (IRT) - Origins: educational testing - Jackman (2002), Clinton, Jackman, and Rivers (2004) apply to roll call voting - Power of IRT: a) Estimate ideal points with few observations b) Makes clear how to extend models c) Tuesday IRT and topic models to scale with votes + text - Clinton, Jackman, and Rivers (2004) intuition about IRT Justin Grimmer (Stanford University) Text as Data November 20th, 2014 20 / 24

Probabilistic Unsupervised Embeddings Principal components is powerful statistical model for unsupervised scaling Item Response Theory (IRT) - Origins: educational testing - Jackman (2002), Clinton, Jackman, and Rivers (2004) apply to roll call voting - Power of IRT: a) Estimate ideal points with few observations b) Makes clear how to extend models c) Tuesday IRT and topic models to scale with votes + text - Clinton, Jackman, and Rivers (2004) intuition about IRT - Rivers (2002) Identification conditions Justin Grimmer (Stanford University) Text as Data November 20th, 2014 20 / 24

Probabilistic Unsupervised Embeddings Principal components is powerful statistical model for unsupervised scaling Item Response Theory (IRT) - Origins: educational testing - Jackman (2002), Clinton, Jackman, and Rivers (2004) apply to roll call voting - Power of IRT: a) Estimate ideal points with few observations b) Makes clear how to extend models c) Tuesday IRT and topic models to scale with votes + text - Clinton, Jackman, and Rivers (2004) intuition about IRT - Rivers (2002) Identification conditions - Bonica (2014a, 2014b) uses IRT (like the one we re about to use) to scale donors Justin Grimmer (Stanford University) Text as Data November 20th, 2014 20 / 24

Probabilistic Unsupervised Embeddings Principal components is powerful statistical model for unsupervised scaling Item Response Theory (IRT) - Origins: educational testing - Jackman (2002), Clinton, Jackman, and Rivers (2004) apply to roll call voting - Power of IRT: a) Estimate ideal points with few observations b) Makes clear how to extend models c) Tuesday IRT and topic models to scale with votes + text - Clinton, Jackman, and Rivers (2004) intuition about IRT - Rivers (2002) Identification conditions - Bonica (2014a, 2014b) uses IRT (like the one we re about to use) to scale donors Monroe and Maeda (2005) and Slopkin and Proksch (2008) develop similar algorithms Justin Grimmer (Stanford University) Text as Data November 20th, 2014 20 / 24

WordFish Objective Function Suppose we have legislator i. Justin Grimmer (Stanford University) Text as Data November 20th, 2014 21 / 24

WordFish Objective Function Suppose we have legislator i. x ij Poisson(λ ij ) Justin Grimmer (Stanford University) Text as Data November 20th, 2014 21 / 24

WordFish Objective Function Suppose we have legislator i. x ij Poisson(λ ij ) λ ij = exp(α i + ψ j + β j θ i ) Justin Grimmer (Stanford University) Text as Data November 20th, 2014 21 / 24

WordFish Objective Function Suppose we have legislator i. Where, x ij Poisson(λ ij ) λ ij = exp(α i + ψ j + β j θ i ) Justin Grimmer (Stanford University) Text as Data November 20th, 2014 21 / 24

WordFish Objective Function Suppose we have legislator i. x ij Poisson(λ ij ) λ ij = exp(α i + ψ j + β j θ i ) Where, λ ij = Rate individual i uses word j Justin Grimmer (Stanford University) Text as Data November 20th, 2014 21 / 24

WordFish Objective Function Suppose we have legislator i. x ij Poisson(λ ij ) λ ij = exp(α i + ψ j + β j θ i ) Where, λ ij = Rate individual i uses word j α i = Individual i loquaciousness Justin Grimmer (Stanford University) Text as Data November 20th, 2014 21 / 24

WordFish Objective Function Suppose we have legislator i. x ij Poisson(λ ij ) λ ij = exp(α i + ψ j + β j θ i ) Where, λ ij = Rate individual i uses word j α i = Individual i loquaciousness ψ j = Word j s frequency β j = Word j s discrimination Justin Grimmer (Stanford University) Text as Data November 20th, 2014 21 / 24

WordFish Objective Function Suppose we have legislator i. x ij Poisson(λ ij ) λ ij = exp(α i + ψ j + β j θ i ) Where, λ ij = Rate individual i uses word j α i = Individual i loquaciousness ψ j = Word j s frequency β j = Word j s discrimination θ i = Legislator i s latent positions Justin Grimmer (Stanford University) Text as Data November 20th, 2014 21 / 24

WordFish Objective Function Suppose we have legislator i. x ij Poisson(λ ij ) λ ij = exp(α i + ψ j + β j θ i ) Where, λ ij = Rate individual i uses word j α i = Individual i loquaciousness ψ j = Word j s frequency β j = Word j s discrimination θ i = Legislator i s latent positions Regression of x ij on ideal points θ i, where we have to learn θ i Justin Grimmer (Stanford University) Text as Data November 20th, 2014 21 / 24

WordFish Objective Function/Optimization Implies the following posterior distribution: Justin Grimmer (Stanford University) Text as Data November 20th, 2014 22 / 24

WordFish Objective Function/Optimization Implies the following posterior distribution: p(θ, α, ψ, β) p(α)p(β)p(ψ)p(θ) N J exp [ (α i + ψ j + β j θ i )] (α i + ψ j + β j θ i ) x ij i=1 j=1 x ij! Justin Grimmer (Stanford University) Text as Data November 20th, 2014 22 / 24

WordFish Objective Function/Optimization Implies the following posterior distribution: p(θ, α, ψ, β) p(α)p(β)p(ψ)p(θ) N J exp [ (α i + ψ j + β j θ i )] (α i + ψ j + β j θ i ) x ij Estimate parameters: i=1 j=1 x ij! Justin Grimmer (Stanford University) Text as Data November 20th, 2014 22 / 24

WordFish Objective Function/Optimization Implies the following posterior distribution: p(θ, α, ψ, β) p(α)p(β)p(ψ)p(θ) N J exp [ (α i + ψ j + β j θ i )] (α i + ψ j + β j θ i ) x ij Estimate parameters: - EM-algorithm i=1 j=1 x ij! Justin Grimmer (Stanford University) Text as Data November 20th, 2014 22 / 24

WordFish Objective Function/Optimization Implies the following posterior distribution: p(θ, α, ψ, β) p(α)p(β)p(ψ)p(θ) N J exp [ (α i + ψ j + β j θ i )] (α i + ψ j + β j θ i ) x ij Estimate parameters: - EM-algorithm - MCMC algorithm i=1 j=1 x ij! Justin Grimmer (Stanford University) Text as Data November 20th, 2014 22 / 24

WordFish Objective Function/Optimization Implies the following posterior distribution: p(θ, α, ψ, β) p(α)p(β)p(ψ)p(θ) N J exp [ (α i + ψ j + β j θ i )] (α i + ψ j + β j θ i ) x ij i=1 j=1 Estimate parameters: - EM-algorithm - MCMC algorithm - Variational Approximation x ij! Justin Grimmer (Stanford University) Text as Data November 20th, 2014 22 / 24

WordFish Objective Function/Optimization Implies the following posterior distribution: p(θ, α, ψ, β) p(α)p(β)p(ψ)p(θ) N J exp [ (α i + ψ j + β j θ i )] (α i + ψ j + β j θ i ) x ij i=1 j=1 Estimate parameters: - EM-algorithm - MCMC algorithm - Variational Approximation Wordfish package in R x ij! Justin Grimmer (Stanford University) Text as Data November 20th, 2014 22 / 24

Applications: German Party Manifestos Wordfish and German Platforms Policy Position 2 1 0 1 2 PDS Greens SPD CDU/CSU FDP 1990 1994 1998 2002 2005 Year Justin Grimmer (Stanford University) Text as Data November 20th, 2014 23 / 24

Applications: German Party Manifestos 2 1 0 1 2 3 4 0.4 0.0 0.2 0.4 0.6 Wordfish Scaling Scaling, Grimmer (2012) Dem GOP Wordfish and Senate Press Releases 2 1 0 1 2 3 4 0.0 0.2 0.4 Wordfish Scaling Density Justin Grimmer (Stanford University) Text as Data November 20th, 2014 23 / 24

The Problem with Text-Based Scaling What does validation mean? 1) Replicate NOMINATE, DIME, or other gold standards? 2) Agreement with experts 3) Prediction of other behavior Must answer this to make progress on pure text scaling Justin Grimmer (Stanford University) Text as Data November 20th, 2014 24 / 24

Text as Data. Justin Grimmer. Associate Professor Department of Political Science Stanford University. November 20th, 2014