[Survey research in the digital age], [Probability and
non-probability sampling], [Computer-administered interviews],
[Combining surveys and big data], [Additions and extensions]
Matthew J. Salganik
Department of Sociology
Princeton University
1) Introduction
2) Observing behavior
3) Asking questions
4) Running experiments
5) Mass collaboration
6) Ethics
7) The future
Sampling Interviews Data environment
1st era Area probability Face-to-face Stand-alone
2nd era Random digital dial Telephone Stand-alone
probability
3rd era Non-probability Computer-administered Linked
http://www.chicagotribune.com/news/nationworld/politics/chi- chicagodays- deweydefeats- story-story.html
I Probability sample (roughly): every unit from a frame population has a known
and non-zero probability of inclusion
I Probability sample (roughly): every unit from a frame population has a known
and non-zero probability of inclusion
I Not all probability samples look like miniature versions of the population
I Probability sample (roughly): every unit from a frame population has a known
and non-zero probability of inclusion
I Not all probability samples look like miniature versions of the population
I But, with appropriate weighting, probability samples can yield unbiased estimates
of the frame population
Main insights from probability sampling:
I How you collect your data impacts how you make inference
Main insights from probability sampling:
I How you collect your data impacts how you make inference
I Focus on properties of estimators not properties samples
Main idea and equation in sampling and estimation:
P
yi /πi
ȳ = i∈s
ˆ
N
where πi is person i’s probability of inclusion
Sometimes called:
I Horvitz-Thompson estimator
I π estimator
Inference from probability samples in theory
)
respondents
estimates
known information about sampling
Inference from probability samples in theory
)
respondents
estimates
known information about sampling
Inference from probability samples in practice
respondents
estimated information about sampling estimates
| {z }
auxiliary information + assumptions
Inference from probability samples in theory
)
respondents
estimates
known information about sampling
Inference from probability samples in practice
respondents
estimated information about sampling estimates
| {z }
auxiliary information + assumptions
Inference from non-probability samples
respondents
estimated information about sampling estimates
| {z }
auxiliary information + assumptions
Imagine that you want to estimate the average height of Princeton students.
I Assume 50% are male and 50% are female
I You stand outside Lewis Library and recruit a non-random sample of 60 Princeton
students
I Males (n= 20): Average height: 180cm
I Females (n=40): Average heigh: 170cm
What is your estimate of the average height?
I sample mean = 173.3cm ( 180∗20+170∗40
20+40 )
I sample mean = 173.3cm ( 180∗20+170∗40
20+40 )
I weighted estimate = 175cm (180 ∗ 0.5 + 170 ∗ 0.5)
I sample mean = 173.3cm ( 180∗20+170∗40
20+40 )
I weighted estimate = 175cm (180 ∗ 0.5 + 170 ∗ 0.5)
I weighted estimate uses auxiliary information and assumptions
I sample mean = 173.3cm ( 180∗20+170∗40
20+40 )
I weighted estimate = 175cm (180 ∗ 0.5 + 170 ∗ 0.5)
I weighted estimate uses auxiliary information and assumptions
I How could this go wrong?
Wang et al (2015)
I about 750,000 interviews
I about 350,000 unique respondents
Wrap-up:
I Samples don’t need to look like mini-populations
Wrap-up:
I Samples don’t need to look like mini-populations
I Key to making good estimates is for estimation process to account for the
sampling process
Wrap-up:
I Samples don’t need to look like mini-populations
I Key to making good estimates is for estimation process to account for the
sampling process
I There is not a bright-line difference between probability sampling in practice and
non-probability sampling
Wrap-up:
I Samples don’t need to look like mini-populations
I Key to making good estimates is for estimation process to account for the
sampling process
I There is not a bright-line difference between probability sampling in practice and
non-probability sampling
I To learn more: Lohr (2009) or Sandal et al (2013)
[Survey research in the digital age], [Probability and
non-probability sampling], [Computer-administered interviews],
[Combining surveys and big data], [Additions and extensions]
Matthew J. Salganik
Department of Sociology
Princeton University
