Replication Codes for ‘Camouflaged Propaganda: A Survey Experiment on Political Native Advertising’
Yaoyao Dai and Luwei Luqiu
This is an R notebook with codes to replicate all the tables and figures in the paper “Camouflaged Propaganda: A Survey Experiment on Political Native Advertising” and its Online Appendices. To replicate the analysis, please download the replication data to the same folder as this .rmd file and run the .rmd file in R. You can also copy and paste the code chunks to a .r file and run the codes only.
Load data and packages
rm(list=ls(all=TRUE))
options(scipen=999)
library(foreign)
library(MASS)
library(stargazer)
library(car)Codebook
| Variable | Variable Type | Values |
|---|---|---|
| G.Ewapo | Group Indicator | \(1\): Primed Wapo Group; \(0\): Otherwise |
| G.Wapo | Group Indicator | \(1\): Wapo Group; \(0\): Otherwise |
| G.Etele | Group Indicator | \(1\): Primed Telegraph Group; \(0\): Otherwise |
| G.Tele | Group Indicator | \(1\): Telegrapg Group; \(0\): Otherwise |
| G.CDw | Group Indicator | \(1\): China Daily Article W Group; \(0\): Otherwise |
| G.CDt | Group Indicator | \(1\): China Daily Article T Group; \(0\): Otherwise |
| Tr.WapoA | Treatment Indicator | \(1\): Received Wapo China Watch article (Primed Wapo and Wapo Groups); \(0\): Otherwise (The other 4 groups) |
| Tr.TeleA | Treatment Indicator | \(1\): Received Telegraph China Watch article (Primed Telegraph and Telegraph Groups); \(0\): Otherwise (The other 4 groups) |
| CorSour | DV: Detecting source | \(1\): Identified source of artical as China Daily ; \(0\): Otherwise |
| PreTrust.Wapo | Pre-treatment Trust in The Washington Post | \(1\): None at all; \(2\): A little; \(3\): Somewhat trust \(4\): A Moderate amount; \(5\): A lot; \(6\): A great deal |
| PreTrust.Tele | Pre-treatment Trust in The Telegraph | \(1\): None at all; \(2\): A little; \(3\): Somewhat trust \(4\): A Moderate amount; \(5\): A lot; \(6\): A great deal |
| PostTrust.Wapo | Post-treatment Trust in The Washington Post | \(1\): None at all; \(2\): A little; \(3\): Somewhat trust \(4\): A Moderate amount; \(5\): A lot; \(6\): A great deal; NA: Respondents did not receive Wapo article or did not answer |
| PostTrust.Tele | Post-treatment Trust in The Telegraph | \(1\): None at all; \(2\): A little; \(3\): Somewhat trust \(4\): A Moderate amount; \(5\): A lot; \(6\): A great deal; NA: Respondents did not receive Telegraph article or did not answer |
| Conv | DV: Level of Persuasiveness | \(1\): Highly unconvincing; \(2\): Unconvincing; \(3\): Somewhat convincing; \(4\): Convincing; \(5\): Highly convincing |
| Female | Demographic | \(1\): Female; \(0\): Male |
| Income | Demographic: Pre-tax household income | \(1\): Less than \(\$10,000\); \(2\): \(\$10,000\) to \(\$19,999\); \(3\): \(\$20,000\) to \(\$29,999\); \(4\): \(\$30,000\) to \(\$39,999\); \(5\): \(\$40,000\) to \(\$49,999\); \(6\): \(\$50,000\) to \(\$59,999\) \(7\): \(\$60,000\) to \(\$69,999\); \(8\): \(\$70,000\) to \(\$79,999\); \(9\): \(\$80,000\) to \(\$89,999\) \(10\): \(\$90,000\) to \(\$99,999\); \(11\):\(\$100,000\) to \(\$149,999\); \(12\): \(\$150,000\) or more |
| Edu | Demographic | \(1\): Less than high school degree; \(2\): High school graduate (high school diploma or equivalent including GED); \(3\): Some college but no degree; \(4\): Associate degree in college (2-year); \(5\): Bachelor’s degree in college (4-year); \(6\): Master’s degree; \(7\): Professional degree (JD, MD); \(8\): Doctoral degree |
Descriptive Statistics
Appendix A Table 1
g.Ewapo <- subset(PNA.data, G.Ewapo==1) #Primed Wapo Group in Figure 1
g.Wapo <- subset(PNA.data, G.Wapo==1) # Wapo Group in Figure 1
g.Etele <- subset(PNA.data, G.Etele==1) # Primed Telegraph Group in Figure 1
g.Tele <- subset(PNA.data, G.Tele==1) # Telegrapg Group in Figure 1
g.CDw <- subset(PNA.data, G.CDw==1) # China Daily Article W in Figure 1
g.CDt <- subset(PNA.data, G.CDt==1) # China Daily Article T in Figure 1t.test(g.Wapo$Age,g.Ewapo$Age)
t.test(g.Etele$Age,g.Ewapo$Age)
t.test(g.Tele$Age,g.Ewapo$Age)
t.test(g.CDw$Age,g.Ewapo$Age)
t.test(g.CDt$Age,g.Ewapo$Age)t.test(g.Wapo$Female,g.Ewapo$Female)
t.test(g.Etele$Female,g.Ewapo$Female)
t.test(g.Tele$Female,g.Ewapo$Female)
t.test(g.CDw$Female,g.Ewapo$Female)
t.test(g.CDt$Female,g.Ewapo$Female)sd(g.Ewapo$Female)
sd(g.Wapo$Female)
sd(g.Etele$Female)
sd(g.Tele$Female)
sd(g.CDw$Female)
sd(g.CDt$Female)t.test(g.Wapo$Edu,g.Ewapo$Edu)
t.test(g.Etele$Edu,g.Wapo$Edu)
t.test(g.Tele$Edu,g.Wapo$Edu)
t.test(g.CDw$Edu,g.Wapo$Edu)
t.test(g.CDt$Edu,g.Wapo$Edu)Detecting PNA
Table 1
Model.source <- glm(CorSour ~ Tr.WapoA + EduP + Age, data=PNA.data, family=binomial(link = "logit"))
Model.source.c <- glm(CorSour ~ Tr.WapoA + EduP + Exp + Lit + TrustMed
+ Age + Edu + Income + Female, data=PNA.data, family=binomial(link = "logit"))stargazer(Model.source, Model.source.c, dep.var.labels = "Correct Source",
covariate.labels = c("Washington Post","Education Priming", "Media Experience","Media Literacy",
"MediaTrust", "Age", "Education", "Income","Female","Constant"),
type="text")##
## ==============================================
## Dependent variable:
## ----------------------------
## Correct Source
## (1) (2)
## ----------------------------------------------
## Washington Post 2.726*** 2.854***
## (0.241) (0.252)
##
## Education Priming 0.413* 0.468*
## (0.238) (0.246)
##
## Media Experience 0.028
## (0.041)
##
## Media Literacy -0.022
## (0.035)
##
## MediaTrust 0.049
## (0.048)
##
## Age 0.011 0.012
## (0.010) (0.011)
##
## Education 0.128
## (0.097)
##
## Income 0.038
## (0.039)
##
## Female -0.110
## (0.252)
##
## Constant -2.271*** -3.754*
## (0.473) (2.045)
##
## ----------------------------------------------
## Observations 444 439
## Log Likelihood -222.462 -213.998
## Akaike Inf. Crit. 452.923 447.995
## ==============================================
## Note: *p<0.1; **p<0.05; ***p<0.01Reputational Costs of PNA
Table 2
Table 3
#Self-Detected
t.test(treated.data$PostTrust.Wapo[which(treated.data$CorSour == 1)],
treated.data$PreTrust.Wapo[which(treated.data$CorSour == 1)], paired =T)
#Not Self-Detected
t.test(treated.data$PostTrust.Wapo[which(treated.data$CorSour == 0)],
treated.data$PreTrust.Wapo[which(treated.data$CorSour == 0)], paired =T)
#Self-Detected
t.test(treated.data$PostTrust.Tele[which(treated.data$CorSour == 1)],
treated.data$PreTrust.Tele[which(treated.data$CorSour == 1)], paired =T)
#Not Self-Detected
t.test(treated.data$PostTrust.Tele[which(treated.data$CorSour == 0)],
treated.data$PreTrust.Tele[which(treated.data$CorSour == 0)], paired =T)Persuasiveness of PNA
Table 4
# Washington Post China Watch article comparing to the same article on China Daily
# Respondents did identify true source on their own
t.test(g.Wapo[which(g.Wapo$CorSour==1),]$Conv, g.CDw$Conv)
# Washington Post China Watch article comparing to the same article on China Daily
# Respondents did not identify true source on their own
t.test(g.Wapo[which(g.Wapo$CorSour==0),]$Conv, g.CDw$Conv)# The Telegraph China Watch article comparing to the same article on China Daily
# Respondents did identify true source on their own
t.test(g.Tele[which(g.Tele$CorSour==1),]$Conv, g.CDt$Conv)
# The Telegraph China Watch article comparing to the same article on China Daily
# Respondents did not identify true source on their own
t.test(g.Tele[which(g.Tele$CorSour==0),]$Conv, g.CDt$Conv)Appendix
Quality of Treatment Articles
Quality of Responses
Appendix A Table 3
ml.source <- glm(CorSour ~ Tr.WapoA + EduP + Age, data=long, family=binomial(link = "logit"))
ml.source.c <- glm(CorSour ~ Tr.WapoA + EduP + Exp + Lit + TrustMed
+ Age + Edu + Income + Female, data=long, family=binomial(link = "logit"))stargazer(ml.source, ml.source.c, dep.var.labels = "Correct Source",
covariate.labels = c("Washington Post","Education Priming", "Media Experience","Media Literacy",
"MediaTrust", "Age", "Education", "Income","Female","Constant"),
type="text")##
## ==============================================
## Dependent variable:
## ----------------------------
## Correct Source
## (1) (2)
## ----------------------------------------------
## Washington Post 2.821*** 2.963***
## (0.259) (0.272)
##
## Education Priming 0.590** 0.676**
## (0.255) (0.263)
##
## Media Experience 0.035
## (0.044)
##
## Media Literacy -0.023
## (0.037)
##
## MediaTrust 0.053
## (0.050)
##
## Age 0.012 0.013
## (0.011) (0.012)
##
## Education 0.128
## (0.103)
##
## Income 0.044
## (0.041)
##
## Female -0.207
## (0.269)
##
## Constant -2.448*** -4.053*
## (0.509) (2.178)
##
## ----------------------------------------------
## Observations 404 401
## Log Likelihood -197.340 -190.458
## Akaike Inf. Crit. 402.681 400.916
## ==============================================
## Note: *p<0.1; **p<0.05; ***p<0.01Appendix A Table 4
Appendix A Table 5
lg.Ewapo <- subset(long, G.Ewapo==1)
lg.Wapo <- subset(long, G.Wapo==1)
lg.Etele <- subset(long, G.Etele==1)
lg.Tele <- subset(long, G.Tele==1)
lg.CDw <- subset(long, G.CDw==1)
lg.CDt <- subset(long, G.CDt==1)Additional Robustness Checks
Appendix A Table 6
# Create a single variable of post-pre difference in trust in hosting media outlet
treated.data$dif.trust.wapo <- treated.data$PostTrust.Wapo - treated.data$PreTrust.Wapo
treated.data$dif.trust.tele <- treated.data$PostTrust.Tele - treated.data$PreTrust.Tele
treated.data$Dif.trustB <- recode(treated.data$dif.trust.wapo,"NA=0") +
recode(treated.data$dif.trust.tele,"NA=0") # Create a single variable of pre-treatment trust in host media outlet
# PreTrustB indicates a Wapo Group respondent's pre-treatment trust to Washington Post
# and a Telegraph Group respondent's pre-treatment trust to Telegraph
treated.data$PreTrustB <- replace(treated.data$PreTrust.Wapo, which(treated.data$Tr.TeleA==1),0) + replace(treated.data$PreTrust.Tele, which(treated.data$Tr.WapoA==1),0)model.diff.trustB <- lm(Dif.trustB ~ PreTrustB + CorSour ,data=treated.data)
model.diff.trustB.c <- lm(Dif.trustB ~ PreTrustB+CorSour + Age + Female + Edu
+ Income + TrustMed + Lit + Exp,data=treated.data)
model.diff.trustB.cc <- lm(Dif.trustB ~ CorSour+Age + PreTrustB + TrustMed + Lit + Exp,
data=treated.data)stargazer(model.diff.trustB,model.diff.trustB.cc,model.diff.trustB.c,
dep.var.labels = "Post-Pre Difference in Trust", covariate.labels =
c("Pre-treatment Trust","Media Trust","Media Literacy","Media Experience",
"Identify Source","Age","Female","Education","Income","Constant")
,type="text")##
## ===========================================================================================
## Dependent variable:
## -----------------------------------------------------------------------
## Post-Pre Difference in Trust
## (1) (2) (3)
## -------------------------------------------------------------------------------------------
## Pre-treatment Trust -0.625*** -0.655*** -0.661***
## (0.052) (0.053) (0.053)
##
## Media Trust 0.040 0.039
## (0.029) (0.029)
##
## Media Literacy -0.023 -0.024
## (0.020) (0.020)
##
## Media Experience 0.068*** 0.066***
## (0.025) (0.025)
##
## Identify Source 0.325** 0.287** 0.279*
## (0.144) (0.144) (0.144)
##
## Age -0.004 -0.002
## (0.007) (0.007)
##
## Female -0.031
## (0.148)
##
## Education 0.098*
## (0.056)
##
## Income -0.036
## (0.023)
##
## Constant 1.428*** 0.644 0.528
## (0.215) (1.119) (1.159)
##
## -------------------------------------------------------------------------------------------
## Observations 303 300 300
## R2 0.328 0.357 0.367
## Adjusted R2 0.324 0.344 0.347
## Residual Std. Error 1.240 (df = 300) 1.224 (df = 293) 1.221 (df = 290)
## F Statistic 73.367*** (df = 2; 300) 27.098*** (df = 6; 293) 18.658*** (df = 9; 290)
## ===========================================================================================
## Note: *p<0.1; **p<0.05; ***p<0.01Appendix A Figure 1
new.data.c <- data.frame(PreTrustB = 4, CorSour = 1, Age = median(treated.data$Age,na.rm=T),
Female = median(treated.data$Female,na.rm=T),
Edu = median(treated.data$Edu,na.rm=T),
Income = median(treated.data$Income,na.rm=T),
TrustMed = median(treated.data$TrustMed,na.rm=T),
Lit = median(treated.data$Lit,na.rm=T), Exp = seq(5,25))
new.data.nc <- data.frame(PreTrustB = 4, CorSour = 0, Age = median(treated.data$Age,na.rm=T),
Female = median(treated.data$Female,na.rm=T),
Edu = median(treated.data$Edu,na.rm=T),
Income = median(treated.data$Income,na.rm=T),
TrustMed = median(treated.data$TrustMed,na.rm=T),
Lit = median(treated.data$Lit,na.rm=T), Exp = seq(5,25))predict.c <- predict(model.diff.trustB.c, new.data.c, interval = "confidence")
predict.nc <- predict(model.diff.trustB.c, new.data.nc, interval = "confidence")#par(mar=c(10,7,10,3))
plot(1,1, xlim=c(5,25), ylim=c(-3,0.5),type="n",ylab="Predicted Post-Pre Change in Trust",
xlab="Media Experience",main="",bty="n")
lines(x= new.data.nc$Exp, y = predict.nc[,1], type = 'l',col = "darkblue", lwd=2)
lines(x= new.data.c$Exp, y = predict.c[,1], type = 'l', col='firebrick2', lwd=2)
#abline(h=0,col = "darkgray", lty = 2,lwd=1.5)
lines(x= new.data.nc$Exp, y = predict.nc[,2], type = 'l',col = "darkblue", lty=2)
lines(x= new.data.nc$Exp, y = predict.nc[,3], type = 'l',col = "darkblue", lty=2)
lines(x= new.data.c$Exp, y = predict.c[,2], type = 'l', col='firebrick2', lty=2)
lines(x= new.data.c$Exp, y = predict.c[,3], type = 'l', col='firebrick2', lty=2)
legend("bottomright",c("Identify Source Correctly", "Identify Source Incorrectly"),
col = c("firebrick2","darkblue"), lty = 1, bty = "n", lwd=2)
Appendix A Table 7
exp.wapo <- PNA.data[which(PNA.data$G.Wapo==1 | PNA.data$G.CDw==1),]
exp.tele <- PNA.data[which(PNA.data$G.Tele==1 | PNA.data$G.CDt==1),]exp.wapo$p.wapo <- replace(exp.wapo$G.Wapo,which(exp.wapo$CorSour==1),0)
exp.wapo$p.cd <- replace(exp.wapo$G.Wapo,which(exp.wapo$CorSour==0),1)
exp.tele$p.tele <- replace(exp.tele$G.Tele,which(exp.tele$CorSour==1),0)
exp.tele$p.cd <- replace(exp.tele$G.Tele,which(exp.tele$CorSour==0),0)model.conv.w <- polr(as.ordered(exp.wapo$Conv) ~ Tr.WapoA + Age + Edu + Exp + Lit
+ TrustMed + Income + Female , data = exp.wapo, method="logistic")
model.conv.wp <- polr(as.ordered(exp.wapo$Conv) ~ p.wapo + p.cd + Age + Edu + Exp + Lit
+ TrustMed + Income + Female , data = exp.wapo, method="logistic")
model.conv.t <- polr(as.ordered(exp.tele$Conv) ~ Tr.TeleA + Age + Edu + Exp + Lit
+ TrustMed + Income + Female, data = exp.tele, method="logistic")
model.conv.tp <- polr(as.ordered(exp.tele$Conv) ~ p.tele + p.cd + Age + Edu + Exp + Lit
+ TrustMed + Income + Female, data = exp.tele, method="logistic")stargazer(model.conv.w, model.conv.t, model.conv.wp,model.conv.tp,
covariate.labels = c("Washington Post China Watch","Telegraph China Watch",
"Perceived as Washington Post","Perceived as Telegraph",
"Perceived as China Daily","Age","Education","Media Experience","Media Literacy",
"Media Trust","Income","Female"),type="text")##
## =============================================================
## Dependent variable:
## --------------------------------
## Conv Conv Conv Conv
## (1) (2) (3) (4)
## -------------------------------------------------------------
## Washington Post China Watch 0.142
## (0.266)
##
## Telegraph China Watch 0.649**
## (0.257)
##
## Perceived as Washington Post 0.604
## (0.422)
##
## Perceived as Telegraph 0.797***
## (0.270)
##
## Perceived as China Daily -0.020 -0.241
## (0.289) (0.516)
##
## Age -0.003 -0.010 -0.003 -0.008
## (0.011) (0.011) (0.011) (0.011)
##
## Education -0.164* -0.069 -0.149 -0.072
## (0.097) (0.101) (0.098) (0.101)
##
## Media Experience 0.017 0.0003 0.022 0.002
## (0.048) (0.046) (0.048) (0.046)
##
## Media Literacy -0.019 0.014 -0.019 0.008
## (0.040) (0.036) (0.040) (0.036)
##
## Media Trust 0.040 0.084* 0.042 0.084*
## (0.052) (0.047) (0.053) (0.047)
##
## Income 0.006 0.065 -0.001 0.074*
## (0.045) (0.041) (0.045) (0.041)
##
## Female 0.477* -0.372 0.473* -0.344
## (0.284) (0.266) (0.284) (0.266)
##
## -------------------------------------------------------------
## Observations 208 217 208 217
## =============================================================
## Note: *p<0.1; **p<0.05; ***p<0.01