Russian Troll Tweet data, Machine Learning with accuracy 99.6%

You can read motivation for my work here: https://myabakhova.blogspot.com/2022/01/russian-troll-data-investigation.html I published EDA, Feature Engineering and Machine Learning on my Kaggle account in 3 parts due to constrains of Kaggle resources:

Part 1. EDA

Part 2. Feature Engineering

Part 3. Machine Learning, a test accuracy 99.6% 

 Conclusion

Analysis of a Twitter account behavior helps a lot in determining paid trolls. The most helpful for detection properties are the ones related to propaganda methods. Apparently trolls have specified guidelines for their posts and they stick to them. I see it as convenient because we can set up filters for catching the most significant phenomena, and then check a whole account activity. 

In addition the most important for prediction features turned out to be not very dependable on languages but mostly on troll account activity. Thus we can do it for other languages, and do not limit it to Russian trolls posting English texts. 

Please upvote it on Kaggle if you like it!

Russian Troll data investigation

I found Russian Troll Data set on kaggle and analyzed it:  

Part 1. EDA

Part 2. Feature Engineering

Part 3. Machine Learning, a test accuracy 99.6%

This project for me is personal. I experienced the propaganda machine of the Soviet Union and am horrified to see it used on Americans. As a young adult in Soviet Russia, I succumbed to brainwashing and had no idea what was really going on. "Everybody always lies" had been the norm. I came to the US in the 90s and I was amazed that in the US, deception is not normalized as in my homeland. I have been surprised when Americans trusted my words while Soviet people would look at me with suspicion regardless of the situation. I am saddened that Americans’ trust has been abused by paid trolls. I worry that deception will be normalized here in the States as well. I believe that propaganda is a form of psychological abuse.

My first encounter with the trolls was in Russian forums. At first I thought that these are badly informed people and I tried to give them links to correct information. Only in a few months when I read an article about Russian troll farms did I realize what was going on. I have seen a guy still posting articles about omnipotent Hilary Clinton who is set on destroying the world for a month after Donald Trump won. You see, a government propaganda machine is bureaucratic, and it took time for them to change their instructions. Or one of them bragged how they scheduled their issues of compromising materials to the most damage, in particular he was proud about “pedophile ring” lies right before the presidential election. This quote shows a more current example: “I would believe in [Biden] Rebuild plan when the potholes on my street will be fixed”, because in Russia a president actually has the power to command local authorities.

When I saw these data become available, I wanted to help prevent people getting brainwashed like I was. I intend to study the data, to extract English tweets and to compare them with tweets from Americans. The difficulty with the data is that Russian propaganda uses 60/40 Göbbels 60/40 method: https://zarinazabrisky.medium.com/rotten-herrings-and-crucified-children-c4c278466985 It means that they mix in posts from real Americans to confuse their detection. With the tables I plan to check if there is a way to distinguish paid Russian trolls from Americans using Machine Learning methods.

The “Blue Planet II” effect: Twitter data wrangling & ggplot2 plot

Introduction

One of my clients graciously donated me the data set for my presentation. It was mined from Twitter choosing posts containing a word “bottle”.

The goal was to investigate if “Blue Planet II” documentary influence can be detected on Twitter. The series of 8 episodes debuted on 29 October 2017 in the United Kingdom, Nordic regions, Europe and in Asia. In the United States, the series premiered on 20 January 2018. Other country dates are published here:

https://en.wikipedia.org/wiki/Blue_Planet_II

I carried out the following actions with the set:

• Investigated the data and removed repeated retweets of an irrelevant post
• Grouped data with respect to bottle usage
• Plotted a timeline with post counts
• Researched and explained significant dates for the plot

R packages and data download

Let us start with downloading packages and data

library(dplyr)
library(magrittr)
library(ggplot2)
library(lubridate)
df = read.csv("tw_sentiment.csv")
str(df)

## 'data.frame':    30584 obs. of  5 variables:
##  $ Index        : int  1 2 3 4 5 6 7 8 9 10 ...
##  $ time         : chr  "01 Jan 2020 23:31:53" "01 Jan 2020 21:01:33" "01 Jan 2020 21:00:06" "01 Jan 2020 18:11:13" ...
##  $ content      : chr  "ayomikun lob arsenal jacket floor vs everton last year boot water bottl week didn t sit well" "rt jeremyjoseph g y club homeless saturday night yeah babi amp got reason re take g y thechas vixen m edg jenli"| __truncated__ "g y club homeless saturday night yeah babi amp got reason re take g y thechas vixen m edg jenlion donat hot wat"| __truncated__ "get budgen water bottl head spend budgen water bottl seen mani better one cheaper liter everywher t co qyfdwdhjyj" ...
##  $ type         : chr  "Reply" "ReTweet" "Tweet" "Tweet" ...
##  $ SentimentQDAP: num  0 0.125 0.129 0.118 0.133 ...

Our data span years 2015-2019. It was harvested from Twitter for posts containing the word “bottle” (in reality it shortened to “bottl”).

EDA

I discovered that in one week there were a lot (almost a thousand) retweets of the following message:

“aw rememb primari school type weather d start freez water bottl can ici water throughout day”

Apparently the initial post was done by some influencer. I removed them as irrelevant, because I intended to see how tweet counts had been changing with time.

df = df[-grep("aw rememb primari school type weather ", df$content) , ]
dim(df)

## [1] 29590     5

Feature Engineering

I divided the posts into the following categories:

• “Env concerned” for messages which appear to refer to environmental impact
• “Water bottle” for vague mentioning of water bottles
• “Hot water bottle” as something not about environment and as such been a good basic trend for Twitter discussions, in case that there was a particular increase in general post activity.

df = df %>%
  mutate(Category = 
  if_else(agrepl("\\bbottl(ed)? water", content, max.distance = 1, fixed = FALSE), 
          "Env concerned",
  if_else(agrepl(" (single?.use)|((re)?fill)|(reusabl)|(biodegrad)|(dispos\\w+)", content, 
          max.distance = 1,fixed = FALSE), "Env concerned",
  if_else(agrepl("((metal)|(aluminium)|( recycl)|(seawe pouch)|(glass( water)?))", content, 
          max.distance = 1, fixed = FALSE), "Env concerned",
  if_else(grepl("\\bhot water bottl\\b", content), "Hot water bottle",
  if_else(agrepl("( (camelba(c)?k)|(tritan)|(sigg)|(loveisland))", content, 
          max.distance = 1, fixed = FALSE), "Env concerned",
  if_else(agrepl("( (infus)|(best)|(beauti))(((\\s)|(\\S))+)? bottl", content, 
          max.distance = 1, fixed = FALSE), "Env concerned",
  if_else(agrepl("plastic(\\s\\S+)? bottl", content, max.distance = 1, fixed = FALSE),
          "Env concerned",
  if_else(agrepl("water bottl", content, max.distance = 1),"Water bottle", 
          NA_character_)))))))))
df = df[complete.cases(df), ]

I need to introduce a date type column in my data.Then I want to summarize post counts by weeks because otherwise my plot will be too busy.

df = df %>%
  mutate(Date = as.Date(substring(time, 1,  11), format = "%d %b %Y"))%>%
  mutate(week= ymd(floor_date(Date, "week")))
  
countsByWeekAndYear = df %>%
  select(week,  Category) %>%
  group_by(week , Category) %>%
  count(Category, name="tweet.n")

The documentary influence on Twitter posts

Below is a time line plot for weekly tweet counts with different post topics. I addition I marked some important dates with vertical lines:

• When first episode had been aired in Europe/Asia/Canada and US.
• 4th episode dates, discussing plastic particles in sea water
• 7th episode dates, covering human activity in the oceans and its consequences.
• Global 24-hour boycotts for plastic packaging which were introduced in wake of the series.

Other events without definite dates:

• The documentary promotion had been started more than a month in advance.
• In April 2018, in response to growing public support directly linked to Blue Planet II, the British government announced that a national ban on single-use plastic products is considered.
• In the summer of 2019 the documentary was shown on BBC America again. I did not find the corresponding schedule.

ggplot(data = countsByWeekAndYear, aes(week, tweet.n, color = Category))+ 
  geom_line(size = .6) +
  scale_x_date(date_labels = "%b %Y", date_breaks="3 months")+
  labs(x = NULL, y = "Weekly number of tweets")+
  theme(axis.text.x = element_text(angle=90, hjust = -.5, vjust=.5))+
  theme(plot.title=element_text(face="bold", color = "darkblue", size=12)) + 
  theme(axis.text.y=element_text(size=8, color = "black"), 
        axis.text.x=element_text(color = "black", size=8))+
  geom_vline(aes(xintercept=as.Date("2017-09-27", format="%Y-%m-%d"), 
                linetype="1st episode"), colour = "blue")+
  geom_vline(aes(xintercept=as.Date("2017-11-19", format="%Y-%m-%d"), 
                linetype ="4th episode"), size=.4)+
  geom_vline(aes(xintercept=as.Date("2017-12-10", format="%Y-%m-%d"), 
                linetype ="7th episode"), colour = "red", size=.4)+
  geom_vline(aes(xintercept=as.Date("2018-01-20", format="%Y-%m-%d"), 
                linetype="1st episode"), colour = "blue")+
  geom_vline(aes(xintercept=as.Date("2018-03-10", format="%Y-%m-%d"), 
                linetype ="4th episode"), size=.4)+
  geom_vline(aes(xintercept=as.Date("2018-04-07", format="%Y-%m-%d"), 
                linetype ="7th episode"), colour = "red", size=.4)+
  geom_vline(aes(xintercept=as.Date("2018-06-03", format="%Y-%m-%d"),
                linetype ="Boycotts on plastic"), size=.4, colour = "purple")+
  geom_vline(aes(xintercept=as.Date("2019-06-03", format="%Y-%m-%d"),
                linetype ="Boycotts on plastic"), size=.4,colour = "purple") +
  ggtitle("A Timeline for Weekly Bottle Tweet Counts") +
  scale_linetype_manual(name = 'Events',
                        values = c("1st episode" = 3,
                                "4th episode" = 2,
                                "7th episode" = 4,
                                "Boycotts on plastic"=5),
                        guide = guide_legend(override.aes = 
                                    list(colour = c('blue', "black", 'red', 
                                                    'purple'))))+
  theme(
  legend.title = element_text(color = "blue", size = 9))

I checked out Twitter user numbers and there had been no significant changes of Twitter users during these years. It appears that they became much more aware about plastic bottles after the series.

Another thing I was asked to do is to check if sentiments changed for the tweets before and after the documentary. It turned out that the amount of posts with the the word “bottl” before the documentary was not sufficient to show after ANOVA application as distinguishable.