animate.css

Donnerstag, 14. März 2019

Analyzing Twitter topics, users & sentiment

Following university project got me into working with large amounts of Twitter data. The data had to be collected, processed, and organized in order to visualize it. The focus is on Twitter users and what stories the data can tell us. Therefore, I focused on how people describe themselves online and what we can find out about groups of people as well as what sentiment can be found in tweets about certain topics.

Twitter profiles give us numerous insights into what kind of people tweet and what topics they're interested in. When you visit a profile you've never seen before you can quite quickly figure out what the person's views are, simply by reading their bio and scrolling through their feed.

Based on the words that are used in a Twitter bio we are able to draw conclusions of a person's political leaning, whether or not it's a family person, or if the person defines him or herself through their job/academic achievements and more...

In addition to the bios we can also look at the topics somebody tweeted about. A popular way to signalize what topic you tweet about is the use of hashtags.

On a broader scale and with more available data these two dimensions allows us to find out more about a larger quantity of Twitter users. Scroll down to the "Bios & Hashtags" visualization to find out more.

Who tweets: Bios & Hashtags



The 2018 United Nations Climate Change Conference


To illustrate this, if we want to find out what people do as an occupation who tweeted about the 2018 United Nations Climate Change Conference using the hashtag #COP24, we can simply select the #COP24 circle in the hashtags section. As a result, in the bio word cloud we can now see the top 5 occupations: Directors, writers, founders, authors and journalists. The larger the letters, the more people used it. If you hover over the bio words with your mouse, you'll see how many exactly included it in their bio.

scroll me

Furthermore, people include words like sustainability, change, world, environment or love in their bios. Another observation we can make is that there are fewer bio words of the categories "Religion" or "Family" compared to other categories such as "Political & Social", "Occupation", "Science & Technology" or "Environment". You can hover over the words to see the category they belong to.


Pro-life movement


Let's have a look at another hashtag. A widely used hashtag by people who are against the practice of abortion is #ProLife. Looking at the results it's evident that the hashtag is most frequently used by users who consider themselves as patriotic or conservative. Many of them are Donald Trump supporters which can be concluded by bio mentions such as Trump, buildthewall, president, MAGA = "Make America Great Again" or KAG = "Keep America Great"1. Bio words of the category "Religion" and "Family" are the second most prominent mentions. "Science & Technology", "Environment" and "Occupation" are used far less. Looking at the "Occupation" category in detail, we can see retired and vets (War veterans) are topping the list.


QAnon: The pro-Trump conspiracy theory


It's not only possible to select hashtags and find their corresponding bio words, but we can also select bio words and find out what the most prominent used hashtags are. To demonstrate this I've come across an alarmingly large number of users who spread a pro-Trump conspiracy theory and often mention it in their bios as well, namely QAnon and WWG1WGA
ℹ To select multiple elements hold CTRL & select.

The BBC reports: The baseless core of the QAnon story is that the Mueller investigation into alleged collusion between the Trump campaign and Russia is actually an investigation into global elites, and that the president is masterminding a secret plan to arrest top politicians and Hollywood stars for corruption and child abuse.2

First of all, on the map we can see that these hashtags are mostly shared within the US. The hashtags are also mostly consistent with the Conservatives' point of view, such as #2A, #2ndAmendment (Supporters of the 2nd Amendment of the United States Constitution, who are in favor of gun possession laws) or #NRA, pro-Trump hashtags (#MAGA, #BuildTheWall, #POTUS), anti-abortion (#DefundPlannedParenthood, #DefundPP). However, also some more extreme opinions can be found, like #LiberalsSuck or #LiberalismIsAMentalDisorder.
Note: This doesn't mean all Conservatives share this view, but rather the extreme alt-right / far-right as mentioned in the BBC article.


Audiences and topics based on countries


Of course, what people in the US tweet about differs from topics that are discussed in other parts of the world. Let's look at some countries individually and see what we can learn about their audiences.

This preselection only contains the most active countries of this dataset. You can use the map selection tools to select other regions.

Ireland and Abortion


If we select Ireland there are two hashtags that stand out, namely #Abortion and #Baby. Even though the Republic of Ireland is almost 15 times smaller than the United Kingdom in terms of population size, #Abortion (8.204 mentions) in Ireland still outweighs the UK hashtag in tweet volume (5.525 mentions).

Why is that so?

The answer is simple. The right to carry out an abortion became legal in Ireland on January 1st 20193. The Twitter data was collected from December 13th 2018 to January 13th 2019 and there were more people who tweeted about it immediately before and after the new law was passed. As we can see in the bio word cloud there are also many religious Twitter users (god, catholic), many of whom might oppose the idea of abortions as a legal right.


India's skilled professionals and Brussel's active role in shaping the European Union


India's Twitter users are quite active when it comes to tweeting about topics related to climate change. They rank 5th among all nations active on Twitter for the collected hashtags. Not only that, what's also striking is that there are relatively many professionals, such as founders, engineers, writers, developers, journalists, bloggers, editors, and the list goes on.

Another interesting (but not necessarily surprising) finding is that an overwhelmingly large number of Twitter users in Belgium use the word European in their bio. Compared to the other European countries in the preselection they have the highest ratio of people who mention it (1.034 mentions). This of course is closely linked to Brussel's role in the European Union.


Explore


Select bio descriptions in the word cloud, hashtags or select areas on the map to find out more about Twitter users tweeting about topics such as climate change, gun control and abortion.

Need some ideas?

You could ask: What hashtags do teachers, students or PhDs use?

Find out what hashtags religious users vs. pro-science users tweet about: Religion vs. Science

Your turn: Reset Viz
ℹ To find hashtags you can also use the search function below the hashtags section.




Sentiment



Now that we know what hashtags and topics users tweet about we can shift our focus to the sentiment thereof and find out what the discourse looks like. With the use of natural language processing techniques it's possible to assess how positive or negative people talk about a certain topic. This is often used by marketers to analyze how people on online platforms view a product or a brand. With Twitter's easily accessible API it's possible to collect large amounts of short texts on certain topics. For the sentiment analysis the AFINN Lexicon developed by Finn Årup Nielsen was used.

In the following visualization you can find out the proportion of positive and negative retweets for specifc keywords or hashtags.

Fridays For Future: The climate strike movement


The #FridaysForFuture movement led by @GretaThunberg, which was soon joined by students from all around the globe, can be characterized as a rather positive debate. The most-retweeted tweets are positive. If you look on the right side of the chart, you can see the percentage of positive, negative and neutral sentiment of the selected retweets.

scroll me

ℹ Positive retweets appear as blue and negative as red circles. The more positive or negative a tweet, the higher or lower it appears on the scale. Click on a circle to view the original tweet or linked source. Note: Some links might not work anymore because the tweet or web resource was deleted or the Twitter account was removed.

Another widely used hashtag among "Fridays For Future" advocats is #ClimateStrike which emphasizes on the school strike aspect of the movement. Although most retweets are classified as neutral, there is a larger proportion of negative sentiment conveyed in these messsages. This might be also due to the fact, that a strike embodies a more provocative and confronting form of protest, which is stressed here.


Climate change


The majority of retweets are concerned with #ClimateChange. As we can see, the sentiments are quite balanced in comparison to the previous hashtags.

Two other hashtags discussing climate change are #ClimateAction and #GreenNewDeal. The latter (U.S. political) debate contains mostly polarized comments and less neutral views.


The abortion debate


If we explore the abortion debate, we can see that the debate is mostly balanced between the three sentiment categories. However, if we shift our focus to the keyword "babies" things look different. This selection features pro-abortionists in particular, who have quite strong negative emotions towards the practice. They use graphic language and their statements often contain terms such as "killing", "slaughtering" or "genocide". It seems like in these pro-abortionist Twitter networks a lot of polarization takes place.


Gun control


Users talking about gun control use additional hashtags such as #2A (gun control opponents) or #GunControlNow (pro gun control laws) besides #GunControl. Words like "violence" for example mostly refer to tweets about gun violence.


2019 - A year of positivity?


Since the data was collected at the turn of the year we can also find out how people talked about 2019. Check out "new year" to see that 2019 started with many positive voices.


Explore


Your turn: Reset Viz
ℹ To find a certain hashtag or keyword, use the search function in the top right corner.



About this project

Collected Hashtags: #ClimateChange, #Abortion, #GunControl, #2ndAmendment, #2A, #ProGun, #ProLife, #DefundPlannedParenthood, #DefundPP, #ProChoice

Collection Period: December 13th 2018 - January 13th 2019 (32 days)

Visualization software: Tableau

This Twitter visualization was part of a project at the University of Tuebingen supervised by Dr. Erwin Feyersinger