Tweet Scraping – Guidelines

Popularity

Our objective is to build a public sentiment indicator, thus we scrape tweets only based on search terms and popularity and not based on any specific users

Region

Tweets scraped are limited to USA since our objective is to determine impact on US Economy

Retweets

Only original tweets are considered, and retweets are ignored.

Tweet Details

•For each tweet, we save the following information – id, text, time, retweet count and likes count

Econo-meter Score Algorithm

Step 1

Separate the positive (+ve) sentiment tweets and negative (-ve) sentiment tweets

Step 2

Compute the sum of the absolute values of the sentiment scores of positive (+ve) and negative (–ve) tweets

Step 3

Compute the percentage of scores of tweets which are positive (+ve) and negative (–ve). See formula below.

Econo-meter Formula

Weight Adjusted Sentiment Score

Retweets Weights

Every retweet is effectively a second person expressing the same opinion as the initial tweet, thus we would give every retweet the same sentiment score as the original tweet.

Favourites (Likes) Weights

Every like is not as convincing as expressing the same opinion as the initial tweet but limited to agreeing to the contents of the tweet. Thus we would give every like a percentage of the sentiment score of the original tweet. Currently taken at 75%.