Stata and the Twitter API (Part I)

​Application-Only Authentication

Why Care Block:
Um, you can scrape your own Twitter data - what's not to love about narcissistic self-reflections?  Also, happy leap day!

This is it – the culmination of all of our hard work and it’s broken up into two posts.  The first (this one) shows us how to get Twitter data, and the second shows us how to post things on Twitter using Stata and our API library.

​This post is brought to you thanks to an openness and willingness to share information.  In today’s day and age, it’s ridiculous to try and hide information or impede the transfer of knowledge to others.  I recently experienced first-hand the negative result openness can have on those who wish to obfuscate information.  I guess these certain individuals are incapable of adapting, sharing, and participating in the global community – perhaps it’s not their fault if they feel threatened, but this does not excuse what I believe to be an original sin when it comes to technology.  Sure it’s difficult to produce new things and sometimes even more difficult to scrap projects you’re emotionally attached to, but to abandon creativity and sequester change is the antithesis of this post.  I’m sure many of you have dealt with an issue similar to this in your line of work, please don’t give up the good fight.  If this sounds familiar, well this post is dedicated to you.

Just like the last Fitbit API post, all corresponding variables are color-coded for your convenience.  The first thing we need to do is set up an application with the Twitter API.  Once you enter all the necessary information, go to the “Keys and Access Tokens” tag to get your Consumer Key and Consumer Secret.  FYI, all information provided below is fake, sorry to disappoint.

​Below the Application Settings section is the “Your Access Token” section which contains the Access Token and Access Token Secret which we will use to get our basic tweet data and search for other interesting things.

​Just like our Fitbit API, we need to first get an Authorization Token by sending a cURL request that takes the Base64 encoded consumer key and client secret.  First let’s set up our variables:

local cons_key "ZAPp9dO37PnzCofN2Nm8n8kye"
local cons_sec "kfbtARFpBIdb515iaS48kYZjWhLoIdbEiAINDVX0c3W3e0fgWe"
local accs_key "1234567890-YFklDWGuvSIGLYPMnAfOZgLgLsMXjKIHqaIr1F5"
local accs_sec "LYvHWfTS6LXjtDPVXchs6dXUG52l41j4HmicYwjr8aStw"

Note: we will not be using the access key or secret until the next post

​And create our encoded consumer information string (recall encode64 is available through this post):

encode64 "`cons_key':`cons_sec'"
local client64 = r(base64)

Now we simply send our request to Twitter:

!curl -k -i -H "Content-Type: application/x-www-form-urlencoded" -H "Authorization: Basic `client64'" --request POST --data "grant_type=client_credentials" https://api.twitter.com/oauth2/token --output "token.json"

Be sure to parse the access token from our outputted token.json file (perhaps this would be a good place to enter error handling methods), I stored the value in the global macro ​$token.  Finally, send one more request that contains the query you’re looking for:

!curl -k -i -H "Authorization: Bearer $token" --request GET "https://api.twitter.com/1.1/statuses/user_timeline.json?count=100&screen_name=_belenchavez" --output "output.json"

In this case, we were able to pull the top 100 things from @_belenchavez’s timeline using the statuses API.  For more information about what you can do with the application only authentication, check our Twitter’s documentation here.  Otherwise, take a look at this frequency cloud of my commonly Tweeted/Re-Tweeted hashtags of 2016 thanks to wordclouds.com.  It even looks somewhat like the Twitter logo!