William Matsuoka
  • Introduction
  • Topics
  • Stata
  • Teaching
    • ECON 641L
    • ECON 640L >
      • Econ-Data
  • Blog
  • About

W=M/Stata

Welcome

Stata and the Twitter API (Part I)

2/29/2016

6 Comments

 

​Application-Only Authentication

Why Care Block:
Um, you can scrape your own Twitter data - what's not to love about narcissistic self-reflections?  Also, happy leap day!
This is it – the culmination of all of our hard work and it’s broken up into two posts.  The first (this one) shows us how to get Twitter data, and the second shows us how to post things on Twitter using Stata and our API library.

​This post is brought to you thanks to an openness and willingness to share information.  In today’s day and age, it’s ridiculous to try and hide information or impede the transfer of knowledge to others.  I recently experienced first-hand the negative result openness can have on those who wish to obfuscate information.  I guess these certain individuals are incapable of adapting, sharing, and participating in the global community – perhaps it’s not their fault if they feel threatened, but this does not excuse what I believe to be an original sin when it comes to technology.  Sure it’s difficult to produce new things and sometimes even more difficult to scrap projects you’re emotionally attached to, but to abandon creativity and sequester change is the antithesis of this post.  I’m sure many of you have dealt with an issue similar to this in your line of work, please don’t give up the good fight.  If this sounds familiar, well this post is dedicated to you.

Just like the last Fitbit API post, all corresponding variables are color-coded for your convenience.  The first thing we need to do is set up an application with the Twitter API.  Once you enter all the necessary information, go to the “Keys and Access Tokens” tag to get your Consumer Key and Consumer Secret.  FYI, all information provided below is fake, sorry to disappoint.
Picture
​Below the Application Settings section is the “Your Access Token” section which contains the Access Token and Access Token Secret which we will use to get our basic tweet data and search for other interesting things.
Picture
​Just like our Fitbit API, we need to first get an Authorization Token by sending a cURL request that takes the Base64 encoded consumer key and client secret.  First let’s set up our variables:
local cons_key "ZAPp9dO37PnzCofN2Nm8n8kye"
local cons_sec "kfbtARFpBIdb515iaS48kYZjWhLoIdbEiAINDVX0c3W3e0fgWe"
local accs_key "1234567890-YFklDWGuvSIGLYPMnAfOZgLgLsMXjKIHqaIr1F5"
local accs_sec "LYvHWfTS6LXjtDPVXchs6dXUG52l41j4HmicYwjr8aStw"

Note: we will not be using the access key or secret until the next post

​And create our encoded consumer information string (recall encode64 is available through this post):
encode64 "`cons_key':`cons_sec'"
local client64 = r(base64)

Now we simply send our request to Twitter:
!curl -k -i -H "Content-Type: application/x-www-form-urlencoded" -H "Authorization: Basic `client64'" --request POST --data "grant_type=client_credentials" https://api.twitter.com/oauth2/token --output "token.json"

Be sure to parse the access token from our outputted token.json file (perhaps this would be a good place to enter error handling methods), I stored the value in the global macro ​$token.  Finally, send one more request that contains the query you’re looking for:
!curl -k -i -H "Authorization: Bearer $token" --request GET "https://api.twitter.com/1.1/statuses/user_timeline.json?count=100&screen_name=_belenchavez" --output "output.json"

In this case, we were able to pull the top 100 things from @_belenchavez’s timeline using the statuses API.  For more information about what you can do with the application only authentication, check our Twitter’s documentation here.  Otherwise, take a look at this frequency cloud of my commonly Tweeted/Re-Tweeted hashtags of 2016 thanks to wordclouds.com.  It even looks somewhat like the Twitter logo!
Picture
6 Comments
twitter verification services link
4/2/2016 11:26:40 pm

The verification process does have a sensible function. Twitter brims with bogus or parody accounts. So when customers are sifting via a summary of likely usernames, it can help to get alerts to aid find the precise man or woman they want to abide by.

Reply
Buy Twitter Followers Cheap link
7/31/2016 08:40:22 am

This is a very interesting post here on what you did with the Twitter API. I had no idea about this, thank you.

Reply
twitter followers cheap link
12/3/2016 10:07:20 am

This is a fantastic website and I can not recommend you guys enough.

Reply
Stata User link
12/26/2016 11:10:17 am

Thank you William for this very interesting post! I have been collecting data from twitter and youtube using python, but I would rather do it directly from Stata.
So, I tried your script (with my authentification keys), but I don't get any output. Could you please assist me?

Reply
psd to wordpress link
1/25/2018 02:27:52 am

I really enjoyed reading this post, big fan. Keep up the good work andplease tell me when can you publish more articles or where can I read more on the subject?

Reply
Water Filtration Des Moines link
10/9/2022 01:47:25 pm

Very thoughtful blogg

Reply



Leave a Reply.

    Author

    Will Matsuoka is the creator of W=M/Stata - he likes creativity and simplicity, taking pictures of food, competition, and anything that can be analyzed.

    For more information about this site, check out the teaser above!

    Archives

    July 2016
    June 2016
    March 2016
    February 2016
    January 2016
    December 2015
    November 2015
    October 2015
    September 2015

    Categories

    All
    3ds Max
    Adobe
    API
    Base16
    Base2
    Base64
    Binary
    Bitmap
    Color
    Crawldir
    Email
    Encryption
    Excel
    Exif
    File
    Fileread
    Filewrite
    Fitbit
    Formulas
    Gcmap
    GIMP
    GIS
    Google
    History
    JavaScript
    Location
    Maps
    Mata
    Music
    NFL
    Numtobase26
    Parsing
    Pictures
    Plugins
    Privacy
    Putexcel
    Summary
    Taylor Swift
    Twitter
    Vbscript
    Work
    Xlsx
    XML

    RSS Feed

Proudly powered by Weebly
  • Introduction
  • Topics
  • Stata
  • Teaching
    • ECON 641L
    • ECON 640L >
      • Econ-Data
  • Blog
  • About