When Twitter meets Python!

Python_Twitter

Reporters love Twitter and geeks love coding. Today, I’m merging the best of both worlds! On the menu: Python scripts to use Twitter to its full potential!

When my friend @TerraCiolfe showed me @WeAreTheDeads project, I said to myself that I really need to learn how to control Twitter through Python. @WeAreTheDeads is a Twitter account publishing the name of a fallen soldiers at the 11th minute of each hour.

Of course, nobody is working behind the screen. A program chooses the soldier in a database and publishes his name, hour after hour. With 119,000 names to publish, the script will run until 2023, according to the author of this great idea, the reporter @GlenMcGregor from the Ottawa Citizen.

With a little bit of research (my sources are at the end of the article), I learnt how to work with Twitter from a Python script. Actually, we can do way more than automatically publish tweets! It’s also possible to extract a lot of data about users and their tweets. For example, you can research specific tweets in a specific location. I created a nice animated map at the end. You’ll see!

PS: This article is an interesting read with or without knowledge of programming. However, if you want to learn how to redo by yourself what I am showing below, begin with my step by step tutorial to learn Python.

#I Tweepy library

Many Python libraries exist to work with Twitter. The company lists them on its website. After much hesitation, I decided to use Tweepy!

It’s not a native library. You need to install it before using it.

If you have pip, it’s easy. Type the following line in your Terminal or Command Prompt.

If you don’t have pip, install it! It’s great to install quickly external librairies and modules. Otherwise, go to the Tweepy GitHub. You’ll find the necessary informations.

#II Twitter Apps

You also need to create your own Twitter App. To do so, go to https://apps.twitter.com/. The app will do the link between your script and Twitter.

This step will allow you to get your consumer key, your consumer secret, your access token and your access token secret. Without them, you won’t be able to identify yourself and your script won’t run.

Token

Don’t forget to give your app the permission to read but also to write.

#III Send a tweet

You are now ready to send a tweet! First, we import Tweepy on line 1, we identify ourselves on ligne 3,4 and 6. Then we publish a tweet with the following lines!

Here is the result:

tweet example

Of course, this script is not doing much by itself. To have something really useful, it would need to be integrated into a bigger script that would react to something (a press release from the police or the fire department, a tweet from another user) or that would be looping many times over.

#IV Users’ information

Twitter keeps all kind of data about yourself. With a script, you’re able to extract a lot of it, for any user!

Here is a short script that will get the information about the Canadian Prime Minister @pmharper.

Here is the result:

Exemple harper

There is a lot of information contained in the tweets as well! Here is another script that will extract the data for the last 200 tweets from the Prime Minister. I added a one second delay, so you can see the script running.

The script in action now! I stopped recording after one minute. I’m sure you understood the concept.

Just for fun, I created a word cloud with the people mentioned in the Premier’s last 200 tweets. His wife comes back most often. The French President, recently in Canada, was mentioned on several occasions as well.

Interesting, isn’t it?

Nuage de mots

#V Search for a specific area

You have probably seen these beautiful maps with geotagged tweets, sometimes updated in real time. They are possible because Twitter keeps the tweet’s latitude and longitude, with the authorization of the users (the Prime Minister, for example, disabled this option).

Nothing is better than a good example. The script below will extract the coordinates of the last 100 tweets published in Canada, because Canada’s id for Twitter is 3376992a082d67c7 (for more information about this, check the Twitter official documentation). To respect the privacy of the users, I won’t extract anything else here.

I modified the script to extract around 1500 tweets, then I used the coordinates and the time of publication in CartoDB (which I talked about in a previous post). Here we go! A beautiful map with geolocalised tweets, appearing depending on the publication time! Zoom to see how precise it is!

I leave you with this. Just imagine all the things we can do with a few code lines! Follow me on Facebook, Twitter or LinkedIn to know when I will be publishing new articles!

In a following post, I’ll show you how to extract data from Twitter’s stream, in real time!

Here are the very useful links I read to learn how to control Twitter with Python scripts (on top of the great forum of http://stackoverflow.com/):

– https://dev.twitter.com/overview/api

– http://tweepy.readthedocs.org/en/v3.1.0/getting_started.html

– https://github.com/tweepy

– http://www.pythoncentral.io/introduction-to-tweepy-twitter-for-python/

8 thoughts on “When Twitter meets Python!

  1. hossein

    hi I am hosein akbarian from iran I have a practise like this.
    can you help me please?
    I can’t run
    #IV Users’ information
    I can’t set username in this sample code

    Reply
  2. Sourabh

    I want to store all this data into a database how can I do that.I am new to python. I have edit and run this code successfully. Please help out.

    Thanks

    Sourabh Choudhary

    Reply
  3. Sujith M S

    hi,
    i have created a tweet crawler using tweepy.but i cant able to exit from on_data(self, data): when a certain condition become true.Its also not posibble to write tweet data into csv in on_data.whats the solution ??? plz help

    with regards
    Sujith M S

    Reply
  4. raghuram

    HI,

    I’ve created a small analyzing application with tweepy, I wonder whether I can host it on internet(not for commercial purpose)

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *