Setting Up Twitter Bots with OAuth

Posted on April 30, 2010 by masnun

Twitter has decided to kill Basic Authentication on the Twitter API from June 30. They have setup a nice website at http://www.countdowntooauth.com/ to let you all know and help you migrate your apps to use the OAuth :)

OAuth is cool. It’s safe and secure for the end user. It’s convenient for the developers as well. Basic Auth entirely depends on the username and password of the user. So, the developer can do whatever s/he wishes with the user account as long as the user doesn’t change the password. On the other hand, if the user for some reason changes his/her password, the application will no longer be able to access the account and provide the desired service. OAuth helps both parties here! When the user authenticates an app via OAuth, it provides the developer with an access token ( a key and a secret ) which is by no way related to the user’s password. It’s unique for every user and application pair. That is every user will have an unique access token only for that application. Now even if the user changes the password, the access token will remain unchanged. The developer can safely store the token and use that to access the user’s account without hassle. Similarly, if the user wants to revoke the access permitted to an application, he or she can easily do that. In that case, the access token becomes invalid and the application loses access to that account.

In the Basic Auth age, it was very easy to develop twitter bots. You just setup the username and password into a configuration file, call the REST API with the login details and you’re done! Yeah, it was quite easy. But it’s not harder now :) Don’t fret, OAuth is also very simple and easy to implement for twitter bots. While you need to go through a two phase OAuth dance to authorize other users, Twitter displays the access token of the developer directly into the dashboard! Thanks to Twitter for making things so plain for developers! With your own access token, you can authorize your apps directly without any further verification.

To get the access token, first go to : http://dev.twitter.com. Login if you’re not already logged in. Use the twitter ID you want to run as a bot. Go to http://dev.twitter.com/apps by clicking the “Your Apps” on top right corner. You will see a list of applications under the logged in twitter account. One big advantage of the basic auth was that you didn’t need to create applications. But now you need! Create an application if you don’t already have one created. In most cases you won’t have one since this is probably the first time you’re using OAuth. In that situation, please create an application. Note down the Consumer Key and Consumer Secret after visiting the application page by clicking on any of the application name. Now, on the right hand navigation bar, you’ll see “My Access Token”. Please visit that section and retrieve your Access Token and Access Token Secret. That’s all we needed. Now let’s do some coding to demonstrate the use of these keys and secrets.

We first need to get a Twitter Client library. If you’re already using one, just check to make sure that it has OAuth support. The work flow is simple. First construct the client with the consumer key and consumer secret. Then set the access token key and the access token secret. Now use the client to make Twitter API calls, in our case, to update statuses!

You can get the OAuth libraries from : http://dev.twitter.com/pages/oauth_libraries . But I recommend using Tweepy with Python and Abraham’s TwitterOAuth with PHP . They are not generic OAuth clients. They were built for Twitter and you don’t need to configure any extra parameters to make it work with Twitter.

Here’s the code samples on how to use the libraries to update status via OAuth.

PHP (Abraham’s TwitterOAuth)

<?php
$consumer_key = "";
$consumer_secret = "";
$access_key = "";
$access_secret = "";
 
require_once('twitteroauth/twitteroauth.php');
 
$connection = new TwitterOAuth ($consumer_key ,$consumer_secret , $access_key , $access_secret );
 
$connection->post('statuses/update', array('status' => "Hello Twitter OAuth!"));
?>

Python (Tweepy)

consumer_key = ""
consumer_secret = ""
access_token = ""
access_token_secret = ""
 
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
bot = tweepy.API(auth)
 
bot.update_status("Hello Twitter OAuth!")
Posted in Blog Post | Tagged , , | 33 Comments

Quick GUI Development with wxPython and wxGlade

Posted on April 28, 2010 by masnun

I don’t know C or C++ that much to develop GUI applications. I hate Java and I’m not really ready to type lines after lines to develop very simple applications. PHP and Python are the only available solutions to me. For PHP, I have PHP-GTK, WinBinder and some obsolete tools like wxPHP, php-tk and a few others. For Python, I have PyGTK, PyQT, Tk and wxPython. I have experimented with PHP-GTK, WinBinder, PyGTK, Tk and wxPython. In fact when I first tried to learn python around a year back, I first started with wxPython but ended up with nothing. Time has passed. I want to develop cross platform applications. So, I had to leave WinBinder which is Windows only. I don’t really like the idea of running Windows apps with Wine. I stuck to PHP-GTK for a while and tried PyGTK too. I liked Tk as well. With Windows installation of Python, they ship Tk. But it was not a very rich tool to develop complex GUI applications. While I liked PHP-GTK, I had to admit PHP is not really suitable for developing desktop GUI apps. I loved the flexibility and strength of Python. I experimented with both PyGTK and wxPython. I didn’t try PyQT though. I use Ubuntu (Gnome) and don’t like KDE much.

Among these two, I will definitely go for wxPython. Because applications developed with wxPython is very easy to deploy on both Windows and Linux. And it’s really very easy to finish the layout using the wxGlade tool. wxGlade generates the skeleton Python code in a beautiful way. It defines all the event handlers as well. The code is very well structured. It subclasses the wx.Frame class and defines specific methods for setting the different properties and laying the widgets out. So, if you ever need to tweak the GUI a bit, you’ll know where to find what :)

Here’s the source code of a tabbed application I built in just 5 minutes. It has a “Home” button. When you click the button, it opens a new browser and takes you to my website.

I wrote only two lines of code.

       import webbrowser
        webbrowser.open_new_tab("http://masnun.com")

Here is the full source code:

#!/usr/bin/env python
# -*- coding: utf-8 -*-
# generated by wxGlade 0.6.3 on Wed Apr 28 13:14:15 2010
 
import wx
 
# begin wxGlade: extracode
# end wxGlade
 
 
 
class MyFrame(wx.Frame):
    def __init__(self, *args, **kwds):
        # begin wxGlade: MyFrame.__init__
        kwds["style"] = wx.DEFAULT_FRAME_STYLE
        wx.Frame.__init__(self, *args, **kwds)
        self.Notebook = wx.Notebook(self, -1, style=wx.NB_BOTTOM)
        self.Notebook_Home = wx.Panel(self.Notebook, -1)
        self.panel_1 = wx.Panel(self.Notebook_Home, -1)
        self.panel_2 = wx.Panel(self.Notebook_Home, -1)
        self.panel_3 = wx.Panel(self.Notebook_Home, -1)
        self.panel_15 = wx.Panel(self.Notebook_Home, -1)
        self.panel_20 = wx.Panel(self.Notebook_Home, -1)
        self.panel_4 = wx.Panel(self.Notebook_Home, -1)
        self.panel_8 = wx.Panel(self.Notebook_Home, -1)
        self.panel_12 = wx.Panel(self.Notebook_Home, -1)
        self.panel_16 = wx.Panel(self.Notebook_Home, -1)
        self.panel_21 = wx.Panel(self.Notebook_Home, -1)
        self.panel_5 = wx.Panel(self.Notebook_Home, -1)
        self.panel_9 = wx.Panel(self.Notebook_Home, -1)
        self.button_1 = wx.Button(self.Notebook_Home, -1, "Home")
        self.panel_17 = wx.Panel(self.Notebook_Home, -1)
        self.panel_22 = wx.Panel(self.Notebook_Home, -1)
        self.panel_6 = wx.Panel(self.Notebook_Home, -1)
        self.panel_10 = wx.Panel(self.Notebook_Home, -1)
        self.panel_13 = wx.Panel(self.Notebook_Home, -1)
        self.panel_18 = wx.Panel(self.Notebook_Home, -1)
        self.panel_23 = wx.Panel(self.Notebook_Home, -1)
        self.panel_7 = wx.Panel(self.Notebook_Home, -1)
        self.panel_11 = wx.Panel(self.Notebook_Home, -1)
        self.panel_14 = wx.Panel(self.Notebook_Home, -1)
        self.panel_19 = wx.Panel(self.Notebook_Home, -1)
        self.panel_24 = wx.Panel(self.Notebook_Home, -1)
 
        self.__set_properties()
        self.__do_layout()
 
        self.Bind(wx.EVT_BUTTON, self.onClick, self.button_1)
        # end wxGlade
 
    def __set_properties(self):
        # begin wxGlade: MyFrame.__set_properties
        self.SetTitle("My App")
        self.SetSize((434, 385))
        self.SetFocus()
        # end wxGlade
 
    def __do_layout(self):
        # begin wxGlade: MyFrame.__do_layout
        MainGridSizer = wx.GridSizer(1, 1, 5, 5)
        grid_sizer_1 = wx.GridSizer(5, 5, 5, 5)
        grid_sizer_1.Add(self.panel_1, 1, wx.EXPAND, 0)
        grid_sizer_1.Add(self.panel_2, 1, wx.EXPAND, 0)
        grid_sizer_1.Add(self.panel_3, 1, wx.EXPAND, 0)
        grid_sizer_1.Add(self.panel_15, 1, wx.EXPAND, 0)
        grid_sizer_1.Add(self.panel_20, 1, wx.EXPAND, 0)
        grid_sizer_1.Add(self.panel_4, 1, wx.EXPAND, 0)
        grid_sizer_1.Add(self.panel_8, 1, wx.EXPAND, 0)
        grid_sizer_1.Add(self.panel_12, 1, wx.EXPAND, 0)
        grid_sizer_1.Add(self.panel_16, 1, wx.EXPAND, 0)
        grid_sizer_1.Add(self.panel_21, 1, wx.EXPAND, 0)
        grid_sizer_1.Add(self.panel_5, 1, wx.EXPAND, 0)
        grid_sizer_1.Add(self.panel_9, 1, wx.EXPAND, 0)
        grid_sizer_1.Add(self.button_1, 0, wx.ALIGN_CENTER_HORIZONTAL|wx.ALIGN_CENTER_VERTICAL, 0)
        grid_sizer_1.Add(self.panel_17, 1, wx.EXPAND, 0)
        grid_sizer_1.Add(self.panel_22, 1, wx.EXPAND, 0)
        grid_sizer_1.Add(self.panel_6, 1, wx.EXPAND, 0)
        grid_sizer_1.Add(self.panel_10, 1, wx.EXPAND, 0)
        grid_sizer_1.Add(self.panel_13, 1, wx.EXPAND, 0)
        grid_sizer_1.Add(self.panel_18, 1, wx.EXPAND, 0)
        grid_sizer_1.Add(self.panel_23, 1, wx.EXPAND, 0)
        grid_sizer_1.Add(self.panel_7, 1, wx.EXPAND, 0)
        grid_sizer_1.Add(self.panel_11, 1, wx.EXPAND, 0)
        grid_sizer_1.Add(self.panel_14, 1, wx.EXPAND, 0)
        grid_sizer_1.Add(self.panel_19, 1, wx.EXPAND, 0)
        grid_sizer_1.Add(self.panel_24, 1, wx.EXPAND, 0)
        self.Notebook_Home.SetSizer(grid_sizer_1)
        self.Notebook.AddPage(self.Notebook_Home, "Home")
        MainGridSizer.Add(self.Notebook, 1, wx.EXPAND, 0)
        self.SetSizer(MainGridSizer)
        self.Layout()
        # end wxGlade
 
    def onClick(self, event): # wxGlade: MyFrame.<event_handler>
        import webbrowser
        webbrowser.open_new_tab("http://masnun.com")
 
# end of class MyFrame
 
 
if __name__ == "__main__":
    app = wx.PySimpleApp(0)
    wx.InitAllImageHandlers()
    MainFrame = MyFrame(None, -1, "")
    app.SetTopWindow(MainFrame)
    MainFrame.Show()
    app.MainLoop()
Posted in Blog Post | Tagged | 3 Comments

Image Resizing With Python

Posted on April 27, 2010 by masnun

At my workplace, I’m bound to work with PHP but not at my home :D For PHP, I’ve seen cool image libraries like the GD library, ImageMagick and in-numerous classes to manipulate images. But what does Python have to offer when it comes to image manipulation ?

Python doesn’t have any native modules to work with images ( a big exception to the “batteries included” motto? ). PIL or the Python Imaging Library is the most popular one I found. It’s just awesome. It will remind you the difference between Python and other programming languages. I was seeking a solution to convert my php image resizing script into Python. I want to distribute the solution among my non techie friends. PHP is overkill for them though I could build a php-gtk app for easy use. But still, the runtime would have been heavy. And I always like Python for distributable apps.

When I looked into the PIL, I was surprised to see how easy it is to resize images. No need to dive deep into the basics of image manipulation. Just open a image and use the resize() method.

Have a look:

import Image,glob
 
files = glob.glob("*.JPG") # Find all the JPG images in the directory
width = 500 # new width
height = 420 # new height
ext = ".jpg" # target extension
 
for imageFile in files:
	print "Processing: " + imageFile
	try:
		im = Image.open(imageFile)
		im = im.resize((width, height), Image.ANTIALIAS)    # best down-sizing filter
		im.save(imageFile + ".resized" + ext)
	except Exception as exc:
		print "Error: " + str(exc)

Isn’t it easy?

To run it, you need the PIL. For Ubuntu, get it by typing:

sudo apt-get install python-imaging

For Windows and other platforms look here:
http://www.pythonware.com/products/pil/index.htm

Enjoy! :D

Posted in Blog Post | Tagged | 1 Comment

Using Moneybookers from Bangladesh

Posted on April 21, 2010 by masnun

Moneybookers is a service similar to paypal except that they have less coverage and they let Bangladeshis sign up for their service and use it efficiently. www.moneybookers.com is the URL. Those who need to receive money from abroad ( mainly for outsourcing ) can use this service for efficient money transfer. Their service is cheap and affordable.

Case Study: My Story

I use ScriptLance for outsourcing and earn a decent amount of money working at my leisure side by side of my studies at KU. I also work as a part time remote web application engineer for BRE LLC. I used to use the verified paypal account of my partner at JOTIL21. But Moneybookers have made things even simpler. Both Scriplance and BRE LLC Authority sends me payment via moneybookers and I receive the money directly into my Dutch Bangla Bank Limited account with just a few clicks. I’ve added and verified my DBBL account to my Moneybookers account and it’s all done!

Adding and verifying your Bangladeshi bank account is quite easy. Let me walk you through:

1) Register at Moneybookers (www.moneybookers.com)

2) Add funds to your Moneybookers account from other online sources. ( Withdraw fund from your employer to Moneybookers ). I withdrew funds from Scriptlance to Moneybookers.

4) Add a bank account with their SWIFT code. SWIFT is a messaging system used by most Banks for international financial messaging ( mostly transaction related ). For DBBL, it is : “DBBLBDDH “.

3) Try to withdraw money. You’ll be asked to verify your address or Bank account. You will be able to withdraw less than 15 USD before you verify. So, I withdrew $14.99 to my DBBL account.

4) It took around 10 days for the money to arrive home. I went to DBBL Khulna Branch and asked for the verification code that came along the transaction.

5) I collected it and put into my moneybookers account. To do that, I tried to Withdraw again. They again asked me to verify, I picked up the option that said something like “I already have the code”. I entered the code and my account become verified.

Now, I can withdraw around $2000+ in a 90 days period. This limit can be pushed further by verifying your home address and debit/credit card. But I’m happy with what I’ve got. They charge around $2.16 for every transaction no matter the amount is small or large. I use the DBBL ATM card to withdraw money from anywhere in BD. :)

Posted in Blog Post | 43 Comments

Using Gedit as a PHP IDE

Posted on April 10, 2010 by masnun

I’ve been using Ubuntu 9.04 as my primary OS for a while now. I love the power and flexibility of this free and open source OS. Besides PHP, I code Python mainly. I use the Netbeans IDE for PHP and the built in Gedit editor for editing all other sorts of text files and scripts. I like the simplicity and lightweight nature of this editor. Netbeans IDE is very good for web application development. Specially, I have always enjoyed the project management features for easy management of my different projects. The main reason I have been using Netbeans is the Alt+Shift+F command. The formatting option that reformats my entire page with appropriate indentation and makes it readable. Yes, it is cool! But I hate the sluggishness of Netbeans. I know, this is the fault of the slow JVM. Also, when Netbeans lose the focus and regains later, I need to right click or interact with the GUI before I can start typing. This was pissing me off. So, I started using Gedit for quick fixes and hacks. I like the font of the editor, not to mention the cool color schemes. I was wondering if there were any plugins for Gedit to format php codes like Netbeans.

No, there is no dedicated plugin for that. But Gedit has the ability to use external tools to interact with the documents. I installed the php_beautifier pear package and used it with Gedit to partially achieve what I was looking for.

1) I downloaded the PHP_Beautifier package from PHP Pear (http://pear.php.net)
2) Installed it by typing:

cd Desktop
sudo pear install PHP_Beautifier-0.1.14.tgz

The pear package installed a command line utility “php_beautifier” which lets us beautify php codes.

3) Open Gedit. Go to: Edit > Preferences. Visit the “Plugins” tab and activate the External Tools command.

4) The plugin will enable you to let external commands help you process the document. Let’s build our php beautifier for Gedit.

5) Go to: Tools > External Tools. Click “New”. Fill up the form like below:
Description: PHP Beautifier
Shortcut Key:
Commands:

php_beautifier -l "Pear()"

Input: Current Document
Output: Replace Current Document
Applicability: All documents.

You can click on the “New Tool” label to rename the tool. Let’s rename it to “PHP Beautify” and close the dialog box.

Now, we are ready to beautify our php codes from Gedit. Just type in some php codes. Go to: Tools > PHP Beautify and see the output. Please Save the document. You can’t edit the document unless you save after the processing.

PS: Please remember that, it beautifies php only. It will not be able to beautify inline html and php together. I am looking for some command line tool that can help me achieve that. And then, I will say goodbye to Netbeans and stick to Gedit :)

Posted in Blog Post | Tagged , | 5 Comments

Power of PHP: Resizing Images on Command Line

Posted on April 3, 2010 by masnun

I’m too busy to explain the script now. In short, it scans the current directory for all the JPG images and converts them into smaller image files hardly compromising the quality. I managed to convert a 211 MB album into a 5.8 MB collection with this script. Modify the $dir varible which holds the location of target directory. Make sure the target directory already exists. Also change the $percent inside the resize function to control the output quality.

Thanks to PHP for such powerful image processing. I just ran the following command on my Ubuntu terminal:

cd /var/www/photos
php img.php

I was done! Here’s the source code:

<?php
$dir = "/home/masnun/Desktop/photos/";
 
$files = glob("*.JPG");
echo count($files). " to process \n";
foreach( $files as $file ) {
        echo "Processing: {$file} ....";
        resize($file, $dir);
        echo " -- done \n";
 
        }
 
function resize($img, $dir) {
$new_loc = $dir.basename($img);
$percent = 30;
$constrain = "";
$w = "" ;
$h = "";
 
// get image size of img
$x = @getimagesize($img);
// image width
$sw = $x[0];
// image height
$sh = $x[1];
 
if ($percent > 0) {
	// calculate resized height and width if percent is defined
	$percent = $percent * 0.01;
	$w = $sw * $percent;
	$h = $sh * $percent;
} else {
	if (isset ($w) AND !isset ($h)) {
		// autocompute height if only width is set
		$h = (100 / ($sw / $w)) * .01;
		$h = @round ($sh * $h);
	} elseif (isset ($h) AND !isset ($w)) {
		// autocompute width if only height is set
		$w = (100 / ($sh / $h)) * .01;
		$w = @round ($sw * $w);
	} elseif (isset ($h) AND isset ($w) AND isset ($constrain)) {
		// get the smaller resulting image dimension if both height
		// and width are set and $constrain is also set
		$hx = (100 / ($sw / $w)) * .01;
		$hx = @round ($sh * $hx);
 
		$wx = (100 / ($sh / $h)) * .01;
		$wx = @round ($sw * $wx);
 
		if ($hx < $h) {
			$h = (100 / ($sw / $w)) * .01;
			$h = @round ($sh * $h);
		} else {
			$w = (100 / ($sh / $h)) * .01;
			$w = @round ($sw * $w);
		}
	}
}
 
$im = @ImageCreateFromJPEG ($img) or // Read JPEG Image
$im = @ImageCreateFromPNG ($img) or // or PNG Image
$im = @ImageCreateFromGIF ($img) or // or GIF Image
$im = false; // If image is not JPEG, PNG, or GIF
 
if (!$im) {
	// We get errors from PHP's ImageCreate functions...
	// So let's echo back the contents of the actual image.
	readfile ($img);
} else {
	// Create the resized image destination
	$thumb = @ImageCreateTrueColor ($w, $h);
	// Copy from image source, resize it, and paste to image destination
	@ImageCopyResampled ($thumb, $im, 0, 0, 0, 0, $w, $h, $sw, $sh);
	// Output resized image
	@ImageJPEG ($thumb, $new_loc);
}
}
?>
Posted in Blog Post | Tagged | 1 Comment

Building A Proxy Scraper with 15 lines of Python

Posted on March 18, 2010 by masnun

Yes, Python is great! It’s beautiful and so on…. I have described the power of Python many times. For now, just the codes :) Here’s a proxy scraper I built a few moments ago. It scrapes the web page at proxy-hunter.blogspot.com and lists the available open proxies.

#!/usr/bin/env python
 
from BeautifulSoup import BeautifulSoup as Soup
import re, urllib
 
url = 'http://proxy-hunter.blogspot.com/2010/03/18-03-10-speed-l1-hunter-proxies-310.html'
document = urllib.urlopen(url)
tree = Soup(document.read())
regex  = re.compile(r'^(\d{3}).(\d{1,3}).(\d{1,3}).(\d{1,3}):(\d{2,4})')
proxylist = tree.findAll(attrs = {"class":"Apple-style-span", "style": "color: black;"}, text = regex)
data = proxylist[0]
for x in data.split('\n'):
        print x

It uses the BeautifulSoup package for parsing HTML. On ubuntu install it with this command:

sudo apt-get install python-beautifulsoup

On other platforms, grab the package from its homepage. Google is there to find the URL for you ;-)

Posted in Blog Post | Tagged | Leave a comment

Using Python on Hostmonster, Umbrahosting And Other General Shared Hosting

Posted on March 11, 2010 by masnun

I’ve a hostmonster and umbrahosting shared hosting where my sites are running smoothly in PHP. I’m perfectly happy with both UmbraHosting and HostMonster. Additionally both the hosting providers have Python support. Being a Python enthusiast, I decided to check out.

On both Hostmonster and Umbrahosting, you can execute Python scripts as CGI scripts. But you must take care of these points:

– They must be placed inside the cgi-bin directory.
– Permission should be 755. (777 triggered server error for me)

I didn’t find the latest version of Python anywhere on these two hosts. Let alone Python 2.5, they have Python 2.4. :( But still I’m happy that I can run Python. The only thing that I didn’t like was the obligation to put my scripts in the cgi-bin directory. So, I couldn’t have URLs like http://masnun.com/hello.py :(

Later today afternoon, an idea clicke in my mind. Can’t I target a subdomain to the cgi-bin directory? The answer is: YES! Just use a .htaccess file to map certain urls to your cgi-bin directory.

 
<ifModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule (.*) cgi-bin/$1 [L]
</ifModule>

Putting the .htaccess in your domain’s root directory will allow you to put a python file inside your cgi-bin directory and visit it from http://example.com/example.py :D

Posted in Blog Post | Tagged | Leave a comment

Sending SMS With AloAshbei Platform (GP APIs) and PHP

Posted on February 4, 2010 by masnun

Here’s a working example of Grameen Phone’s AloAshbei Platform.

 
<?php
/*
* Demo of the SMS API
* Author: maSnun
* URL: http://masnun.com
*/
 
 
// I hosted the WSDL on my own host
$soap = new SoapClient("http://masnun.com/wsdl/wsdl.php");
 
// Set the parameters in an array
 
$a['registrationID'] = "masnun";
$a['password'] = "******";
$a['sourceMsisdn'] = '8801711960803';
$a['destinationMsisdn'] = '88017********';
$a['smsPort'] = 7424;
$a['msgType'] = 4;
$a['charge'] = 2.00 ;
$a['chargedParty'] = '8801711960803';
$a['contentArea'] = 'gpgp_psms';
$a['msgContent'] = 'Hello GP API!';
 
 
try {
    var_dump( $soap->sendSMS( array ("SendSMSRequest" => $a) ) );
} catch (Exception $e) {
 
    var_dump($e->getMessage());
 
}
 
?>

Example Response:

 
masnun@ubuntu:~$ cd random
masnun@ubuntu:~/random$ php gpsoap.php
object(stdClass)#2 (1) {
  ["SendSMSResponse"]=>
  object(stdClass)#3 (2) {
    ["status"]=>
    string(2) "OK"
    ["msgID"]=>
    string(17) "20100204183047653"
  }
}
masnun@ubuntu:~/random$
Posted in Blog Post | Tagged | 2 Comments

Hosting your Twitter Bots on Google App Engine

Posted on February 2, 2010 by masnun

UPDATE: Download the improved version of the application. I have made some code changes after the blog post got posted to reddit. The source code is now more readable and the app easily configurable.

I am amazed to see the implementation of cron jobs on Google App Engine. I was hosting my twitter bots on my paid shared hosting. My scripts were written in php and they were run via cron jobs set from my cPanel. Of course the cPanel cron UI is now quite familiar to me, to us who use it very often. But as a matter of fact, I can’t run per minute cron on my host. I should have a minimum interval of 15 minutes between two execution of the same cron job. I was amazed when I first discovered that Google App Engine lets me run cron every minute. Since they calculate my usage in different types of quotas, they hardly care about how often you schedule your cron. So, I decided to port my twitter bots to GAE and Python.

The idea was pretty simple. I already had a Yahoo! Pipe setup which mashes up multiple RSS feeds into one. My app will be collecting data from the pipe in JSON format. Then it’ll check the last tweeted entry and tweet the newer entries. Pretty straightforward algorithm. I registered a new application for my bots and started developing it on my local machine.

I integrated the u.nu url shortening service with their API quite easily. Though Python 2.6.2 has a built in module for JSON handling, Python 2.5 doesn’t have it. So, I had to install simplejson for JSON decoding. Later, I moved on to finding a good twitter client for my app. I first tried python-twitter. I have previously used it and it worked fine. But the application was generating some error messages when I integrated into GAE. I was puzzled and tried to find if there was something wrong in my coding. After about an hour, I concluded that my code was okay! There’s something wrong with the client. A bit of googling revealed that “python-twitter” doesn’t work on GAE without any hacks :’(. I had to find an alternative and I chose tweepy for that. It was working fine with the app.

Finally after 3 hours, the app was ready and it’s working! GAE doesn’t permit long execution period for a process, so I had to decrease the amount of tweets per request. But I covered that up by setting a more frequent cron job :) . I would try the new taskqueue API soon to see if that has any help for my app.

My twitter bots have proved handy in the past and it feels great that I can host them on a scalable and efficient platform like Google App Engine — without paying a single dollar per month! :D

Download Source : http://masnun.googlecode.com/files/Twitter_BOT_GAE.zip :)

Posted in Blog Post | Tagged , , , | 4 Comments