iPhone: getting IMAP push notifications with Python and Prowl

Posted on by saghul 1 Comment
I’m a happy iPhone user. I was lucky to get the first iPhone soon after it was available and I just bought a new iPhone4. This is my first time having unlimited data plan on a mobile phone 🙂 so first thing I did was to configure my email accounts, both personal and work.
It felt weird to configure a GMail account as Microsoft Exchange in order to sync contacts and get push notifications, but that did the trick. For my work email account, I didn’t have push notifications, and polling is not any cool. I know very little about email itself, but I found that in order to have push email you need an IMAP IDLE compatible server and client.
As you may see below, the server does support IMAP IDLE:
[gist]https://gist.github.com/579679[/gist]
but it looks like iPhone Mail app doesn’t 🙁
I started looking around and finally found the solution: use an application called Prowl (2.39€) which will push whatever notification it gets through a REST API. The idea is basically to create a console IMAP IDLE client and use the Prowl API to send a new notification. Sounds like fun!
There is a Perl implementation here (http://github.com/mschmitt/GhettoPush/) but I don’t like Perl very much, so I had to do it in Python, of course 😉
I also found some other Python implementations which I didn’t like for one reason or another so I coded a new one using imaplib2 by Piers Lauder (http://www.cs.usyd.edu.au/~piers/python/imaplib.html) and prowlpy, which I forked on GitHub and did a couple of really silly modifications (http://github.com/saghul/prowlpy).
It works on the background and it can connect to an arbitrary name of accounts (configured with an ini-style file) and send push notifications on behalf of them. It also logs to syslog, if running in the background.
This Prowl thing is just awesome, this is an example of what you can do with it, but anyone can write a really simple script to get instant notifications on his/her iPhone. Just give it a try!
The libraries:
References:

 

Happy pushing!

Python: list vs. set

Posted on by saghul 1 Comment

In Python we have several types of objects for storing values in a
array-like way: lists, tuples, dictionaries (they are really more like
hash tables) and sets. Lists and sets might look alike but they are
different. Lets do a quick test:

[gist]https://gist.github.com/569053[/gist]

In the above snippet, we can see that iterating over all items in a

list takes just a little longer tan a a set: 8.99us vs 6.87us we are
talking micro-seconds here!

Now, if we look at how long it takes to verify is the list or set
contains an object, we can see the difference: 1.99us vs 140ns. Sets
are an order of magnitude faster!

Why is that? We can think that a python list is like a C linked list.
It’s time complexity when checking for a value is O(n). Sets, on the
other hand, are implemented as hash tables, so the key is hashed and
instantaneously found (or not) with a time complexity of O(1).

For the curious, this is the list object declaration in Include/listobject.h

[gist]https://gist.github.com/569078[/gist]

and this is the set object declaration, in Include/setobject.h

[gist]https://gist.github.com/569085[/gist]

More on python and hash tables here:

http://sites.google.com/site/usfcomputerscience/hash-tables-imp

:wq

Choosing a JSON library

Posted on by saghul Leave a comment

When I download a library for using whatever API I usually see that people tend to use different JSON libraries, and sometimes I just don’t have it installed, but I know I got another one which could do the job just fine.

A while ago I ran across this while I was having a look at the Tropo WebAPI library (http://github.com/tropo/tropo-webapi-python) and ended up adding the following code:

try:
    import cjson as jsonlib
    jsonlib.dumps = jsonlib.encode
    jsonlib.loads = jsonlib.decode
except ImportError:
    try:
        from django.utils import simplejson as jsonlib
    except ImportError:
        try:
            import simplejson as jsonlib
        except ImportError:
            import json as jsonlib

Most used JSON libraries are ordered regarding speed. You might want to read a nice comparison between libraries here:
http://blog.hill-street.net/?p=7

:wq

Creating a “smart” Twitter bot

Posted on by saghul Leave a comment

A while ago I was bored at home and decided to play a bit with the Twitter API. As a result I created a simple Twitter Bot which retewitted everything with the *#asterisk* hashtag (http://twitter.com/asteriskbot).>

It’s functionality was very simple andI didn’t even touch the (crappy) code for months, but Twitter decided to drop the basic authentication support in favor of OAuth so it was the right time for updating the bot.

Since the bot was created lots of other bots have, and it feels annoying to get the same tweet over and over again, so I though I’d try to make the bot a bit *smarter*. The main goal was to detect if a tweet was already tweeted (or RTd) not to repeat things. AFAIK there are 2 types of retweets:

  • “some text RT @user1 RT @user2 some super cool tweet here!”
  • “some super cool tweet! (via @user)”

The current bot detects both formats by using regular expressions and extracts the tweet itself. However, I felt this could not be enough, because we could get the retweet *before* the real tweet so it’s be great to save the entire tweet and then look for it.

As the bot was using a SQLite database I decided to use its *Full Text Search* (fts3) capability, inspired by a colleague. With FTS searching for an entire string is amazingly faster than doing a regular SELECT query. Here is an example taken from the SQLite website http://www.sqlite.org/fts3.html

CREATE VIRTUAL TABLE enrondata1 USING fts3(content TEXT); /* FTS3 table */
CREATE TABLE enrondata2(content TEXT); /* Ordinary table */

SELECT count(*) FROM enrondata1 WHERE content MATCH 'linux'; /* 0.03 seconds */
SELECT count(*) FROM enrondata2 WHERE content LIKE '%linux%'; /* 22.5 seconds */

The (silly) code for this bot is available on GitHub: http://github.com/saghul/twitterbot
Happy tweeting!