Hal is an SMS-based information engine I designed and built at the AstonHack hackathon earlier this month. Since then, I've added a few extra features, done some major code-cleanup, and made the source code available at my Github. This post will go through the main features of Hal, and some of the design decisions I found most interesting.
First of all, a brief description of what Hal does. Simply put, it's a phone number you can text with questions or requests for information, and which will do its best to give useful and accurate responses. For instance, a message of "Tell me about Alan Turing" would return a short description of his life's work and notable historical facts; and a message of "translate 'eggs' to Japanese" would return "Japanese: 卵".
In order to actually send and receive text messages, I used the Twilio service. One of its features is to provide a UK SMS number which, on receiving a text message, will send an HTTP request (in this case, of type POST) to a given URL. Based on the response to this HTTP request, Twilio can perform many actions, the most relevant of which being to send an SMS reply to the original sender. The response that the remote server gives must be in TwiML, a Twilio-specified markup language similar to XML which specifies the actions Twilio should take next.
In order to use this service, I wrote a Python Flask server which provided a single API endpoint: this endpoint accepted POST requests from Twilio, generated a response, formatted this response in TwiML and responded via HTTP with this TwiML document. Twilio's servers handle the actual message-receiving and -sending, so I only have to parse the incoming message, then generate and format a response.
The code for the web server is in run.py and hal/views.py. There's barely anything there - Flask makes setting up this sort of thing incredibly easy, so (as the docstring in run.py says), all the heavy lifting is done in the response generation.
The single received view in views.py imports the gen_response function from generator.py. The purpose of this function, as the name suggests, is to generate and return a reply to a given SMS. Really though, this function acts solely as a wrapper for parse, the other function in generator.py, returning a friendlier error message than the None returned by parse if we can't figure any kind of intelligent response out.
In parse, we've reached the first significant design decision. The parse function relies on a list of pairs - handlers - being available. In each pair, the first object is a callable, which tests whether a message corresponds to a certain "type" of message (i.e. does the message want information about a person? Does it want a currency conversion?). If this first callable returns a truthy value (along with any useful information it's picked up in the process of determining the message's type), the second callable will be called, and its return value propagated upwards as the response to our incoming text message.
The handlers name is imported from handlers.py. Essentially, handlers.py does little more than define lots of these (test, on_true) pairs, following the is_<typename> and <typename> naming convention. At the end of this file, the handlers name is created, using a list literal. I'm aware that with some introspection, this list could be generated dynamically, but there doesn't seem to be a particular need for it right now - when features are added, it doesn't take a great deal more effort to add their names to the handlers list.
To implement the testing functions, a few methods are used, which mainly rely on ugly-ish regular expression parsing. For a limited subset of the English language like this, it's not an overly fragile solution, although some normalisation (e.g. str.lower().strip()) is needed to stop the expressions becoming too monstrous. For purely prefix-based detection, I've included a relatively simple basic_match function which matches against a list of potential prefixes. An example of this could be testing whether a message wants information about a person, the prefixes for which might include "Tell me about", "Who was/is" and "Information on".
Each actual handler function implements their information-gathering in a different way. There's a fair few API calls using the excellent requests library, some web scraping, and even (gasp) some regex-based HTML parsing. For a very limited, known subset of HTML, on a single small webpage. Essentially, the handlers are the "bolt-ons" of this project - they're cheap to add, and don't require any changes to the project structure.
All that remains is to actually format the response, for which Twilio provides a Python library. The (incredibly simple) code for this is in utils.py, and takes up all of 5 lines, including a docstring. Also in utils.py is get_last_message, which pops the message just POST'ed to us out of Flask's request object.
And that's all there is to it! Python, Flask and Twilio really did all the hard work; all I had to do was write some regular expressions, and google around for some nice APIs to use. Hal's still in development (I want to add a persistent User context, which will significantly increase the work my server does), so watch this space for more.