Jiva Technology

Agile Case Study: Developing Infocow for Futurelab

This week at Jiva we finished our ninth “sprint” of work on a project for Futurelab and we’re feeling proud and a little sentimental at seeing it released into the wild and taking our hands off the controls, for a while at least. The project known as Greater Expectations is funded by BECTA and is the brainchild of Bristol’s hi-tech education think-tank Futurelab. Their researchers and strategists conceived of what has, through our Agile development process come to be Infocow, a website listing online resources with the potential to empower young people.


Our development cycle in this project was one two-week iteration per month in order. This was designed to accommodate Futurelab’s rigorous programme of focus groups, user testing and consultation. Through this process, groups of young people from interested schools became ‘partners’ in Greater Expectations and shaped the features, design and identity of what developed into Infocow.

The project is now rolling out gently and there are a several more development iterations planned for the coming months. Looking back at the technical challenges, the interesting work fell into three broad areas:

Social networking tools

We knew from early research that social networks were likely to be key to Infocow’s success by viral marketing. We took the decision early to try integrating users’ Facebook networks into the site as a way to promote content demonstrated to be of interest to an individual’ peer group. Initially we worked with the Facebook API directly via the Facebooker gem (software library for Ruby, our language of choice). User feedback told us that what people really wanted was the ability to log in with their Facebook or WindowsLive credentials, so we backed out the Facebooker code and implemented the generalised authentication and social network interface from JanRain’s RPX. This allows users to login to Infocow using their existing social network account of choice and gives us a single API for dealing with these third parties. At present we can do everything we need with the RPX API, but we also have the option to fall back to lower level tools such as Travis Reeder’s mini_fb or the 3rd parties’ raw APIs if we need to get really creative.

Search and recommendations

Quite a number of mockups and wireframes were tossed around in the early days as we tried various interfaces for navigating the hundreds of websites listed. Following our Agile principle of trying out the simplest thing which could possibly work we went ahead and coded up a search interface based on Digg’s clean filtering system. This turned out to be super-easy using Mat Brown’s excellent Sunspot library which provides a Ruby wrapper for Solr the web component in Apache’s Lucene project, the Big Daddy of text search. As it’s early days and we have insufficient usage data to build a fully fledged recommendations systems, we’ve implemented a pretty naive recommender but look forward to plugging in some more sophisticated similarity and clustering algorithms.

Admin tools and reports

We created a whole suite of tools for administrators and ‘partners’ (youngsters recruited to moderate the site and curate content) to monitor and manage the websites listed and keep tabs on their popularity and comment-worthiness. A fair amount of work went into ensuring that both featured and user-generated content could be flagged as inappropriate by users or automatically picked up as being spam or obscene. Akismet, of course, is the defacto mechanism for testing content for spam, but we were surprised how short the standard lists of obscenities on the web seem to be, so we had to do a bit of collation of f*ck-filter lists to get to a point where some of our choice expletives were caught. If you have a particularly good list of swear-words, then please do email it in. We’ll put it to good use.

Whilst Jiva has to date been primarily a “pure-play” startup building applications such as Beanbag and Tutorhub, we have found this client project immensely rewarding. It has been a pleasure to work with the specialists at Futurelab who have a deep understanding of their field and to offer up our own imaginative technical solutions to fit the User Stories they provide. Hopefully, there will be plenty more to come.


Should probably mention that h…

Should probably mention that http://tutorhub.com went into closed beta yesterday. Ping me if you want an invite.

XMPP, Jabber, BOSH and all that

Until now, we’ve not given much in the way of technical updates from Planet Jiva, but with the impending release of Tutorhub, I thought it would be worth sharing some of our technical trials and tribulations. From the word go, our technical vision required a need for real time ‘conversation’, blending IM style features with those found in run of the mill web applications. This is known as making life hard for yourself.

In contrast to the fundamentally asymmetric, request-response model that underlies almost all web based communication (even Web 2.0 sites that provide dynamic updates of information), instant messaging applications require any connected party to be able to send or receive a message at any time.  Not all messages receive, or require, a response. The established open standard, used by Jabber, Google Chat and Facebook Chat, is XMPP. The recent release of the draft standard for the BOSH protocol, together with the emergence of a couple of open source JavaScript libraries which support XMPP over BOSH, allowed us to embed XMPP based communication channels within Tutorhub, our new web application for the online tutoring market. It’s been a complex undertaking. It necessitates moving from a two party request-response communication pattern to an architecture where the web browser must communicate both with the web server (via HTTP) and with the IM server (via XMPP over BOSH). In addition the IM server and the web server must route messages to one another, mediated by software ‘bots’. These messages must be queued to ensure that they are reliably received and processed asynchronously without causing undue delay in the request-response communication. We used AMQP for this, another emerging open standard developed for use in high volume transaction processing environments such as banking. XMPP allows us to effectively route queries to relevant tutors based on the subject of the query, without requiring tutors to constantly poll the server for updates, a practice known as ‘semantic query routing’. The result? Hopefully, a much more scalable architecture.

T-1 and counting

The days before a launch are always scary, even if its a closed beta. What if they think my baby is ugly? What if my baby stops working? OK, the analogy is running a little thin now, but you get the drift. Still, its an exciting time for us in an exciting market. I have a feeling the world of education is going to be transformed in the next couple of years, judging not just by the projects we’re working on, but some of the stuff I see our peers developing. At the risk of being of being a hammer salesman that sees all the world’s problems as a nail, I think tech will play a big part in that transformation. In the not so recent election, Barack Obama made the comment, “one of my fundamental beliefs … is that real change comes from the bottom up”. I’ll second you on that, Mr President. Maybe its time for us to stop looking to Governments to make the education system better for our children and start taking the small steps at the bottom that can lead to genuine revolution. Time will tell.


Regus House
1 Friary

Temple Quay
United Kingdom