Three Months Open Development

Jeni Tennison

When the tech team started in January, we had some immediate demands from the rest of the ODI team, to support the business:

Within the technical team, we had another layer of objectives. We didn’t just want to build these systems, we wanted to build them in the right way. We recognised the golden opportunity that we had, with very few legacy systems, to use the best modern technologies in what we built. With no legacy processes, we could Do Agile Right, from the start. And with the ODI’s mission-driven focus, we could be completely open about our work at every stage.

I had a further goal. I wanted to demonstrate how to weave open data into the heart of what an organisation does. If we were going to build systems to support our work, open data was going to be at their heart, not an appendix.

So how have we done?

Small Pieces, Loosely Joined

One of our tech team principles is to borrow mercilessly. We don’t want to recreate existing tools and services when we can reuse what’s already out there.

One of the joys of starting a new organisation at this point in time is “there’s an app for that”. So we reused those apps: Google Apps for our mail, calendars and documents; Eventbrite to manage our events; CapsuleCRM to keep track of our contacts; Xero for our finances; Expensify for our expenses.

But nowadays there’s not only an app for that, there’s an API for that. Each of these tools provides a mechanism for developers to get hold of the data that they hold, and most also offer ways to create new data. Our job was simply (relatively) to customise the apps and join them together. (There’s even an app for that, but we didn’t use it.)

What are the lessons we’ve learned from this approach? Well, of course there are trade-offs. We’re reliant on services which could disappear, or radically change, outside our control. We have to pay for them, but it would have cost us time and money to develop them ourselves too.

From a data perspective, the most interesting thing is that every service has a completely different way of structuring information about even common things like people or locations. When using APIs, the developer documentation is crucial because they’re seldom self-documenting, and existing open source tools and libraries for using the code really smooth the way, even if they’re not complete.

Our Open Data

Some of the joining that we’ve done has required us to expose data. For example, to get the prices of the training courses show on the website or to display information about the lunchtime lectures on the training dashboard. In the new data that we’ve generated, we’ve adopted the vocabulary where we can.

In other cases, it’s been as easy to expose data as not, so we’ve done so not because we’re using it but in case anyone else wants to. For example, making iCal feeds of the busy/free calendars for our meeting rooms, or JSON versions of the information about our members, is easy to do.

The bits that have proven difficult have exposed some areas where we want to do work. The biggest of these is about how to expose, in a machine-readable way, the licence under which particular data is being made available. We plan to start a small project to create some easy-to-use guidance about associating licences with data.

At a social level, we’ve had to think about privacy, confidentiality and commercial considerations. The data generated by the services that we’re using is only accessible to us, so we can choose how much to expose and how much to hide. For example, we’ve chosen to let everyone see how much our space is being used, but hide information about who is attending meetings in our rooms.

Our Open Development

As well as supporting the rest of the team, and publishing open data, the last three months have been an opportunity for us to put into place the tools and practices that make it easy for us to work quickly, create good code, and provide practical contributions to the wider community. The way we work is summarised in these slides on Open Development.

What Next?

This has been a really important phase for the team, setting in place the principles of how we work, while getting real work done that has supported the rest of the organisation.

We’re now looking forward to the next phase, where we turn our sights on to building the tools and practices that should make it easier for people to publish, consume and collaborate over data, based at on the projects that we’re doing with our members.