Is Agile Stupid?
Another thing I have going round in my head is that one Ruby Rogues panelist (coraline?) said that agile is stupid, because only planning ahead two weeks at a time makes no sense. I think the implication is that you need to have a longer term plan than that. In contrast my intuition is that having a very detailed 6 month, 2 year and 5 year plan is similarly stupid, although hey, maybe that's me just getting life wrong :-) I mean I have rough 6 month, 2 year and 5 year plans which are approximately:
- 6 months: at least double the number of AV premium members and/or get in one paying charity contract
- 2 years: have an order of magnitude more AV premium members and/or multiple paying charity contracts
- 5 years: have AgileVentures running under it's own steam so that I can do more exploration of whatever takes my fancy at the time
I could make much more detailed plans than that, but the further I go out the more uncertainty there is and the greater the danger that I'm making a detailed plan that will never be used. Maybe planning itself is a worthwhile pursuit; maybe we learn something through the process of planning? Maybe by making detailed plans that we throw away, we develop stronger plan making skills for future plan making? It's just that, in my experience, plan making activities tend to run afoul of a lack of relevant information. I've been in a startup company spending lots of time planning a detailed domain model that then gets thrown out as the business changes, or the technology behaves differently from expectations; but maybe I just haven't had enough experience about those situations when the detailed planning bears lots of fruit?
The Agile concept as presented in the "Agile Development using Ruby on Rails" MOOC suggests that the icebox of user stories represents the long term plan. We might only choose a small number of stories to work on in a two week sprint, but trying to plan which stories to work on each week for the next 20 weeks seems doomed to failure, unless you are hugely much better at predicting how long things take than anyone I know, or have a skill with technology again far in excess of anyone I've ever met. The process of long term planning we use in the LocalSupport project is via Pivotal Tracker's "kanban" style columns. Every feature we ever thought of, or the client requested, goes in the icebox. Anything voted on for expected complexity goes in the backlog, which we order so that items more important to the client are at the top.
Everything is in the context of the client's overarching goal of trying to build a hub for the residents and 3rd sector (charity and non-profit) organisations to find out about the services available and network for volunteering purposes. Just exactly what features will best support that is an open question that we're trying to answer by a process of exploration; a process of trying to deliver individual features to see which ones get used, which ones the client likes etc. Our longer term goal is to keep improving and pivoting to see if we can really start providing value to our client and by extension the residents of Harrow, the borough of London the software is currently targeted at; but I'd still call this all Agile, and I hope it's not completely stupid :-)
The planning and management of some other projects I'm involved in are proceeding along similar lines. WebSiteOne, the software behind the AgileVentures site, uses Waffle (basically GitHub issues) to manage the features that we want to release. In this case we don't have a single client we're building the system for, just a user base, so we prioritise our backlog by what we think would be most effective in supporting our users, particularly those paying for premium membership. Yesterday we just pushed out the bare bones "update your credit card update" feature that we had developed last week. We did the simplest thing possible - there's no nice UX to allow the average user to discover the functionality. There's a single endpoint that you need to be told about. We're just doing the absolute simplest thing to test if the system will work - although note the functionality is totally wrapped in acceptance tests. It's almost as if we were deploying with a feature flag and we weren't showing the UX path to the feature components yet; in order to move fast we didn't even create those UX components.
So I was pretty pleased when the feature worked yesterday for one of our early adopting premium members who needed to change their credit card in a hurry. But, shouldn't we be making a detailed plan for rolling out the rest of the functionality? And indeed lots of other functionality related to credit cards? Well we have an "epic" ticket that collects together some of the other features we're hoping to deploy to make card management easier for our premium members, and administration simpler for us; but the real question in my mind is, are we ever going to be able to scale the number of premium members? Perhaps we should be investing more time in the static site that we have to promote our services to charities and non-profits? Those are difficult questions to answer, or at least to predict the answers with certainty. So only working on the simplest sliver of functionality that can fit into a day, or functionalities that can fit into a two week sprint is a way of hedging our bets to see if we can get more information about whether the technology works (or we can get it to work in a reasonable time), and whether any significant numbers will use what we are deploying.
Maybe I'm missing a beat in terms of creating polished products. Maybe hedging our bets means that each of the projects individually doesn't shine sufficiently in terms of quality to make anyone want to use any of them? Maybe instead of trying to carefully gather information about technologies and what people seem to want to use, we should be throwing everything in on one project, and just going nuts to make that look totally awesome, have an incredible UX, a wonderful codebase? Maybe we'll have selected the wrong thing that no one actually needs, but I guess we'd at least have something in our portfolio that we could be really proud of, and the awesomeness of which would open other doors? Even if so I'd still be tempted to work with an Agile backlog rather than a Gant chart - I'm not a fan of long term planning, but I've been all sorts of wrong before :-)
The other example I can give from yesterday, which in my mind was also driving this blog post, is that Michael and I rolled out some basic http authentication to the ProjectScope system to ensure that the sensitive API tokens required some sort of login. In the longer term we want to easily manage different teams of people getting access to the system and so we'll want a more complex feature that will work with GitHub authentication or something, but again, we don't know how many people will ultimately be using ProjectScope. This fall it's going to be Armando and his Berkeley TAs and in the Spring it will be some AV folks. We got the http authentication out quickly, which in Rails is as simple as:
http_basic_authenticate_with name: "cs169", password: ENV['PROJECTSCOPE_PASSWORD'], except: :index
Although of course getting all the tests sorted took an order of magnitude longer, and then there was deploying and testing manually, but with that item out we can say that this system is ready to be used without any major security holes. Now we let out a breath, and look through the backlog and weigh up the different features that we might add. Improve the interface? Add documentation for adding new project metrics? More complex access management? Improve the efficiency of some of the individual metric gems? To me Agile isn't about a lack of planning, it's about a process of breaking down the longer more complex planning process into a series of component problems, and then being able to reason about their relative importance, decide which one to explore first and then keep updating our understanding of that sub-problem and the few others connected to it. I'm going to stick with it for at least another six months, and then just maybe you'll see me with a Gant chart :-)