Ryan Singer on the problems with Agile today

Agile is often viewed as a panacea to the common pitfalls of product development, allowing teams to ship faster, with higher quality, through a customer-focused lens. But what if Agile itself has flaws? Is there a better way to develop products? We sat down with Ryan Singer, Head of Strategy at Basecamp, to discuss why companies are adopting the alternative development model described in their recent book, Shape Up.

Basecamp has written a lot about product development over the years. What was the impetus for this new book?

A lot of the core principles in the book were already in play from the beginning, when we were just three people building Basecamp. When there’s only three of you, you don’t need to formalize what you’re doing, or turn it into a system or a process. It’s just something that you do naturally.

As we took on bigger projects, we needed to come up with a more structured way to tackle those projects and meet new challenges. In 2009, we had a big tech project called 37 ID, when we first prototyped some of these techniques. Later on in 2012, we redesigned Basecamp from scratch to build Basecamp 2. That was the second opportunity to really test these techniques, because we were challenged by the scope of the project.

Along the way, we grew to over 50 people in the entire company, with about a dozen on the Product team. We needed a way to articulate how we make decisions, and create a framework that new people could put into practice. I’d been trying to find the right language for things we’d been doing intuitively. So, I hosted a workshop to find out more about what people are struggling with in product development, and to prototype the material that became this book. I wanted to make sure it would connect with people.

What do you see as the main problems with Agile?

Two main issues jump out. The first one is working in two-week sprints. Two weeks is simply not enough time to actually accomplish anything meaningful. You can’t build a product feature end-to-end. You can’t start something, finish it, ship it, and move on to something else in two weeks.

On top of that, you have this high overhead and the extra cost of stopping what you’re doing and getting together to decide what you want to do every two weeks. You lose the time of that meeting, but you also lose all the momentum the team has gained. It takes time to hit a stride again.

The second problem is that when teams work in sprints, they end up in a situation that I call the “paper shredder.” Somebody takes a project they want the team to do, and then they have to break it down into tasks that can be slotted into a sprint. It’s like running a project through a paper shredder.

You end up giving people these individual tasks, hoping the project will somehow become whole again in the end. It never works out like that. There’s a difference between the work that we think that we have to do upfront – the “imagined tasks” – and the work that we discover we actually have to do once we dig in. Those are the “discovered tasks.”

Once they start, the people actually doing the work often discover it’s more complicated than they thought, or there’s more to it than was estimated. Or maybe they realize once it’s coded or in the interface that the specified task wasn’t even what they really wanted. So then the actual work takes longer than they thought it would, and work that was put into the sprint that doesn’t get finished, and it gets pushed down to the next sprint. Now you’re on this treadmill. More and more work, another two weeks, another two weeks, and it’s not clear where the end is in sight.

That sounds rough. How are the people doing the work affected?

Yes – the third big pain point with Agile is that it creates really bad morale. When you use a paper shredder, it reduces the team to ticket takers, or code monkeys, or grunt workers.

Instead of being responsible for thinking about how all the pieces come together, making decisions, and having ownership over the integration of all the different tasks into something that really works, they’re just tasked with doing one little piece and checking a box. Executing tasks from a queue day after day is really demoralizing.

So, what’s the alternative to Agile?

In the method we describe in the book, we try to address some of these problems. Instead of having too little time our cycles last six weeks, which is plenty of time to accomplish something meaningful.

Instead of putting the project through a paper shredder and assigning tasks, we assign them the entire project. Then the people doing the work actually discover, capture, and track their own tasks as they figure out what the real work is. As a result, the team has greater morale, more enthusiasm, and a feeling of satisfaction and accomplishment.

During that six-week cycle, we give them uninterrupted time to find their own solution, applying their own creativity and expertise to figure out how to design and execute the work. At the end of six weeks, they get to ship it, deploy it, and high-five each other and say, “Hey, look! We got something done! We moved the needle, we got the product to a better place, we did something that made a difference!” How great is that?

It sounds like you empower the teams. Is there a product owner or product manager?

I’ve tried to get away from titles that can be ambiguous. Some places call it a product manager, but what is a product manager? In some cases, a product manager is making strategic decisions about what to do, and sometimes they’re leading the design. But a lot of times, they’re actually shuffling around a lot of priorities that are getting pushed on them from above.

Rather than relying on titles like “product manager” or “product owner,” I’ve separated out the actual activities that need to take place – and those could be embodied within one person, or split up among different people. This depends very much on the scale of the organization.

Before we make a commitment and give a project to a team, we have to shape that project. Shaping is the pre-work that we do to figure out what we’re actually trying to do. What are the boundaries of what this project is and what it isn’t? What’s the main outline of the solution? What’s our definition of the problem? What have we done to remove risk from it as much as we can, and improve our odds of success, so that if we were to bet on this project and we were to give it to a team, then they would have a clear idea of what to do with it and their odds of actually being able to finish it within the allotted time are higher.

Tell me more about the shaping process.

Just like how every omelet starts with raw eggs, every project starts with unshaped work. This could be a request from a customer, an idea the CEO had in the shower yesterday, or some other kind of raw idea. You can either do the shaping to clarify the boundaries and expectations about what’s going to happen when you make a bet on the work, or you can hand it off to a team, cross your fingers, and hope for the best.

Shaping is about defining the work at the right level of roughness, and the right level of abstraction. We don’t want it to be so vague that we’re just hoping the team comes up with the right approach. But we also don’t want it to be so concrete that we’re giving the team specific tasks that are going to blow up in our faces, since we don’t actually know what the real work is yet. We’re being more deliberate upfront about what we want strategically, so the team has guardrails to improve their odds of success.

Can larger organizations use this method?

I now see at least two different organizations adopting the “shape up” approach at a massive scale. They have hundreds of developers, and thousands of people overall, and they’re thrilled with the change that’s happening. They’re having different conversations, with new language and the ability to articulate the different phases of the work, and tease apart why things are going wrong, when they’re going wrong, and how to approach their work differently.

The main structure of the book is divided into three parts: shaping, betting, and building. We need to get out of this black-and-white tension of either specifying the whole project upfront and giving people tasks to do, or leaving it up to the team to figure it out. Once we make a bet on a project and it gets handed off to a team, then this team is responsible for the project. They are the ones who are going to have the most information about the actual challenges and interdependencies and tradeoffs that need to be made to execute the work. We’re giving them total freedom to come up with exactly what the solution is.

But at the same time, our risk is reduced because they have boundaries on what they’re doing, because of the shaping work that we did prior. That helps them to focus their efforts and know what to solve and what to ignore. At the same time, there’s just so many thousands and thousands of little details and problems that they have to work out. It’s quite a lot for them to come up with on their own.