Study finds 268% higher failure rates for Agile software projects

One standout statistic was that projects with clear requirements documented before development started were 97 percent more likely to succeed.

I'd like to work in that company.

Try medical software and devices. The requirements and specs are mandatory before doing anything. It’s actually very fun and I have less burnout thanks to this.
- I couldn’t disagree more.
  In medical I would end up being apart of endless retirement gathering meetings, then draft up the SOW doc only to have stakeholders change requirements when they were reviewing the doc. Then months later once the doc was finally finished and I could do the development, when UAT time finally came, they’d say the build wasn’t what they wanted (though it matched the written requirements).
  Most of the projects I saw executed in the last 4 years either got scrapped altogether or got bogged down in political bs for months trying to get the requirements “just right”.
  It was a nightmare. You could blame me, or the company, or bad processes all you want, but I’ve never had fun on a waterfall project, especially not in medical. (Though, in my opinion, we are severely understaffed and need like 4 more BAs.)
No thanks. It's way more fun to be part of the decision process. If a manager can anticipate all of the requirements and quirks of the project before it even starts, it's probably going to be a really boring, vanilla project at which point it's probably just better to but the software.ä somewhere else.
Creating something new is an art in itself. Why would you not want to be a part of that?
Also: Isn't it cheating to compare the two approaches when one of them is defined as having all the planning "outside" of the project scope? I would bet that the statistics in this report disregard ll those projects that died in the planning phase, leaving only the almost completed, easy project to succeed at a high rate.
It would be interesting to also compare the time/resources spent before each project died. My hunch is that for failed agile project, less total investment has been made before killing it off, as compared to front loading all of that project planning before the decision is made not to continue.
Complementary to this, I also think that Agile can have a tendency to keep alive projects that should have failed on the planning stage. "We do things not because they are easy, but we thought they would be easy". Underestimating happens for all project but for Agile, there should be a higher tendency to keep going because "we're almost done", forever.

I haven't read the article yet, but surely they can't be juxtaposing waterfall as the alternative to agile. The modern alternative, especially in small to medium businesses, would be kanban.
- Kanban is Agile. They are pushing Impact Engineering.

That's because they forgot the meaning of the word agility and want to apply the rules what ever the cost

And also because it's a comfortable cover up for any kind of money saving stupidity. We don't need proper requirements engineering, we're agile. We don't need an operations team we're doing an agile DevOps approach. We don't need frontend Devs, we're an agile team you all need to be full stack. I have often seen agility as an excuse to push more works towards the devs who aren't trained to do any of those tasks.
Also common problem is that still tons of people believe agile means unplanned. This definitely also contributes to projects failing that are just agile by name.
- 100% my experience.
A lot of places seem to view it as "we just work from the backlog" with no requirements on when features are delivered, or their impacts on other parts of the project.
You still need a plan, goals and a timeline. Not just a bucket of stuff to get done.
Or, even worse, they want to apply some of the rules, cherry-picking bits and pieces of a framework without truly understanding it.

Today, new research conducted for a new book, Impact Engineering, has shown that 65% software projects adopting Agile requirements engineering practices fail to be delivered on time and within budget, to a high standard of quality. By contrast, projects adopting a new Impact Engineering approach detailed in a new book released today only failed 10% of the time.

All you need to know about this study.

It almost sounds like a project team that is actually and actively looking to solve known and recurring problems instead of "just do whatever everyone else is kind of doing" might be why they are successful.
It's the difference between "how should we go about this" vs "see how we go" regardless of what you label those approaches as.
- I think the take away should be:
  new research conducted for a new book, Impact Engineering,
  By contrast, projects adopting a new Impact Engineering approach detailed in a new book released today only failed 10% of the time.
  So the people who want to sell you 'Impact Engineering' say 'Impact Engineering' is better than Agile.... Hardly an objective source.
  Even if they have success with their 'Impact Engineering' methodology, the second it becomes an Agile-level buzzword is the second it also becomes crap.
  The short of the real problem is that the typical software development project is subject to piss poor management, business planning, and/or developers and that piss poor management is always looking for some 'quick fix' in methodology to wave a wand and get business success without across the board competency.

A more proper title would be “study finds 268% higher failure rates for poorly planned software projects”.

“Agile” as a word is mostly an excuse of poor planners for their poor planning skills.

Yeah, Agile isn't really at fault here. If done right - if you've got a scrum master, a proper product owner, proper planning and backlog grooming, etc. - it works really well. The problem is some companies think Agile is just "give the devs some pie-in-the-sky hopes and dreams, let 'em loose, and if they don't give half a dozen execs exactly what they want (despite their massively conflicting ideas on what they want), cancel the project."
- In one the worst “poor planning” projects I’ve been in the product owner just kept sneaking in new “high priority” issues to the top of the backlog throughout the sprint. I don’t think we had a single sprint where we ended up with fewer open issues in the backlog than when we started.
  Needless to say, he was the main reason why I quit.
- In my experience it's just kanban, but make the devs feels guilty between sprints for not meeting their goals.
- Yeah, Agile isn’t really at fault here. If done right
  This is what ticks me off about the "Agile" brand, it's chock full of no true Scotsman fallacy (if a team failed while doing "Agile", it means they weren't being "Agile").
  I can appreciate sympathizing with some tenets as Agile might be presented, but the popularity and consultancy around it has pretty much ruined Agile as a brand.
  Broadly speaking, any attempt to capture nuance of "best practices" into a brand word/phrase will be ruined the second it becomes "popular".
Agreed. The problem is people mistake “zero planning and structure” to mean “agile”. Of course it fails.
Agile to me was always mini waterfall. You always know who’s doing what, why, and what success looks like on a 2 week sprint horizon. When you see people on a sprint without a clear understanding of what they are doing over the next couple of weeks - then you know your project is in trouble for sure.
I don't have much direct experience working in agile since I tend to work on the business side but I can tell you that the term agile is WAY overused. So many projects are described as agile when they are just waterfall with more steps. Leaders love to say they are working in agile because it sounds 'techy' and cool, but I don't think they fully appreciate what it is vs other methods. I wonder if a lot of the failed projects described in the article are some of those agile in name only kind of things.
An even better title would be "'Study' by firm pushing new technique finds old technique is bad."

If you know exactly what you need, then specs are great. Proven solutions for known problems are awesome. Agile is pointless in that circumstance.

But I can count on one hand the number of times stakeholders, or clients, actually know what they want ahead of time and accept what was built to spec with no amends.

When there is any uncertainty, changing a spec under waterfall is significantly worse. Contract negotiation in fixed price is a fucking nightmare of the client insisting the sky is red when the signed off spec states it's to be green.

If you know exactly what you need, then specs are great.
If you know exactly what you need and the specs are great, then you barely need project management framework at all.
Maybe I just work at shit companies, but it feels unrealistic to expect this this level of maturity from assigned work.
- Well, exactly.

Personally, I was never great with agile projects. I get that it’s good for most and sort of used it when I was a CTO but as a solo developer, there are days when I’d rather eat a bowl of hair than write code and then some days, I’ll work all night because I got inspired to finish a whole feature.

I realize I’m probably an exception that maybe proves the rule but I loathed daily stand-ups. Most people probably need the structure. I was more of a “Give me a goal and a deadline and leave me alone, especially at 9am.” person. (Relatedly, I was also a terrible high school student and amazing at college. Give me a book and a paper to write and you’ll have your paper. If you have daily bullshit and participation points, I’ll do enough to pass but no more.)

Stand-ups can become so proforma. What did you do yesterday? I coded. What are you doing today? I am going to code. Do you have any blockers? No. It gets a little repetitive after a while.
- I did twice a week when I was management: once at the start of a sprint, once on the first Friday where we only identified blockers, and once the following Wednesday where we talked about what can ship and be ready for QA.
  The goal was to have a release fully ready on Thursday so Friday could be for emergency bug fixes but most releases are fine. If everything is perfect, great! Everyone go have a three day weekend. If QA catches a bug or two, we fix it and then ship.
  If a deadline is gonna slip, just tell me when you know. It’s not usually a big deal.
- I think you are missing the part where you help others with their blockers.

Feels like the old php metric. PHP had a ton of great code and successful projects but it also attracted very bad devs as well as very inexperienced devs leading to a real quality problem.

Honestly kinda see thing in a lot of JavaScript applications these days. Brilliant code but also a ton of bad code to the point I get nervous opening a new project.

My point? It may be a tough pill but it's not the project framework that makes projects fail, it's how the project is run.

I witnessed a huge number of failed projects in my 25-year career. The cause was almost always the same: inexperienced developers trying to create a reusable product that could be applied to imagined future scenarios, leading to a vastly overcomplicated mess that couldn't even satisfy the needs of the original client. Made no difference what the language or framework was or what development methodology was utilized.
- I feel like that's the same underlying issue: The requirements are not understood upfront.
  If a customer cannot give you any specific information, you cannot cut any corners. You're pretty much forced to build a general framework, so that as the requirements become clearer, you're still equipped to handle them.
  I guess, the alternative is building a prototype, which you're allowed to throw away afterwards. I've never been able to do that, because our management does not understand that concept.
- I've seen a lot of contractors over promising timelines too. "No matter how hard you push and no matter what the priority, you can't increase the speed of light."
  But yeah exactly.
- Preach brother!
Yeah, look at the most prolific language at a given time. There's your crappy projects or your soon-to-be-crappy projects. What are the universities and 'coding academies teaching'? That's going to be the crappiest stuff in the world when those students come out.
So too it goes with 'management', the popular 'self-help' style crap of the moment is what crappy teams will adopt, and no matter what methodology it is, that crap team is still crap, and it will reflect on that methodology.

Does that surprise me? Not at all. "Agile" was never about making programming better. It was a management buzzword from the start.

We once had a manager who came to me with the serious idea "to make the development process agile". He had heard of this in a discussion with managers from other companies. The problem? I'm the only person in this department. I program everything alone. How the F should I turn my processes "agile"?

- I think he wanted it more like Product Owner, Scrum Master, Architect, Stakeholder, New product development, Tester, Integrator, Team member, Agile architect, Agile Coach, Developer, Team lead, Technical expert, Product Designer, Business Analyst, Programmer, and Specialist for at least eight hours a day in each role...

Isn't it more that people tend to use agile as an excuse for not having any kind of project plan.

It'd be interesting to know how many of those agile projects actually had an expert project lead versus just some random person who was picked who isn't actually experienced in project management.

In my experience It's not about a project plan for features, but actually doings things correctly instead of doing the minimum to finish what you need to do on the current sprint.
I'd say it's that people tend to use Agile because consultants tell them they can be piss poor managers dealing with the crappiest developers and stupid business ideas and still make awesome stuff if they just make everything buzzword compatible.
I'd say projects without much of an upfront project plan can still be very successful, but it's all about having a quality team, which isn't something a two week 'training and consultancy' session isn't going to get you, so there's no big marketing behind that sort of message.
Isn’t it more that people tend to use agile as an excuse for not having any kind of project plan.
I'd say it's more about continuously milking customers on projects that never seem to end. I've never done software project management, but I have seen it's "tenets" applied to other types of projects. The results were arduous - to say the least.
- I've seen it being done even on internal projects though. Things within an organization.
  It tends to be that they start developing a feature and then someone comes along and says, ooh wouldn't it be nice if it did x, so they modify it to have x feature. Then someone decides it should be able to sync with Azure (there's always someone that wants that), so Azure sync is added, but now that interferes with x, so that has to be modified so that it can sync as well. Then we get back to original product development which is now 3 weeks behind schedule.
  Repeat that enough times and you can see why a lot of this stuff fails.
Agreed. We follow agile, and we have a team of product owners who know where the project is likely headed in the next 3 years. Our sprint to sprint is usually pretty predictable, but we can and do make adjustments when new requirements come in. The product team decides how and when to adjust priorities, and they do a good job minimizing surprises.
It works pretty well imo, and it hinges on the product team knowing what they're doing.

I'm all for and good eye rolling at institutional Agile (basically checkered with bad management who doesn't know what to do, but abuses buzz words and asserts Agile instead), but this article has a lot of issues.

For one, it's a plug for someone's consultancy, banking on recognition that, like always, crappy teams deliver crappy results and "Agile" didn't fix it, but I promise I have a methodology to make your bad team good.

For another, it seems to gauge success based on how developers felt if they succeeded. Developers will always gripe about evolving requirements, so if they think requirements were set in stone early, they will proclaim greatness (even if the users/customers hate it and it's a commercial failure).

This article doesn't make any sense. A project's "success" can't really be measured in any objective way like the article is implying. Even saying that a project is "on time" is a vague statement depending on the situation, and it's not a good way to measure the quality of the end result or the efficiency of the development team.

Imo waterfall is an imagined beast for most software devs today. I worked on many successful waterfall projects. It was nowhere as bad as the caricature that people imagine.

Cries in waterfall.

... I cannot count the number of times at my different workplaces where we had an agile process, dailies and everything else of the agile BS for projects which where either trivial or not solvable. No worries, the managers, product owners and agile coaches made money and felt good, we developers went for greener pastures...

Agile is a scam, nothing they do is based on any facts and when you challenge agile coaches / other people which profit it is always 'I believe' or 'proven by anecdote'.

Combine this with the low quality of people in the average software projects and you have a receipt for failure.

Writing the requirements first at least forces people to think trough a project (even if only superficial), so I am not surprised the success rates for this projects goes up.

Agile has its uses, but like everything you need a bit of both. You need a bit of both waterfall and agile.
Example : you need to have your requirements before development, yes. But how far do you go in your requirements? If i were to make all the requirements for my current project ill still be busy in 3 years and will have to redo bits due to law and workflows changing. however , we need requirements to start development. We need to know what we need to make and what general direction it will be heading to a make correct software/code design.
Agile also teaches you about feedback loops, which even with waterfall, you need to have to know that what youre developing is still up to spec with what the product owner is expecting. So even with waterfall, deliver features in parts or sit together at least once every x weeks to see if youre still good with the code/look/design.
Pure agile is bullshit, but so is pure waterfall. Anything that isnt a mix is bullshit and in the end, it all depends on the project, the team and the time/money constraints.
- Exactly!
  I worked at one Agile place they had all their sprints and milestones in a Gantt Chart waterfall. They also did big design up front and a lot of process. They had do all kind agile and scrum training, but it was the most process heavy place I worked.
- Good points, and I mostly agree with you, especially with feedback loops!
  Still, I never argued for waterfall. This is a false dichotomy which - again - comes from the agile BS crowd. The waterfall UML diagram upfront, model driven and other attempts of the 90s/early 20s were and are BS, which was obvious for most of us developers, even back then.
  Very obviously requirements can change because of various reasons, things sometimes have to be tried out etc. I keep my point, that there has to exist requirements and a plan first, so one can actually find meaningful feedback loops, incorporate feedback meaningfully and understand what needs to be adapted/changed and what ripple effects some changes will have.
  Call it an iterative process with a focus on understanding/learning. I refuse to call this in any way agile. :-P

The few times I’ve been on an agile project it amounted to start writing without understanding what product we’re building.

Yeah. Which actually doesn't have to be bad as long as leadership accepts that this exploratory work (sometimes called a "spike") might have to be thrown away, if findings reveal better paths.
The trouble begins when you start shipping your proof-of-concepts (without immediately paying back that tech debt).
It very quickly becomes an unmaintainable mess.

I'm always sceptical about results like these. I was told that waterfall always failed when I'd worked on successful waterfall projects with no fails. The complaints about waterfall were exaggerated as I think are complaints about agile. The loudest complaints seem to always be motivated by people trying to sell sonething

My crazy wacko conspiracy theory - software development is just a really weird discipline, most of the people in the field are bad at it, and it doesn't have the same amount of standardization and regulation that other engineering fields have, so doing it "right" looks a lot fuzzier than doing, say, civil engineering "right".
The biggest thing though is that most people are bad at it. It's really hard to evaluate high level organizational concepts like waterfall vs. agile when we still have developers arguing over the usefulness of unit tests.
- I so agree with you. Especially that software engineering is not like actual engineering. Ironically that's the first point of the agile manifesto - is all about the people and interactions, not the tools and processes. That's why I'm leery about these grand claims about agile failures when half the time they mean scrum and just doing scrum isn't agile (see point one of the manifesto)
- I think it's more that they are trying to solve the problem by changing the dev team processes, when the biggest factor of success is developing the RIGHT thing. But since most tech managers have risen up from the ranks of devs, and they have a hard time understanding that other people have valuable skills they don't, they have no idea how to hire good designers and refuse to listen to them when they happen to get one.
Ignoring the success and failure of agile and waterfall. Waterfall was just a way more enjoyable development experience for me. That would probably change if the cycle was lower though. Also doesn't help that many managers I've had don't follow the rules of agile/SCRUM. Seems like people use it as an excuse to be able to change things on any given day but those cycles are supposed to be planned, not the plans.
- Yeah actually i hadn't thought about that aspect of it, but I did enjoy waterfall projects much more.

It seems very biased to say the least. A title like that would be ok if it was comparing agile to pre-existing models like waterfall. A valid title for this would have been "new sw development methodology seems to have a much lower failure rate than agile dev. "

ALSO I would like to see the experiment repeated by independent researchers.

"new sw development methodology seems to have a much lower failure rate than agile dev, claims people who stand to make money if new sw development methodology has a lower failure rate. "

what matters when it comes to delivering high-quality software on time and within budget is a robust requirements engineering process and having the psychological safety to discuss and solve problems when they emerge, whilst taking steps to prevent developer burnout.

I haven’t read the book they’re advertising here, but I’ve found these challenges to be socially created, not caused by agile.

Oh, so Agile is only done by autonomous AI?
- In the article, they’re proposing a solution to agile (“Impact Development” or something). The quote I listed above is talking about how Impact Development is supposed to provide those things. That said, I don’t blame agile for projects not having those things, it’s the people’s fault. So changing methodologies likely won’t help.
  In short: yes, make AI do all project management :P

As someone who practices agile software development I find this baffling. I've never started a new project without at least 3 weeks worth of research and requirements gathering (and obviously more as the project rolls on). There are seriously companies out there who are like, "Mmm, I feel like this is gonna be an Electron project. Let's just lay the groundwork and we'll flesh out the nitty gritty in a week or so." 😱

Not gonna read it because we, elsewhere in engineering land, have been forced to eat Agile shit from the water hose to make us better and faster. Fucking hell! I can't re-compile a mirror if it comes out wrong!

I hope "New Impact" includes hammers.

An Agile Project eh. Like an Agile Waterfall process? cool. Cool cool cool.

I know PMI has an Agile thing but by and large Agile can't be "projects" and vice versa.

Oh well, time to switch back to the waterfall model I guess

lol, no.

Move fast and break shit!

Fun mental exercise - remove the formalism behind agile methodologies out of software development. How is that any different from driving another human being mad?

I have altered the specifications. Play I do not alter them any further.

This is the best summary I could come up with:

Even though the research commissioned by consultancy Engprax could be seen as a thinly veiled plug for Impact Engineering methodology, it feeds into the suspicion that the Agile Manifesto might not be all it's cracked up to be.

One standout statistic was that projects with clear requirements documented before development started were 97 percent more likely to succeed.

"Our research has shown that what matters when it comes to delivering high-quality software on time and within budget is a robust requirements engineering process and having the psychological safety to discuss and solve problems when they emerge, whilst taking steps to prevent developer burnout."

A neverending stream of patches indicates that quality might not be what it once was, and code turning up in an unfinished or ill-considered state have all been attributed to Agile practices.

One Agile developer criticized the daily stand-up element, describing it to The Register as "a feast of regurgitation."

In highlighting the need to understand the requirements before development begins, the research charts a path between Agile purists and Waterfall advocates.

The original article contains 502 words, the summary contains 175 words. Saved 65%. I'm a bot and I'm open source!