Water-Scrum-Fail
It wasn't waterfall. It wasn't scrum. It was both and it was neither and it was doomed to fail from the get-go.
Once you've been doing this for a while, you'll understand that pretty much every aspect of the discipline, at least in a professional setting, has a trade-off.
To put it as simply as possible, we can't have everything, we can't do everything. At some point time, money, deadline pressure, prioritisation of other projects or one from a myriad of other concerns will cause us to have to make some tough decisions.
This could be dropping a feature, putting off a refactoring that we know we need to do, not fixing that bug with the automated tests that causes 1 in 5 to fail, etc., etc. All of the above fall into the category of purposefully taking on technical debt. That can be a good thing if the alternative is the project's cancellation.
Like any debt, if not dealt with over time, technical debt can grow out of control. At some stage, it must be paid down. Otherwise, you are continuously jeopardising the long term future of the project for short term gains. That situation can not be allowed to go on indefinitely. If you've seen this happening and have told the responsible project management people and nothing is being done, seriously consider looking for another job, because this approach *will* cause headaches and stress for every stakeholder.
Left unresolved, adding each new feature and fixing each new bug leads to more and more regressions and the software gets increasingly work-hardened and brittle as you continue to hammer at it. Refactoring and paying down the other technical debts is the annealing process that allows things to move forwards again. Things will break if you don't take this time. People, too.
If you don't, the fact that each feature or bug fix is taking longer than it should can result in the adoption of crunch. Overtime is required to complete basic tasks. Two-week sprints are two weeks plus the extra few hours a day. Velocity now looks to be on track, so the project managers think it's fine. Except you're doing half as much again as you should be.
Burnout is next. Velocity falls off a cliff. Little to nothing is getting done. This could apply to the whole team or just one or two people. If you're already doing overtime as well the problem is exacerbated: there's no time left in the day to claw anything back. The project's technical debt ends up getting compounded, causing the long term issues that have been festering away to grow in size. By this stage, even short term gains are hard to come by.
All the stakeholders are now completely stressed out. Patience has worn thin. Some kind of eruption is inevitable. Depending on how it's handled, how pot invested the parties involved are, this could be the blissful end of the project, releasing everyone from its yoke. Alternatively, it's time to take a long hard look at the failings that have led to this point, heed warnings from those who have been vocal in providing them and make changes to the approach.
At the beginning of this post, I spoke of trade-offs. The development approach for this ill-fated project has been a defiant amalgamation of waterfall and scrum. Playing at Agile, not fully committing. Taking waterfall's Big Design Up Front, without the nuance Joel Spolsky provides here:
"This specification is simply a starting point for the design of Aardvark 1.0, not a final blueprint. As we start to build the product, we'll discover a lot of things that won't work exactly as planned. We'll invent new features, we'll change things, we'll refine the wording, etc. We'll try to keep the spec up to date as things change. By no means should you consider this spec to be some kind of holy, cast-in-stone law."
https://www.joelonsoftware.com/2005/08/17/the-project-aardvark-spec/
BDUF is still relevant in Agile. You need a solid overview of what the project needs to achieve, how critical pieces need to work to fulfill the requirements. What's important here though, is both the level of abstraction at which the spec goes into and the level and nature of the details. Crucial also, is the project's stakeholders' framing of this document. It is not possible for it to cover every detail, nor is it possible for it to be a 100% accurate description of the final software. Nor should it be.
In an Agile development workflow, the spec should serve as a high-level overview of the final software and it should go into sufficient detail on the critical aspects of the business logic that cannot and should not be second-guessed. We're talking about a functional spec here not a technical one, so application architecture, database tables, database types and the like do not have a place. Nor should there be a place for talking about a model before the extent of what is needed to be built is even known.
The spec should also be narrow in its scope. It is there for the development team to understand what they need to build. It is there for the testing team to understand what they need to verify. It is not a manual. It is not there to describe how business processes should interface with the software. Going beyond a narrow scope merely makes it overly difficult for the aforementioned two groups to do what they need to do to build the software.
An over scoped spec creates unnecessary bugs, it is as simple as that. Bugs are a fact of software but unnecessary bugs are completely avoidable technical debt. There's more to do than there should be, so more time has to be spent. That's how a deadline slips further away. Even if the deadline stays on track, it's time that could have been spent elsewhere: an opportunity cost.
What's the solution?
Have a narrowly scoped spec that consists of a high-level overview of the requirements and critical details where necessary. (Have a good project manager to do this.)
Ensure all stakeholders are aware of for what and for whom the spec is actually for.
Always be mindful of where and how you're taking on technical debt and recognise the times and places where you can pay it down. (An empowered senior engineer is best placed to make these calls. Empowered in the sense that they can say "this sprint must contain this technical debt paydown" and be listened to, even at the expense of feature development.)
If you're going to follow Agile, have some follow-through. Don't just play at it. A half-arsed approach leads to half-arsed results.
If you're following Agile, consider also the lessons of lean manufacturing as applied to software by Eric Ries in The Lean Startup. If the spec is as I've described, it won't be difficult to extract from the project owner(s) what an MVP should contain and how it should function. From there, you are eliminating deadline related problems whilst also getting the software into the hands of the users who will actually shape it at an early enough stage where reshaping is easier.
Most waste in software development comes from features that weren't as necessary as first thought, bug fixes that were misprioritised over more critical fixes. Getting the software into the hands of actual users early ensures that you're building what is needed, not what the stakeholders think is needed. That can prevent a mad scramble for a patch release when it could have simply been a regular sprint's development. We're now avoiding a potential crunch. We're building something that is actually being used. The users don't need as much training because the software has been shaped around how they're actually using it.
This isn't some fictional utopia. It's a function of taking the right approach, making the right trade-offs at the right times and not repeating the mistakes of the past.