The Chaos Report, or Comprehensive Human Appraisal for Originating Software, is a famous report about the IT industry. The Standish Group has published it since 1994. New editions of this report are being released every two years, more or less.
The main goals of the Chaos Report are to establish the number of projects that ended successfully and analyze factors (particularly the human factor) that influence the software development process. That’s why the report is a comprehensive source of knowledge about IT project management in software development.
The report reveals that only 31% of IT projects end successfully. It also explains how a project manager affects the success of the project and describes the success rate.
In this article, we tried to collect the most important information and give you a better idea of what the famous Chaos Report is.
Key information
- The Chaos Report divided the studied projects into three groups: successful, challenged, and failed.
- Many factors influence the success of the project; among them, we can find a good sponsor, a good team, and a good place.
- Organizations should move away from the project-based approach to developing software, leave managers behind, and reduce the use of software that facilitates project management.
Are you looking for an experienced UX agency?
What does the Chaos Report contain?
The report analyzes a sample of around 50.000 projects from various market segments and divides them into three groups:
- Successful projects: Finished on time with all planned features and within budget.
- Challenged projects: Exceeded the deadline and budget and offered fewer features than initially planned.
- Failed projects: The work on them was canceled.
The report published in 2020 (the newest version of the report) indicates the following:
- 31% of projects are entirely successful
- 50% of projects end up exceeding the deadline and budget
- 19% of projects are canceled
When we compare the report findings to the results of the previous study available on the Internet, we’ll observe that the percentage of successful projects (success rate) steadily stays around 30%.
Success rate — the percentage of successful projects
Until 2015, the report adopted the definition of success rate consisting of three factors. A successful project was software delivered on time, within budget, and with all planned features (a well-defined project scope).
However, this definition changed in 2015. To be more precise, a project finished on time, within budget, and with satisfactory results began to be considered a successful project.
The Standish Group made that decision because such a criterion better described the goals of managers and client satisfaction.
The new definition resulted in a 7% decrease in the success rate.
What influences a project's success?
The report also describes a dozen of factors that influence the project’s success.
Among them, we can find the following factors:
- Executive support — the management supports employees emotionally and financially.
- Emotional maturity — a collection of behaviors describing how employees cooperate.
- User involvement — encouraging users to share their experiences and taking their opinions into account.
- Optimization — increasing the business efficiency and optimizing processes.
- Skilled personnel — describes the high level of employee skills in technology and business.
- SAME (Standard Architectural Management Environment) — a collection of practices regarding software production, deployment, and operation.
- Proficient knowledge of Agile — determines the knowledge and skills in Agile methodology.
- Modest execution — describes processes consisting of simple, automated elements and limited use of project management tools.
- Project management expertise — a set of skills in project management.
- Clear business objectives — the ability to understand and align project goals with business objectives.
New success factors
In 2020, The Standish Group presented three new success factors.
- Good place
- Good team
- Good sponsor
Good place
A good place is a work environment where the team works on software. It consists of a sponsor, a team, and all other employees who work with them during the project. The influence of other employees can have a negative or positive impact on software development, so it’s important to continuously train and improve employees' professional qualifications.
Good team
A good team is the driving force of the project and has the most significant impact on the end result. The sponsor motivates, guides, and instructs the team. However, it ultimately depends on the team whether it will be able to deliver the expected results. One of the Standish Group’s recommendations is to form small teams.
Good sponsor
The Standish Group defines a good sponsor as the heart of the project, without which it can’t exist. According to them, the most important aspect that leads to success is continuously improving the skills of a sponsor so that they can effectively lead and support the team during the project. Simultaneously, it’s the easiest part of the project to improve because every team has only one sponsor.
It’s also worth mentioning that Chaos Report has been stating for years that using Agile methodology to manage projects is much more effective (around 3 times more) compared to projects managed in line with the Waterfall model.
The controversial conclusions of the Chaos Report 2020
According to the article, Project Managers Fail to Help Software Projects (Standish Group Chaos 2020), The Standish Group reached surprising conclusions that contradicted the recommendations presented in the previous editions of the report.
Can we treat software development as a project?
The Standish Group thinks that companies should stop treating software development as a project because, by definition, a project has an end. Instead, the group recommends using a continuous development approach with no defined end. Software should be continuously developed, maintained, and improved.
Additionally, the group emphasizes that dividing the entire project into smaller projects isn’t a solution either. It leads to working on smaller parts that still have a determined budget and deadline, and once the project is released into production, it turns out that users have different needs.
Hence, the recommended approach is small and continuous improvements that users can easily digest and test.
Are project managers truly needed?
The following controversial statement in the report is the conclusion that project managers do more harm than good during the project. Why this statement is so controversial? Because it contradicts the recommendations that The Standish Group has been preaching for 30 years.
Previous reports stressed the importance of a skilled project manager; however, in 2020, The Standish Group withdrew this recommendation. Instead, they identified a different group of factors that determine project success—good place, good team, and good sponsor.
The data presented in the abovementioned article show that the success rate of projects led by highly skilled managers is only 23% (for projects conducted in non-Agile methods). In projects without a manager, this metric rises to 58%.
In contrast, the statistics for projects conducted in Agile are as follows: the success rate for projects with a highly skilled manager is 18%, and for projects without a manager, it reaches 91%.
Based on the above data, the group concluded that project managers often generate unnecessary paperwork, which increases decision-making time and slows down the project's progress.
Another interesting finding from the Chaos Report 2020 is the negative influence of project management tools. The Standish Group based its conclusions on the Clarity software tests. The tests showed that using these types of tools lowers the success rate and increases the project cost.
The future of software projects
In the "Review Standish Group – CHAOS 2020: Beyond Infinity", we can find a division of software development history into 4 stages.
- The first stage lasted from 1960 to 1980, called “Wild West.”
- The second, “Waterfall Period,” lasted from 1980 until 2000.
- The “Agile Period” started in 2000 and continues to this day.
- The last stage, which is still to begin, was named the “Infinite Flow Period.”
The Standish Group says that we’re now at the end of the “Agile Period,” and we’re approaching a point when organizations will move away from budgets, plans, managers, and Scrum masters.
This stage will focus on integrating direct budget into the development process and minimizing activities related to project management. Software changes will be introduced continuously in small increments.
In light of these changes, The Standish Group announced that the Chaos Report 2020 will be the last edition. Since the approach to software development will change and result in moving away from projects, there is no reason to continue to analyze them.
Summary
Chaos Report is undoubtedly an extensive document providing interesting conclusions and data regarding the IT industry and software development. Organizations can find helpful pointers regarding areas in which they can improve.
We can summarize the main recommendations like this:
- Teams should use the Agile methodology.
- Instead of creating projects, we should focus on continuous and small increments.
- We should focus on improving factors such as a good place, a good team, and a good sponsor.
- We should avoid assigning managers to projects and reduce the use of project management tools.
Frequently asked questions
What is the Chaos Report?
The Chaos Report is a collection of conclusions based on the analysis of more than 50.000 IT projects. Its goal is to provide information on the success of software development projects.
Where can you download the full version of the report?
The newest version of the report (2020) can be purchased on The Standish Group website, and the 2015 version can be downloaded for free.
What does the Chaos Report show?
The Chaos Report shows that 31% of projects end successfully, providing all features included in initial assumptions, 50% of projects end with exceeded budget and deadline without meeting all the requirements, and 19% of projects are canceled.
What are the most important factors influencing the success of IT projects?
A good place, a good team, and a good sponsor are some of the most crucial factors influencing project success. Organizations should pay particular attention to them and optimize their processes based on them.
How to approach IT project management?
The Standish Group recommends that IT companies should stop assigning managers to projects based on Agile methodology to improve workflow. Additionally, they recommend minimizing the use of project management tools that reduce the success rate and increase project cost.
