Complex Project Failures: How Labels, Hierarchy & Ego Create Disasters in Management
I recently read an article and Facebook post that got under my skin.
Frankly, as an electrical engineer with 27 years experience in software and product development, a former member of APEGA, and owner of one of those “iron rings”, I disagree with almost everything in The Atlantic’s Programmers: Stop Calling Yourselves Engineers.
The article is clearly written from a place of ignorance about software development.
One of my missions as an Agile trainer, coach and industry leader is to debunk the myth that ego and hierarchy leads to success, and to demonstrate that collaboration is critical in our software-driven society.
A Case Study in Truth: Volkswagen Fail vs Toyota Success
The Atlantic article uses an example regarding Volkswagen citing “The Volkswagen diesel-emissions exploit was caused by a software failing, even if it seems to have been engineered, as it were, deliberately.”
This shows the author has little or no understanding of what happened at Volkswagen and failed to do the necessary research to find out. If he did, he would have discovered that managers told the engineers that 230 euro for the “add blue” emissions subsystem was too costly, and ordered them to find another way, without a budget.
In my certified Scrum courses, this is what I refer to as “belief in magic”. Most of the dysfunction described by the author in this article is not because of the lack of detailed planning or intricate documents but rather bad management decisions that employees, engineering ring or not, are forced to comply with.
Look at it this way… with $100+ billions in failures in software development and IT over the last 40 years, will four years of studying math and an engineering certificate fix these problems?
No, it won’t. It’s not the job title or credentials that causes failure. It is the organization that leads failure.
Scrum, Agile, Computer Science, Engineering, and Reducing Risk
Scrum and Agile are becoming increasingly popular, as noted by LinkedIn’s top 10 careers for 2017. The reason is because Scrum and Agile are the most effective risk mitigation strategies we know for software development.
The complexity of large software products dwarfs the complexity of traditional engineering projects like bridges and buildings.
Now, a bridge and all the engineering complexity required to keep it safe is serious and safety is critical. In fact, a Quebec bridge disaster inspired the Ritual of the Calling of the Engineer (written by Rudyard Kipling) and the iron ring. But, in terms of design and engineering, software is far more complex.
I have worked with brilliant people with advanced degrees in engineering and computer science. I have also worked with brilliant people who came to the world of software development from the fields of music and psychology. I have admired the huge talents of developers whose only education was through learning on their own. In one company I managed the senior product designer had a high school degree. His colleagues thought he was brilliant and so did I.
Bill Gates failed to complete his first year at Harvard. Does that make him any less of a person?
The Fail of Engineering & Software Without Organizational Change
The ideas in the original article were tried over and over for decades and they failed.
Companies like Google, Apple, Uber and Tesla are moving faster as they continually create tomorrow’s products today, while sleepy manufacturers take 4 years to release a new car that is the same as the old car.
I know this because these companies call me. They want to know how to work like Google and Facebook so they can innovate faster AND build quality in. Why should a farmer sit in a combine all day just to point it down a row of grain when it is possible to automate the machine’s navigation?
Smart People Seek to Make an Impact
A friend quit mechanical engineering after he learned that effectively all of the engineering calculations he had to do to design most components at his employer had been done in the 1940s and were published in tables. He went into IT because the work was more interesting to him since it was new and the problems weren’t solved.
One of my clients makes pacemakers and they use Scrum to build the software that resides in the pacemaker. They use Scrum and automated testing to demonstrate to the FDA that with every iteration (1 to 4 weeks) the working system has no bugs or defects.
Most of the failures cited in this article were failure of management to:
- understand technology
- empathize with the challenges of building and maintaining complex systems
- give staff the time to build quality in
Instead we see management treat IT and software development as purely a cost center without a clear connection to return on investment (ROI). This creates a “cheaper is better” mentality that causes huge dysfunction and waste (see my video on the $8 billion spent on the botched healthcare.gov rollout, a website saved by using Agile techniques and engineers from Google).
There are many problems you can’t engineer or code your way out of.
Organizational Change or Bust
Systems thinking and organizational design are potential solutions to fix the issues mentioned in the article. Many of these problems are rooted in managers making decisions about how the work should be done when they themselves are disconnected from how the work gets done. Would you appreciate having a hospital administrator instruct the surgeon on how to perform surgery on you? Probably only if the administrator is also an expert on your procedure.
Peter Senge wrote about systems thinking and organizational design years ago in the Fifth Discipline. There is no excuse to waste months and years and blame entire job functions when talented engineers, software developers, business managers, and others can create and deliver consumable, useful products as a team, every single week under a different organizational approach.
