When to employ Agile and when not to. Can Agile and Waterfall work together?

Introduction

This article is the 2nd installment in a series of two articles focusing on understanding different problem complexities and learning to select the optimal strategy for solving a specific problem type.
Last time we started by discussing the different complexities problems can have. We also visited the different strategies we need to employ to solve problems with different complexities.

In this article we will continue by focusing on the different organizational structures, cultures and business processes necessary to solve different types of problems.

Click here to read the previous article: Previous Article

How to read this article

This article uses evidence based coaching exercises to help you come up with your own views, generate insights and reflect on your own situation. Although it contains a lot of useful information about organizational design, the article was not designed to be solely read or skimmed through. It has multiple short exercises which, together with the information in the text, can help you reach deeper insights and generate useful actions.
Skipping any of the exercises in this article will eliminate a huge portion of the value that you could take away. It is highly recommended that you do each exercise as you encounter them and before continuing with the article.

Problem Complexity determines the optimal organizational design

In the following paragraphs we will discover how complexity and the level of task interdependence influence the optimal organizational structure, business processes, leadership style and organizational culture.

Task interdependence is the extent to which multiple people have to work together, interactively exchanging help and resources to deliver end value to the customer. As Complexity increases from Simple to Complex, typically task interdependence also increases.

In reality, I’ve found that most organizations use structures, processes and have a culture that is unfit for the type of problems they are trying to solve. In the long run they experience a range of issues from loss of business, to failed products, low employee retention, not being able to attract the best talent, a passive workforce, lots of waste in the system, etc.
I will encourage you to compare your organization’s current structure, ways of working and culture to the ones listed below and see if it is a good fit for the type of problems your organization solves.

What if my organization solves lots of different types of problems?

Most organizations solve lots of different types of problems; however, usually there is a dominant one.
Organizations that deal with lots of different complexities usually have multiple departments or subgroups which tackle very different types of problems and have very different structures and cultures.

What is the dominant type of problem your organization solves?
Out of all the problems your organization solves what percentage of them is Simple? Complicated? Complex? Complex Adaptive? Chaotic?
Your answer doesn’t need to be 100% accurate, this is not a test, an approximation will suffice. We will use these numbers to compare your organization’s current structure, processes and values with the optimal one later.

Disclaimer

For each organizational design I’ll present a range of strategies, behaviours and values that work in practice. I don’t necessarily endorse the corresponding behaviours.
For example, organizations that solve Simple Problems tend to mistreat people; they believe people are ‘resources’ or cogs in a machine. This happens everywhere unless government regulations stop it.
I’m presenting these designs purely from the perspective of what has proven to work in practice; some of these designs are against my own personal values and beliefs.

How should I structure my organization to solve “Simple” Problems?

What processes, leadership style and company culture do I need to be successful?

The solution can be determined at the outset

As we’ve seen, one of the key characteristics of simple problems is that the solution to them can be determined at the outset. This solution is not going to change or the change is insignificant (otherwise we would be talking about Complicated or Complex problems). Most physical labour falls into this category.

Separate decision making from execution

Consequently when solving Simple Problems, it’s a good idea to separate decision making from execution.
Execution of a predefined plan is a repetitive activity in this domain, we don’t expect to learn anything new, and the need for feedback is negligible. These plans typically require either unskilled labour or labour that is highly specialized in doing one thing only.
It makes sense in this case to have highly educated, skilled experts who design the “right way” of doing things up front. After all, there is an optimal way of brushing your teeth if you want to avoid cavities and there isn’t too much value in constantly revisiting this strategy.

The successful organizational structure for Simple Problems is hierarchical

The concept of the manager is born with the intent to design and plan activities and then try to control the behaviour of employees.
Early industrial revolution activities fall into this category like putting coal into a steam engine using a shovel and physical labour.

Top 5 things organizations that are successful solving “Simple” Problems value

1. Compliance: follow the plan, policies, processes and use predetermined tools

These organizations will expect and value obedience over and above anything else. Which makes sense as following the predefined plan guarantees good outcomes in almost all instances.

2. Making early commitments and then delivering them on time and on budget

The requirements are known up front and the way to fulfil them as well. It is possible to commit to a deliverable and calculate the time, effort, money, etc. necessary to deliver.
Success is determined by how hard and fast doers can work, how accurately management can plan and how good management is in enforcing the rules.

3. Standardization

There is an optimal way of cutting out a piece of wood. This is knowable up front, if we learn further details later, we will want everybody to adopt this one optimal way. There are best practices in the Simple Problem domain and these organizations put effort into embedding these best practices.

4. Managers

The only job of workers is to execute a predefined plan, they are typically unskilled or low skilled and viewed as easily replaceable, like cogs in a machine (i.e. Taylorism), they have little if any leverage in market negotiations and there is no or little handover costs in case a worker needs to be replaced. Furthermore simple problems typically don’t require much teamwork so building personal relationships doesn’t matter much.
Managers on the other hand are highly educated and most of the company’s success depends on their capability to plan accurately and then control the behaviour of the workforce.
The company perceives managers as more valuable than workers. Managers also perceive themselves as more valuable. Progress in career is measured by progressing upwards in the hierarchy. There cannot be other measures of career progress as the task is simple, the solution is known and the value delivered is almost granted given that market conditions don’t change significantly.

5. Alpha Male Management Style

Successful management behaviour will be a dominant one, based on telling and instructing, as half of the everyday job of managers is forcing people to comply. We’ve seen in industry that many of these organizations also employ tactics like creating fear or lying to the workforce. As much as I hate to say this, for physical labour solving Simple Problems, these tactics work.

How does it feel to work for an organization which solves Simple Problems?

It can feel like being in an army or working for a church: obedience, carrying out orders without question and believing (or at least pretending to believe) what people at the top believe is paramount for survival.

“Working for an organization solving Simple Problems can feel like being in an army or working for a church.”

Exercise

Let us stop here for a second and think a little bit.
How much of the structure, processes, behaviours, characteristics of your organization match the ones described above?
Rate this on a scale from 0 to 100, where 100 means they are identical, 0 means there isn’t a single thing that overlaps.

How does this compare against the extent to which your organization solves Simple Problems?
Again rate this on a scale from 0 to 100. (0-none, 100 – all problems are Simple)

How should I structure my organization to solve Complicated Problems?

What processes, leadership style and company culture do I need to be successful?

The ideal organizational structure to solve Complicated Problems is driven by two fundamental needs:

1. Planning is still the right strategy to solve the problem so we will need some hierarchy and a willingness to follow and execute plans.
2. We expect to discover either some of the requirements or some of the activities to fulfil those requirements as we execute our plan or before we start (and we know which ones). Consequently we will need an organization which is capable of learning, analyzing and then feeding back new information to management quickly. This creates the need for a creative, self-motivated and proactive workforce and a partnership between management and the workforce. We will also need a management that is capable of reacting to new information quickly.

Most areas of normal office work fall into this category.

These organizations need a different management style

Organizations successful in solving Complicated Problems still have managers but this management is not based on Taylorism.

Managers are still responsible for overall delivery but are not capable of making all the decisions themselves anymore due to the increased complexity of the underlying problem.

They need information. They are not the only experts anymore. People specializing in certain areas, consultants, change agents, analysts, architects, etc. are all contributing to decision making. The doers also need to feed back findings and learnings frequently. Some of the decisions may even be made by voting or consensus.

Flat hierarchy, partially decentralized decision making and strong everyday collaboration

These organizations are characterized either by relatively flat hierarchies, partially decentralized decision making and a strong everyday collaboration between managers, specialists, consultants and those executing the plan (workers).
Occasionally you will find that some specialists or consultants get paid more than the highest ranking managers as they improve the learning and adaptability of the organization that much.

Lean starts to come into play here: we don’t separate decision making from execution anymore as trying to do so would hinder our efforts to adapt our plans when risks materialize, new information surfaces or analytical activities are completed.

Top 5 things organizations solving Complicated Problems value:

1. Compliance and learning: follow the plan, processes and use predetermined tools but raise issues immediately, deviate from plan if necessary

Following the plan is still the primary value but the output is not fully known and the way to get there may also have gaps in it. The capability to explore, discover and analyze missing details and to learn from experience is crucial.
An environment which actively encourages and rewards learning and feedback is necessary.

2. Some customer collaboration but mostly making early commitments and then delivering them on time and on budget

Most of the requirements and the way to fulfil them are known up front. When there is uncertainty, it is known what needs to be investigated and this is typically done as a front-loaded activity. Customer collaboration is becoming increasingly important but the emphasis is still on making early commitments, creating and following plans.

Some corrections might be necessary to those early commitments but with the right techniques (e.g. buffering, collaboration, risk management, etc.) it is possible to deliver on time and on budget.

Success depends on how accurately we can plan, how fast we can learn and provide feedback, and how much we can execute a plan or follow a policy.

Plans are created in collaboration between managers and doers. Many decisions are delegated to doers (make decisions at the closest point to the work).

3. Light-weight standardization, the emphasis is on transparency, learning, feedback, and adaptation

All the headaches and uncertainty come from the known unknowns. The capability to learn these as fast as possible through experimentation, consultants, gathering data, analysing existing data, etc. is what separates this domain from the simple one.

There will be commonalities between activities and projects which are worth sharing but rigid standardization gives place to using good practices, templates, as every situation has lots of moving parts and parameters that make it slightly different than another one.

There are commonalities and good practices but there are no best practices in this domain.

4. Positional Power (Managers) AND Substance Power (Specialists, Doers)

The company still perceives managers as more valuable than doers as ultimately they are responsible for delivery. However, managers are not the only experts anymore. Specialists (e.g. architects) and consultants are highly valued as well. Most of the workforce is usually highly educated, not easily replaceable and highly valued by the company. Replacing doers can be difficult and expensive but not catastrophic so people can have an increased level of safety allowing them to be more creative, adaptive and proactive.
Progress in career is measured by progressing upwards in the hierarchy or becoming a highly specialized expert in an area.

5. Collaborative Management Style, based on partnership, valuing merits over obedience

Attitudes valuing position and hierarchy are replaced by attitudes valuing merits and knowledge. Workers feel respected, valued and listened to.
Fear is extremely counter-productive as it reduces the likelihood of people speaking up, sharing relevant information so that corrections to the plan can happen in due course.
Manipulation, a culture of lying or withholding information is counter-productive as well, as things are expected to change and if people constantly tell their managers what they want to hear, then all corrections put into place will be based on a false reality and will lead to failure.

How does it feel to work for an organization which is successful in solving Complicated Problems?

It can feel like working for a small or midsized company.

Learning and merits are important but so is the capability to execute predefined plans and follow instructions.

Exercise

How much of the structure, processes, behaviours and characteristics of your organization match the ones described above?
Rate this on a scale from 0 to 100, where 100 means they are identical, 0 means there isn’t a single thing that overlaps.

How does this compare against the extent to which your organization solves Complicated Problems?
Again, rate this on a scale from 0 to 100.

How should I structure my organization and processes to solve Complex Problems?

Exercise: How many of the details of your job are known to your boss?

This next exercise is useful if you work for an organization solving Complex Problems (e.g. software development, digitalization, data science, biotech, psychotherapy, etc.)

Take a second to think about all the relevant information you encounter during your everyday job.

• Did the customer make an important comment in a usability testing session? Can you see an important detail in the usage of the website you are developing?
• How is the code structured (if you are a coder)? Do parts need refactoring?
• Did you learn about a bug in a technology or tool you are using that is having a significant impact on your work?
• How much do you know about the capabilities of your team mates you work with? Do you know whom can you rely on for certain types of tasks? What about personality traits?
• How frequently do you learn something that is substantial, significant or even crucial from the perspective of the project or product development you are working on? Every week? Every day? Multiple times a day?

Now try to put yourself into the shoes of your boss for a second.

• On a scale from 0 to 100 what percentage of this information does (s)he know or learn herself/himself if you don’t tell her/him?
• What percentage of this information does (s)he learn from you?
• What is your boss’s overall understanding (including things (s)he learns herself and things (s)he can only learn from you)?

Are any of these numbers higher than 20%?
What ratings would you give to your boss’s boss? Take your time with this. Are any of those ratings higher than 5%?

Who has the most detailed, pertinent and current information about the requirements, customers, technology, tools, etc.?
If new requirements surface, existing ones change or we learn more about our capability to deliver, who will see this information first?

For Complex Problems, the answer in the vast majority of cases is those doing the work.
Change is so fast and there is so much information to pay attention to that only those doing the job have a chance of comprehending it and responding to it in due course.

Exercise: How much control do you have over aspects of your work which are critical for success?

Imagine that you could be the CEO of your company for one day. You can change anything about the organization’s structure, processes, policies, tools, technology, etc. There are no limits. Whatever change you enact will remain in place after that one day.

Now list out the top 5 most critical things you would change to improve your chances of being successful in your job. Take a couple of minutes to think this through, don’t rush with the answer.

• Would you give yourself the right to make decisions in an important area where you have all the relevant information but for some reason people who don’t have all the relevant information decide for you?
• Would you get rid of a bureaucratic process that leads to inefficiencies?
• Would you replace that tool or piece of software that is just simply unfit for purpose but has been forced upon you as a result of standardization (it worked fine elsewhere)?
• Would you ban some of the pointless meetings you have with stakeholders who don’t need to be involved and replace them with meetings with people who provide valuable input to your work?

What would you do?

Now that you have the list, rate each and every one of them based on how much say you have in changing these today (so not as the CEO). Rate this on a scale from 0 to 100, where 0 is no influence or control and 100 means you can change it alone, as many times as you want to whatever you want.

What does this tell you about the concept of management in this domain (not any specific person)?

The Taylorist concept of management is replaced by coaching and leadership

The original concept of management was born while people were trying to solve Simple Problems as things rarely (if ever) changed and we needed a group of highly educated experts capable of designing the optimal or right way of doing things. Managers had access to the knowledge, skills and information necessary to design and plan the best actions.

The diametrical opposite is true when we are trying to solve Complex Problems. The concept of management (as a set of behaviours described above) does not exist anymore as it would be extremely harmful to the bottom line. It’s replaced by coaching and coaches.
Coaches work on creating an environment where the workers can be successful; structures, policies, processes and technology is designed and selected together, in a partnership.

Please note that I’m talking about behaviours and concepts, not job titles. I know organizations where somebody has the word ‘manager’ in his job title but acts like a coach not like a Taylorist manager.

Organizations successful in solving Complex Adaptive problems typically have small teams of doers who are responsible for delivery, there is no project or delivery manager, or hierarchy.

Leadership is distributed, dispersed, shared and fluid, coaches ensure an environment conductive for the specific work and decision making is in place.

E.g. in the Scrum framework, it is clearly stated that the Scrum Master is not a delivery manager and he is not responsible for delivery. How could (s)he be when there isn’t a fixed plan up front, most of the information about the requirements and how to fulfil them is yet to be learned and the people doing the job will learn these?
There is no project manager as this role is impossible to do effectively when let’s say 7 developers are constantly and continuously learning new information every minute. How do you create a plan when you don’t even know the detailed activities that should be in that plan?

If somebody forces himself into a project manager position, he will quickly become a bottleneck as all information and most decisions will have to flow through him. This usually, slows down the speed of adaptation to be below the speed necessary for value delivery and project survival.
Apart from adapting slowly the other threat is the lack of effectiveness, i.e. doing/developing the wrong product or features. The assumption is that the project manager will make as good or better decisions as the doers would have had. This is almost never the case when we are dealing with Complex Problems as the PM usually has incomplete, inaccurate, second hand and outdated information while the doers have access to the most current and detailed information.

The key here is that decision making has to be decentralized, decisions have to be made at the closest point to the work as there is so much information arriving at such a high frequency that central decision making is both ineffective and inefficient.
Managers focus mainly on strategic decisions.

What about the Product Owner?

You might have heard about the concept or role of the Product Owner (not to be confused with the Product Manager, they are totally different in practice but similar in name).
The Product Owner is a subject matter expert whom has a thorough understanding of the product/service being created/developed, the market and the customer.
Based on the feedback and the new information that emerges every day he does make decisions about what features to include, which customers to serve, etc. He might have decision making authority but he doesn’t make most of his decisions alone and his role is a doer role (when solving Complex Problems).

He spends his time talking to customers, conducting interviews, analysing the response of the market to the product, working with other doers developing the product, designing or reviewing features and functionality, learning about what needs to be developed and how. He is not a stakeholder manager and not a marketing agent doing road shows, although occasionally he may choose to do these activities if it is not detrimental to his primary job.

Workers will have to develop new skills to solve Complex Problems

Responsibility for delivery and working in a highly complex and volatile environment means that the doers will have to develop a plethora of new skills many of which have been in the domain of management before.

This will require significant investment in time and effort and a good coach and trainer. Without being exhaustive, just a few examples of these skills:

• How to make decisions in teams sometimes under significant uncertainty and time pressure.
• Self-organizing: assigning work to themselves based on a number of factors (priority, who has what skills, etc.) and quickly adjusting this when the environment changes (e.g. something is not a priority anymore or something is more difficult than expected it could use an extra pair of hands, etc.).
• Proactive day-to-day collaboration: interactively exchanging help and resources to get the job done, load balancing, proactively identifying and helping out others whom are stuck or blocked.
• Designing and monitoring metrics to get feedback about the product or aspects of delivery (e.g. product quality, speed of learning, speed of delivery, etc.)
• Breaking down work to small pieces and reducing complexity; designing experiments; recognizing and separating important information, prioritizing.
• Etc.

It’s worth noting that many organizations have the tendency to replace the meaning of the words ‘Leadership’ and ‘Coaching’ so that they are essentially the same as ‘Management’.
It’s not the label that matters but the underlying thinking and the actual everyday behaviours.

Top 5 things organizations solving Complex Adaptive Problems value

1. Individuals and interactions

Multiple highly skilled and often specialized people will need to work together to achieve the desired results. How they interact, exchange information, help and relate to each other and make decisions together is the primary factor of success. Complexity is high and so is task interdependence.
Proactive engagement, self-belief, self-motivation and self-organization are all essential for success.
Great teams are more important than the product itself because only great teams can build great products.

2. Customer collaboration and delivering value to the customer early and frequently

Since most of the requirements are not known up front or are going to change and we don’t even know which one and in what way, the only way to control uncertainty is to constantly and continuously deliver value to the customer. This way if we learn that we want to shutdown the project early we still have delivered permanent value to the customer. Also only by continuously delivering value can we learn more about the requirements and our capability to deliver them.

The metaphor is building a large residential tower floor by floor and fully finishing each floor before proceeding to the next one (water, gas, electricity, flooring, painting, utilities, appliances, etc.), instead of building the concrete frame for all floors first and then doing the plumbing for all floors, then the electricity for all floors, etc. Customers ‘move in’ to this half-finished building while we are building it, they start realizing value and we start learning about customer behaviour, what they are missing. We also learn about the challenges we encounter during building.
I know the metaphor is bad, one would probably never build a building this way but this is exactly the optimal way of building software. The difference is that software is infinitely malleable while concrete is not. So it’s easy to adapt the end product to the new discoveries we’ve made.

Success depends on how fast we can learn about the requirements and about how we will deliver those requirements (e.g. limitations and peculiarities of some technologies or a tailor made delivery process which is optimal for our current situation).

3. Adaptation (based on transparency, learning and feedback)

Standardization across projects is going to be harmful in the vast majority of cases. All situations are unique and complex. There are no best practices and there are no good practices either.
Emergent practices reveal themselves as the result of experimentation, frequent end-to-end value delivery and customer feedback.
There is great emphasis on stopping frequently and reflecting on requirements, customers’ feedback, technology, processes, tools, etc. How is this working for us?

Everything is up for change at any moment, nothing is sacred. The team that adapts the fastest and delivers the most value early and frequently is the team that wins the game.
Progress is not measured by how much we follow a plan as plans are high level and outcome focused, they only contain detailed activities for the next few days.

4. Egalitarian attitude instead of a hierarchical one

Positional power gives space to substance power. In an environment like this, it is counterproductive to differentiate between ideas or feedback based on the position of the messenger. Successful organizations adopt an egalitarian attitude and mindset. The importance of position and hierarchy dissipates.

Psychological Safety is also paramount to reach high-performance as all progress depends on creative, proactive contribution and discovery of new information which requires a fearless organization.

These companies have a Scandinavian style leadership and attitude.

5. Servant Leadership and coaching instead of classic Management

The role of the manager disappears and gives way to the leader or coach: a person who focuses on creating the environment where the team can make the optimal decisions and adapt as fast as possible but (s)he doesn’t get involved in the decisions themselves and (s)he doesn’t direct or control people.
E.g. ensuring that time is allocated to reflect on ways of working is a duty a leader has (creating the right environment). He might also facilitate the conversation between team members using techniques like the Mutual Learning framework but he would not dictate that team members use a certain technology or tool. Of course, if the Leader himself has substantial subject matter expertise in an area, then (s)he would be expected to share that just as anybody else.

In Silicon Valley many venture capitalists focus on investing in software development companies which have a high proportion of software engineers compared to the number of ‘managers’ or ‘business people’ in the company as this has proven to be a good predictor of success in software development (which frequently deals with extremely Complex Problems).
The most valued person is the person doing the job. Doers are extremely difficult to replace due to a number of factors:

• They are highly skilled and specialized (scarcity on the market) and usually have a deep understanding of the requirements and/or the way to fulfil those (e.g. understanding the existing codebase in software development). Handover costs typically range from a few weeks to several months.
• Doers are closest to the details and many of the details are only known by them, losing a person could mean losing critical information either permanently or for a long time until it’s rediscovered through experiments or re-learned from documentation by somebody else.
• The knowledge they create and the dynamics they develop when they interact with each other, learn each others’ skills, expectations and the best way to interact with each other. There is a saying in team coaching: if you add or remove a single member from a team you have a different team now. The team dynamics reset and restart.
• They are responsible for delivery.

Progress in career is measured by how much value your products deliver and how much you learn and hone your individual skills.

How does it feel to work for an organization which is successful in solving Complex Problems?

It can feel like working with friends or hanging out with classmates at university.

Everybody is equal, people listen to each other with curiosity and speak up with honesty and courage when they see something important; they connect to each other and know each others’ strengths, weaknesses, expertise, thoughts and feelings.

There is little or no back stabbing in organizations like this, partially due to the close and unique connection developed between team members and partially due to the lack of incentives:

There is no hierarchy so progress cannot be made by backstabbing someone.

Exercise

How much of the structure, processes, behaviours and characteristics of your organization match the ones described above?
Rate this on a scale from 0 to 100, where 100 means they are identical, 0 means there isn’t a single thing that overlaps.

How does this compare against the extent to which your organization solves Complex Problems? Again rate this on a scale from 0 to 100.

The Scale from Complicated to Complex Problems

It’s worth noting that although the categorization of problems into ‘Complex Adaptive’ is useful, in practice problems sit on a scale between Complicated and Complex. There are more complex and less complex problems based on how much percentage of the requirements and the way to deliver them is known up front and how much percentage is changing.
E.g. in an average software development around 75% of the requirements are not known up-front or are going to change. In an average research-based data science project around 95% of the requirements or the way we will deliver them are not known up front; usually only a high-level end goal is known.

The higher the complexity of the problem, the more it needs the structure, processes and Values described above. The lower the complexity and the closer it is to Complicated Problems, the closer the structure can be to that described under the Complicated Problems section.

How should I structure my organization and processes to solve Chaotic Problems?

The structure and concepts are the same as for Complex Adaptive problems, except that this organization is even more fluid and is willing to temporarily become extremely hierarchical if that’s what necessary to execute the action that stabilizes the situation. The emphasis is on acting quickly, trying out multiple things and then doing more of those which stabilize the situation.

The biggest risk of solving Chaotic Problems

The risk here is that the organization will be never able to exit this emergency state of hierarchy even after the emergency has been resolved. Good example is Caesar and the Roman Empire, he never abdicated his total power after the emergency had gone away.

While it’s futile trying to control a Chaotic Problem and the outcome of it, it’s paramount to have checks and balances in place to control how the organization will function after the emergency has been resolved.

E.g. time limit the authority of anybody elected to a decision making position.

Without these checks the Chaotic Problem will be solved but the organization’s maturity level and capability to solve Complex Problems will be severely reduced; it will go back to solving Simple Problems only.
We can observe this process as authoritarian regimes gradually become less and less adaptive, less and less capable and performing over time.

Top 5 things organizations solving Chaotic Problems value

1. Individuals (and their judgement calls) and increased interactions
2. Acting and moving toward stability
3. Adaptation: learning quickly what actions stabilize the situation (feedback and transparency is vital)
4. Fluid swapping between egalitarian and hierarchical attitudes, decentralized and centralized decision making as necessary
5. Checks and balances to automatically return to a previous state after the Chaotic Problem has been solved, without further effort

Exercise

How much of the structure, processes, behaviours and characteristics of your organization match the ones described above? Rate this on a scale from 0 to 100, where 100 means they are identical, 0 means there isn’t a single thing that overlaps.
How does this compare against the extent to which your organization solves Chaotic Problems?
Do you do a lot of fire fighting?

Why is it so hard to change organizational design?

Unfortunately most of today’s organizations still think they operate in the domain of solving Simple Problems and they employ concepts like management as if preparing a presentation, repairing a car, developing software or cutting out pieces from wood all had the same intrinsic complexity.

There is nothing wrong with solving any type of problem. Using organizational designs which were not intended for the type of problem at hand is not useful though.

Evolving an organizational design that works in a specific context is hard because we’ve been doing something in one specific way for so long that it has become part of our identity. Changing it feels like killing ourselves. It is, of course, not. We are not our ego characters and the stories we tell about ourselves but it can feel like we are.
This can make us incapable of making the decisions that work for us.
We can get blocked by the fear of loss and the fear of failure. Loss of status, loss of power, loss of control. We might even know intellectually that these fears are not helpful but knowing doesn’t help, we fight relentlessly to avoid these fears. And the cost is huge.
We can also get lost in judgements, evaluations and what the ‘right way’ of doing things is, instead of focusing on what our goal is and what works.

When things go south, we rationalize, procrastinate and desperately search for reasons why the decisions we made about organizational design were the right one.
And sadly we succeed in doing this because the human mind can justify any relationship between any two things. We start avoiding reality, decreasing transparency and eliminating or sidelining voices that don’t say what we want to hear. Doing what works gives place to everyday convenience.

We fight to avoid facing the extremely painful truth: we have a choice.

We can all make different decisions, starting now, and nothing external or internal has to change for this.

The cost of avoiding contact with reality and not moving toward an organizational design that works for us is high. Reduced individual well-being, hollow, shallow, fake or damaged working relationships, loss of profit or business, products or features no one uses and decreased competitiveness.

Summary: when to employ Agile and when to employ Waterfall

Simple and Complicated Problems require quite different strategies than Complex Problems, I think there is benefit in summarizing this.

	Simple & Complicated Problems	Complex (Adaptive) Problems
Requirements and how to deliver a solution	Mostly known up-front, it’s clear what’s missing.	Most of it is unknown up-front or are going to change
Dealing with uncertainty	Planning (enumerating, elaborating and sequencing activities, comparing execution against expectations)	Emergence and Evolution (continuous end-to-end value delivery in short cycles (days) to learn about the requirements and how to deliver them.
Values	Making Early Commitments	Customer Collaboration
	Delivering on time and on budget	Early and frequent value delivery
	Getting it right the first time, compliance	Adaptability, responding to change
	Plans, processes, policies, tools	Individuals and interactions (as the right process and plan is not known)
Assumptions	The customer knows what she wants	The customer doesn’t know what she wants but she knows it when she sees it
	Customer changes her mind infrequently and only in early stages of the product lifecycle	Customer changes her mind frequently and even in late stages of the product lifecycle
Who is responsible for delivery?	Managers	Doers
Dominating deliver method	Waterfall and classic Project Management (e.g. PRINCE2); Lean in some contexts	Lean and Agile

Do you need help? Get in touch.

I’m using professional coaching and evidence based methods from the latest research within psychology to help my clients with the behaviour change they need to achieve their goals so that leaders and organizations can identify and grow the optimal organizational design and make the hard decisions which work in their context.
If you need help with any of these, please don’t hesitate to contact me, I’m here to help: Contact Me.

Also please subscribe if you liked this article and share it on social media: