We should all embrace ‘Embracing Complexity’

Occasionally you come across a book which says all that you wanted to, only better, and with more proof and analysis than you could hope to assemble yourself. Well, Embracing Complexity is that book for me. It brilliantly describes the technical background to complexity, then argues forcefully that we should adopt a complexity worldview. In this review I’ll summarise the writers’ key points, and pull out some implications of what I currently think of as a modern masterpiece.

 

 

Twenty years ago I was studying to be a physicist. I soon realised I didn’t have the deep maths to do anything valuable in the field, and so I moved on. Back then I had read about complexity. James Gleick’s book on Chaos had popularised many elements. Once I joined Forum I and a few colleague tried to bring complexity thinking into our work, but without great success. We knew we wanted to get beyond a mechanical approach to change but it wasn’t clear what complexity was, especially when compared to systems thinking, nor how to put the principles into action.

Enter Embracing Complexity: Strategic Perspectives for an Age of Turbulence by Jean G. Boulton, Peter M. Allen, Cliff Bowman. As it happens, I know one of the authors, Jean, and was at the book launch back in September 2015. But when I read the book it really resonated with my past reading and current experiences of trying to create profound change. The effectively has a series of big claims, and then applies those claims to the domains of management, strategy, economics and, politics.

Let’s start with the big claims. The first, and the foundation stone for the rest of their thesis, is that we can treat complexity as a worldview, one which “sees the world as essentially interconnected, and rich with forms and patterns that have been shaped by history and context…[it] reminds us of the limits of certainty, it emphasises that things are in a constant process of ‘becoming’ and that there is potential for startlingly new futures where what emerges can be unexpected and astonishing”.

The primary contrast, of course, is with the mechanistic worldview we have inherited from Newton, that the world is just a clock that will predictably tick round. The other contrast is with a worldview inherited from a later physics, that of thermodynamics. Thermodynamics says the best way to understand a liquid or a gas is by treating the very large number of molecules as independent agents. Predicting an individual agent is hard, but the sheer statistics of their bouncing off each other like billiards balls, mean you can predict how the gas or liquid will move towards equilibrium. (In this sense, thermodynamics is Newton plus statistics, and is a subset of mechanistic worldview.) Beinhocker shows in The Origin of Wealth it is thermodynamics that inspired the orthodox economics that we know.

What do the authors mean by a complexity worldview? In a nutshell, they are saying the world is:

• Systemic: the world cannot be understood through taking apart he bits and understanding them separately. Factors work together synergistically, that is, the whole is different from the sum of the parts. We live as part of patterns of relationships.
• Path-dependent: history matters and the sequence of events is a key factor in giving shape to the future.
• Sensitive to context: one size does not fit all, and the way change happens and the way the future emerges is dependent on the detail and particular events and patterns of relationships and particular features in the local situation. By generalising, we risk throwing out the very information that sheds light on why things happen and what might happen next.
• Emergent, uncertain, but not random: although the future does not follow smoothly from the past, neither is what happens random. The world is neither chaotic not predictable but somewhere in between.
• Episodic: things are becoming, developing, and changing but change happens in fits and starts. The intriguing thing about the world is that on the surface patterns of relationships and structures can seem almost stable for long periods of time, although micro-changes may be going on under the surface. And then radical change can happen suddenly and new patterns of relationships can self-organise and some completely new features that could not have been predicted may emerge.”

 

Individually and collectively, these aspects are strongly at odds with the mechanistic worldview, where the whole is the sum of the parts, where there is no past contained in a situation, just the current dynamics, where the same starting point should merely wind forward again and again. The authors claim that their complexity worldview is many ancients had already intuited and what Darwin recovered when he wrote about evolution. Their main descriptions of complexity rely more on ecology than on physics, perhaps because the maths is just too hard to translate into English.

Here is their central tenet of complexity theory:

“it is in the detail and variation, coupled with interconnection, that provide the fuel for innovation, evolution, change, and learning.”

Any proposed worldview has to do a few things. First, how is it different and bette, than others? As well as the contrast to the mechanistic worldview, the authors contrast with systems thinking:

“systems thinking deals with stable patterns and history deals with the particularity of events, conditions and individuals – but complexity thinking marries the two and provides us with a sophisticated and unique theory of change.”

Which leads to the key weakness of systems thinking: “simple systems analysis are helpful in describing stability, they tells little about change – what may cause it, and what may emerge as a result”.

The other thing a worldview must do – especially one that claims to ‘include and transcend’ the previous ones – is explain how the others can give useful explanations. If the mechanistic worldview is built from the wrong foundations, why hasn’t it simply fallen over? Why can it be useful? Here is their answer:

“When things are very stable over a long time, the macro-characteristics of complex systems tend towards behaviour that looks machine-like and predictable”

 

Why does this matter for sustainability? Well, if you take the mechanistic worldview then we can solve the sustainability question one issue at a time. You don’t need to look for a deep layering of causes behind, say, climate change. And the way to solve problems is simple. You just reduce your emissions and move on to the next problem. Any attempt at change which has a list of things to do, in order, that can be written in advance and then never changed is subscribing to this worldview (though they may not know it). In a mechanistic worldview holistic is so much horseshit.

But people know intuitively that “sustainability is a property of a web of relationships means that in order to understand it properly we need to shift our focus to the whole, and learn to how to think in terms of relationships, in terms of interconnections, patterns and context” (Boulton et al quoting Evitts, S., Seale, B., Skybrook, D. (2010)). Developing an Interconnected Worldview: A. Guiding Process for Learning). That cannot be done within a mechanistic worldview. It needs something more.

What Boulton et al is that something more, the worldview through which we can address the sustainability crisis of our times.

 

The complexity worldview gives us two new lenses onto change.

First, it allows us to understand change as a cycle of an ecology. Let’s imagine there is an absence of order. From this period of chaos there is self-organisation, patterns of relationships between the different parts that emerge and sustain. These parts and their relationships start to co-evolve, that is changes in one induce in and are the response to changes in another. Over time the web of web of relationships becomes mutually sustaining. This is a period of self-regulation where incremental differences lead to incremental changes. Without further external shocks this moves to lock-in, as the  relationships become tighter, more rigid, more efficient. But they are so finely tuned that a shock from outside the immediate system leads to collapse. And we start again.

This cycle is familiar to anyone in an organisation as it starts, finds its niche, grows, standardises, becomes more rigid and then struggles to change until too late. The move to self-regulation is ‘needed’ for efficiency but contains the seeds of the organisation’s failure. My point of view on why there are so many business books on leadership and change is because lots of people find themselves in charge in that ‘self-regulation’ or lock-in phase. They are trying to do something about it, but it is hard, because the organisation is defined now by the self-sustaining, self-reinforcing relationships – by what made it successful until now. It’s why we shouldn’t expect the current incumbents to be the source of fundamental change.

Embracing Complexity says that the lock-in is not inevitable, it happens when the framing context is stable, and so no push to keep self-organising. The suggestion is to keep in changing more than the context might otherwise need you to, so you don’t become fixed. There’s an interesting parallel here with Taleb’s notion of AntiFragile, where things get better when stressed. His point of view is that change and shocks are inevitable, and so better to design situations to benefit from that, rather than merely recover (his take on resilience). Needless to say, any senior manager will tell you how much attention and sheer willpower is required to avoid useful ways of getting things done from becoming standardised routines of thought and deed.

The authors are keen to address one pervasive mis-understanding: self-organising “does not necessarily imply that the situation [is] just left to itself”. Self-organisation occurs when there are top-down intentions, as well as bottom-up activity. As such, enabling self-organisation is not just a case of senior managers or government getting out of the way.

Second there is describing change. This is a bit technical, but very much worth it. There are some definitions that we need:

• “State space – a representation of the information in a situation or system through looking at the set of its constituent dimensions or variables rather than representing it in time and space. Contains a map of attractor basins and areas where there are less definite features.”
• “Attractor basins (or attractors) – Stable configurations of interacting variables which show what combinations of factors mutually reinforce each other.”

These two terms help us unpack three types of change (examples are mine, which means they might be wrong): within attractor basins, between existing ones and evolution of new ones.

1. “Unfolding change” staying in the same attractor basin. Through the 90s Tescos simply beat Sainsburys in being an out-of town supermarket, with a mix of brands, a mix of price points and a great variety of choice.
2. “Self-organisation – in to a new attractor basin. giving different configuration of relationships but no new dimensions.” Lidl and Aldi have a different formula compared to the existing grocers: low cost, all own-brand, less variety.
3. “Evolution (or emergence) – the process of variation, self-organisation and selection that leads to emergence of new qualities and types.” On-line retailing is a massive disturbance, adding new factors to include in what makes you successful (i.e. new dimensions in state space), which means that new configurations of mutually reinforcing factors (i.e. new attractor basins) are there to find. These new attractor basins will also affect the supply chains and the customers, who will all co-evolve as the basins do.

Another way in to this (again, my example) would be Brian Eno. In the early 70s he started treating the studio as a musical instrument, not just a passive way of recording other instruments. This change the number of dimensions in the state space: different sounds were available, as were different compositional structures and very different results. I’m listening to one of those results as I type, Music for Airports. This is widely credited with forming a new genre – ambient. In the terms above, in the new, enlarged state space there are new configurations that work – new attractor basins. He found one of those with this album, and then others followed, further developing (i.e. self-organising) the genre.

 

What are the implications?

The authors have a list of principles for how project mangers should behave, given this worldview. But I think it could apply to anyone:

• Aspire
  • Take a wide, systemic view of emerging trends
  • Articulate long-term goals and intentions and use these to prioritise actions, and determine which opportunities to seize
  • Judge whether to persist when there is little evidence of change
• Anticipate
  • Think a few steps ahead during implementation. What might be the consequences of actions…or events in the wider context?
  • Thank through where critical junctures / possible tipping points might occur
  • Scan for changes in the context
• Adapt
  • Adapt to unexpected events and unintended outcomes as they happen
  • Seize opportunities, making choices in line with the long-term goals
  • Plan activities and projects but review progress regularly, take note of changing circumstances and modify if necessary
• Customise
  • Take account of contextual and historical factors in developing plans and programmes. Do not expect there to be a universally applicable best approach
  • Experiement – plito approaches, learn from what works.

 

When it comes to organisational strategy, the fundamental insight is to remain ambidextrous: “An organisation needs both to be effective in competing in today’s market with today’s economic and social and political conditions, but also to have the ability to make judgements about the future in order to be able to adapt to any changes as well as to, in some cases, catalyse change.”

But attending to both today and tomorrow is tough. Their conclusion is the importance of constantly testing how dynamic your contest tis. If it is stable, then you can bias toward effectiveness (mostly today, and a little of considering tomorrow). If it is fluctuating, then develop a core competence of entrepreneurialism, anticipating future trends and tastes. If unstable then build for emergence.

For all these, it is crucial to keep experimenting, seizing opportunities, learning and adapting to circumstances — it’s just a question of degree.

The implications for economics get the most hefty treatment from the authors. Neoclassical thinking is attacked for requiring an equilibrium which will never come, for seeing futures as independent of the past, of relying on a super-rational decision-maker (Homo Economics) that is a dangerous fiction, and for separating economics from society, and growth from development.

They stress two things that come from ecology. First, that in nature there is both competition and collaboration:

“Ecological competition is a common feature of the bio-economic relax, but so also is inter-dependency, cooperation, symbiosis and division of labour. Moreover, competition is not the fundamental ‘organising principle’ in the economy of nature, as many theorists have asserted. The touchstone is the problem of earning a living an reproducing – adaptation – and both competition and cooperation are subsidiary phenomena. The are contingent ‘survival strategies’.”

Second, that evolution is “about the survival of the fittest ecology rather than the survival of the fittest individual. The evolutionary process becomes ‘variation—self-organisation—selection’, rather than ‘variation—selection’.”

This has big implication for innovation. We should not look to individual companies for innovation. Instead, innovations come from the competition between, and collaboration within, whole ecosystems of organisations. These innovations are driven by the factors pushing on an ecosystem, whether that is competition from another or an incentive from a customer or a regulation from government or more. Innovation is not manna from heaven, or the result of one lone genius. I’ll be returning to this when I write about Grubb’s Planetary Economics, and when thinking about a pro-sustainability innovation policy.

It also has big implications for competition policy. “One thing Smith seems to have concluded is that this process [of self-organising] works best when the players in the system are of similar size, where there is equal power and diversity of choice, and potential for interconnectivity.” When that is not the case, then we move into a situation where the big can use their dominance to get bigger. If business leaders are told that the purpose of private enterprise is to make profit, then they will use their dominance to remove risk, “to simplify their markets and make suppliers and supply chains and even consumers act in the way that suit them. Indeed, an unregulated market…tends to allow ‘lock-in’ – the big get bigger, and the dominant more dominant.such dominance can ‘win’ in the short-term or even in the medium-term, such organisations are less able to adapt to changing circumstances. They be more efficient but they are less resilient”.

Hence the need for all those books on change, leadership and innovation. Hence, also the case for an active government: “Regulation is needed to counter the tendency for the big to get bigger, the powerful to get more powerful, the risks taken to get more risky, and economic forces to win over concerns for equality and sustainability.”

 

What does all this mean for growing the political economy we need for a sustainable future?

There’s a lot to unpack and explore. Some things that come to mind now, and doubtless will develop and change:

• I’ll try to use ‘state space’ to describe the possible ways of we organise ourselves in society – the interplay of economics, law, politics and more that set the operating context for individuals and organisations, and are expressions of dominant beliefs in society (that is, the political economies).
• The complexity worldview provides a theoretical basis to my experience and intuition that the mechanistic worldview is part of the problem, and needs to be replaced with something that allows for humans acting as part of nature, not separate.
• We need to organise ourselves for evolution, to create a variety of ecologies that can self-organise, then select on the basis of ‘fit’ and then develop further. It all points to the  importance of learning from experiences, plus using and developing what is there already. We need to accept there is no single, perfect answer but there are always unintended consequences.
• That will require removing the existing lock-in – which crowds-out the evolution in the short-term but makes the chance of collapse higher in the medium-term.
• Part of the existing lock-in is an inner dynamic of ‘big get bigger’ (what in system archetypes is called Success to the successful). The implication is a need for all players to have relatively similar power, rather than a small number who win big.
• One of the dimensions in state space for us to play with is the type of actor that is important in a political economy. Political economy has traditionally considered actors of nation-states and individuals 9then, depending on your flavour, perhaps class). My thinking has been that we’re seeing the emergence of a different sort of political economy, made up of collaborations between non-state actors. Don Tapscott calls these Global Solution Networks, and claims these are new multi-stakeholder models for global cooperation, problem solving and governance. In the language of Embracing Complexity these are ecologies that are evolving, and which then frame the behaviour and incentives for businesses, other organisations and people.  They may be the vehicle to promoting well-intentioned send-organisation, and fulfil the author’s call-to-arms: “We have a responsibility for ‘seeding the system with good ingredients’, for shaping intention, for surfacing values, and for spotting where the powerful are ‘winning’ at the expense of the less powerful — and by so doing locking-in resources and locking-out competition”.
• We’ll have to address the profit motive. I’ve had a belief up until now that what matters is outcomes, not motives. So, if someone made the pro-sustainability investment for commercial reasons or ethical reasons, what matters is they made the investment. Embracing Complexity says that’s not good enough. Jean in particular (in conversation with me) has said that the end can never justify the means as it is the means that are added to the system and we may never reach the end. Putting that into practice means, I think, accepting the starting motive someone has as inevitable, but trying to change that motive – and the beliefs underlying it – over time.

 

I’m sure that list will be longer by the time I finish my sabbatical! There are also two ironies I’m aware of. The first is one core message of Embracing Complexity is to avoid universal claims, while using a complexity as a worldview is to have a universal claim. The other is that the authors warn “we should not assume that the patterns that emerge at a one scale will necessarily be repeated at a larger scale”. Just because science can ‘prove’ the natural world is best understood through complexity, doesn’t mean that automatically it is the best way to understand the human world.

The authors do address this head on in one chapter, and draw the conclusion that although humans are not molecules, the extra features we bring (like intention) can be brought into a complexity worldview. Of course, when considering beliefs it is impossible to stand back and make an objective appraisal. What are you now standing on, from which you compare mechanistic vs complexity worldview? There complexity one has a couple of things in its favour. One is that it can explain why the mechanistic worldview works where it works, and also why it doesn’t in other circumstances. The other is that a ’experiment-and-learn’ approach at complexity worldview’ core means it is not fixed, but constantly being tested and refined. Therefore, it contains within it an inner dynamic to adapt with experience, not passively stay the same.

As such, I want to experiment with it for now. Over the coming weeks I’ll be applying it directly to questions of bending the curve on climate change, a pro-sustainability innovation policy, an enabling narrative for business, the digital revolution – and how we can grow the political economy we need for a sustainable future. It’s going to be fun!