Basic Patterns in How Adaptive Systems Fail

Chapter 10 1

Chapter 10: Basic Patterns in How Adaptive

Systems Fail

David D. Woods and Matthieu Branlat

This chapter provides one input to resilience management

strategies in the form of three basic patterns in how adaptive

systems fail. The three basic patterns are (1) decompensation –

when the system exhausts its capacity to adapt as disturbances /

challenges cascade; (2) working at cross-purposes – when roles

exhibit behaviour that is locally adaptive but globally mal-adaptive;

and (3) getting stuck in outdated behaviours – when the system

over-relies on past successes. Illustrations are drawn from urban

fire-fighting and crisis management. A working organisation needs

to be able to see and avoid or recognise and escape when the

system is moving toward one of the three basic adaptive traps.

Understanding how adaptive systems can fail requires contrasting

diverse perspectives.

The Optimist-Pessimist Divide on Complex Adaptive

Systems

Adaptive System Sciences begin with fundamental trade-offs –

optimality-brittleness, (Csete and Doyle, 2002; Zhou, Carlson and

Doyle, 2005) or efficiency-thoroughness (Hollnagel, 2009). As an entity,

group, system, or organisation attempts to improve its performance it

becomes better adapted to some things, factors, events, disturbances, or

variations in its environment (its ‘fitness’ improves). However, as a

consequence of improving its fitness with respect to some aspects of its

environment, that entity also must become less adapted to other events,

disturbances, or variations. As a result, when those ‘other’ events or

variations occur, the entity in question will be severely tested and may

2 Resilience Engineering in Practice

fail (this dynamic is illustrated by the story of the Columbia space

shuttle accident; e.g., Woods, 2005a).

The driving question becomes whether (and how) an entity can

identify and manage its position in the trade-off space? In other words,

can an organisation monitor its position and trajectory in a trade-off

space and make investments to move its trajectory prior to crisis

events? The pessimists on complexity and adaptive systems (e.g.,

Perrow, 1984) see adaptive systems as trapped in a cycle of expansion,

saturation, and eventual collapse. The pessimist stance answers the

above questions with ‘No.’ Their response means that as a system

adapts to meet pressures to be ‘faster, better, cheaper,’ it will become

more complex and experience the costs associated with increasing

complexity with little recourse.

Resilience Engineering, on the other hand, represents the optimist

stance and its agenda is to develop ways to control or manage a

system’s adaptive capacities based on empirical evidence. Resilience

Engineering maintains that a system can manage brittleness trade-offs.

To achieve such resilient control and management, a system must have

the ability to reflect on how well it is adapted, what it is adapted to, and

what is changing in its environment. Armed with information about

how the system is resilient and brittle and what trends are under way,

managers can make decisions about how to invest resources in targeted

ways to increase resilience (Woods, 2006a; Hollnagel, 2009).

The optimist stance assumes that an adaptive system has some

ability to self-monitor its adaptive capacity (reflective adaptation) and

anticipate/learn so that it can modulate its adaptive capacity to handle

future situations, events, opportunities and disruptions. In other words,

the optimist stance looks at human systems as able to examine, reflect,

anticipate, and learn about its own adaptive capacity.

The pessimist stance, on the other hand, sees an adaptive system as

an automatic built-in process that has very limited ability for learning

and self-management. Systems may vary in how they adapt and how

this produces emergent patterns but the ability to control these cycles is

very limited. It is ironic that the pessimist stance thinks people can

study and learn about human adaptive systems, but that little can be

done to change/design adaptive systems because new complexities and

unintended consequences will sabotage the best laid plans. Resilience

Chapter 10 3

Engineering admits that changing/designing adaptive systems is hard,

but sees it as both necessary and possible. Resilience Engineering in

practice provides guidance on how to begin doing this.

This chapter provides one input to resilience management

strategies in the form of three basic patterns in how adaptive systems

fail. The taxonomy continues the line of work begun by Woods and

Cook (2006) who described one basic pattern in how adaptive systems

behave and how they fail. The chapter also illustrates these patterns in

examples drawn from urban fire-fighting and crisis management. To

develop resilience management strategies, organisations need to be able

to look ahead and either see and avoid or recognise and escape when they are

headed for adaptive traps of one kind or another. A taxonomy of

different maladaptive patterns is valuable input to develop these

strategies.

Assessing Future Resilience from Studying the History

of Adaptation (and Maladaptation)

The resilience/brittleness of a system captures how well it can adapt to

handle events that challenge the boundary conditions for its operation.

Such ‘challenge’ events do occur (1) because plans and procedures have

fundamental limits, (2) because the environment changes over time and

in surprising ways, (3) because the system itself adapts around successes

given changing pressures and expectations for performance. In large

part, the capacity to respond to challenge events resides in the expertise,

strategies, tools, and plans that people in various roles can deploy to

prepare for and respond to specific classes of challenge.

Resilience, as a form of adaptive capacity, is a system’s potential for

adaptive action in the future when information varies, conditions change,

or when new kinds of events occur, any of which challenge the viability

of previous adaptations, models, plans, or assumptions. However, the

data to measure resilience comes from observing/analysing how the

system has adapted to disrupting events and changes in the past (Woods,

2009a, p. 500). Past incidents provide information about how a system

was both brittle, by revealing how it was unable to adapt in a particular

evolving situation, and resilient, by revealing aspects of how it routinely

adapted to disruptions (Woods and Cook, 2006). Analysis of data about

4 Resilience Engineering in Practice

how the system adapted and to what, can provide a characterisation of

how well operational systems are prepared in advance to handle

different kinds of challenge events and surprises (Hollnagel et al., 2006).

Patterns of failure arise due to basic regularities about adaptation in

complex systems. The patterns are generalisations derived from

analysing cases where systems were unable to prepare for and handle

new challenges. The patterns all involve dynamic interactions between

the system in question and the events that occur in its environment.

The patterns also involve interactions among people in different roles

each trying to prepare for and handle the events that occur within the

scope of their roles. The patterns apply to systems across different

scales – individuals, groups, organisations.

Patterns of Maladaptation

There are three basic patterns by which adaptive systems break down,

and within each, there is a variety of sub-patterns. The three basic

patterns are (1) decompensation, (2) working at cross-purposes, and (3)

getting stuck in outdated behaviours.

Decompensation: Exhausting Capacity to Adapt as Disturbances /

Challenges Cascade

In this pattern, breakdown occurs when challenges grow and cascade

faster than responses can be decided on and effectively deployed. A

variety of cases from supervisory control of dynamic processes provide

the archetype for the basic pattern. Decompensation occurs in human

cardiovascular physiology, e.g., the Starling curve in cardiology. When

physicians manage sick hearts they can miss signals that the

cardiovascular system is running out of control capability and fail to

intervene early enough to avoid a physiological crisis (Feltovich, Spiro

and Coulson, 1989; Cook, Woods and McDonald, 1991; Woods and

Cook, 2006). Decompensation also occurs in human supervisory

control of automated systems, for instance in aviation. In cases of

asymmetric lift due to icing or slowly building engine trouble,

automation can silently compensate but only up to a point. Flight crews

may recognise and intervene only when the automation is nearly out of

capacity to respond and when the disturbances have grown much more

Chapter 10 5

severe. At this late stage there is also a risk of a bumpy transfer of

control that exacerbates the control problem. Noticing early that the

automation has to work harder and harder to maintain control is

essential (Norman, 1990; Woods, 1994; Woods and Sarter, 2000

provide examples from cockpit automation). Figure 1 illustrates the

generic signature for decompensation breakdowns.

The basic decompensation pattern evolves across two phases. In

the first phase, a part of the system adapts to compensate for a growing

disturbance. Partially successful initially, this compensatory control

masks the presence and development of the underlying disturbance.

The second phase of a decompensation event occurs because the

automated response cannot compensate for the disturbance completely

or indefinitely. After the response mechanism’s capacity is exhausted,

the controlled parameter suddenly collapses (the decompensation event

that leads to the name).

Figure 1. The basic decompensation signature.

The question is whether a part of the system, a supervisory

controller, can detect the developing problem during the first phase of

6 Resilience Engineering in Practice

the event pattern or whether it misses the signs that the lower order or

base controllers (automated loops in the typical system analysis) are

working harder and harder to compensate but getting nearer to its

capacity limits as the external challenge persists or grows? This requires

discriminating between adaptive behaviour that is part of successful

control and adaptive behaviour that is a sign of incipient failure to

come.

In these situations, the critical information is not the abnormal

process symptoms per se but the increasing force with which they must

be resisted relative to the capabilities of the base control systems. For

example, when a human acts as the base control system, he/she would

as an effective team member communicate to others the fact that they

need to exert unusual control effort (Norman, 1990). Such information

provides a diagnostic cue for the team and is a signal that additional

resources need to be injected to keep the process under control. If

there is no information about how hard the base control system is

working to maintain control in the face of disturbances, it is quite

difficult to recognise the seriousness of the situation during the phase 1

portion, and therefore to respond early enough to avoid the

decompensation collapse that marks phase 2 of the event pattern. The

key information is how hard control systems are working to maintain

control and the trend: are control systems running out of control

capability as disturbances are growing or cascading?

There are a number of variations on the decompensation pattern,

notably:

• Falling behind the tempo of operations (e.g., the aviation expression

‘falling behind the power curve;’ surges in demands in emergency

rooms – Wears and Woods, 2007; bed crunches in intensive care

units – Cook, 2006).

• Inability of an organisation to transition to new modes of functioning when

anomalies challenge normal mechanisms or contingencies (e.g., a hospital’s

ability to manage mass casualty events – see Committee on the

Future of Emergency Care in the US, 2006; Woods and Wreathall,

2008 provide a general description of this risk).

Chapter 10 7

Working at Cross-purposes: Behaviour that is Locally Adaptive, but

Globally Maladaptive

This refers to the inability to coordinate different groups at different

echelons as goals conflict. As a result of miscoordination the groups

work at cross-purposes. Each group works hard to achieve the local

goals defined for their scope of responsibility, but these activities make

it more difficult for other groups to meet the responsibilities of their

roles or undermine the global or long term goals that all groups

recognise to some degree.

The archetype is the tragedy of the commons (Ostrom, 1990, 1999)

which concerns shared physical resources (among the most studied

examples of common pools are fisheries management and water

resources for irrigation). The tragedy of the commons is a name for a

baseline adaptive dynamic whereby the actors, by acting rationally in the

short term to generate a return in a competitive environment, deplete

or destroy the common resource on which they depend in the long run.

In the usual description of the dynamic, participants are trapped in an

adaptive cycle that inexorably overuses the common resource (a

‘pessimist’ stance on adaptive systems); thus, from a larger systems view

the local actions of groups are counter-productive and lead them to

destroy their livelihood or way of life in the long run.

Organisational analyses of accidents like the Columbia space shuttle

accident put production/safety trade-offs in a parallel position to

tragedies of the commons. Despite the organisations’ attempts to

design operations for high safety and the large costs of failures in

money and in lives, line managers under production pressures make

decisions that gradually erode safety margins and thereby undermine

the larger common goal of safety. In other words, safety can be thought

of as an abstract common pool resource analogous to a fishery. Thus,

dilemmas that arise in managing physical common pool resources are a

specific example of a general type of goal conflict where different

groups are differentially responsible for and affected by different sub-

goals, even though there is one or only a couple of commonly held

over-arching goals (Woods et al., 1994, Chapter 4). When the activities

of different groups seem to advance local goals but undermine over-

arching or long term goals of the larger system that the groups belong

8 Resilience Engineering in Practice

to, the system-level pattern is maladaptive as the groups work at cross-

purposes. Specific concrete stories that capture this pattern of adaptive

breakdown can be found in Brown (2005), who collected cases of safety

dilemmas and sacrifice judgments in health care situations.

There is a variety of sub-patterns to working at cross purposes.

Some of these concerns vertical interactions, that is across echelons or

levels of control, such as the tragedy of the commons. Others concern

horizontal interactions when many different groups need to coordinate

their activities in time and space such as in disaster response and

military operations. This pattern can also occur over time. A sub-

pattern that includes a temporal component and is particularly

important in highly coupled systems is missing side effects of change

(Woods and Hollnagel, 2006). This can occur when there is a change

that disrupts plans in progress or when a new event presents new

demands to be handled, among other events. Other characteristic sub-

patterns are:

• Fragmentation over roles (stuck in silos; e.g., precursors to Columbia

space shuttle accident, Woods, 2005a).

• Failure to resynchronise following disruptions (Branlat et al., 2009).

• Double binds (Woods et al., in press).

Getting Stuck in Outdated Behaviours: The World Changes but the

System Remains Stuck in what were Previously Adaptive Strategies

(Over-relying on Past Successes)

This pattern relates to breakdowns in how systems learn. What was

previously adaptive can become rigid at the level of individuals, groups,

or organisations. These behaviours can persist even as information

builds that the world is changing and that the usual

behaviours/processes are not working to produce desired effects or

achieve goals. One example is the description of the cycle of error as

organisations become trapped in narrow interpretations of what led to

an accident (Cook, Woods and Miller, 1998).

This pattern is also at play at more limited operational time scopes.

Domains such as military operations offer a rich environment for

Chapter 10 9

studying the pattern. When conditions of operation change over time,

tactics or strategies need to be updated to match new challenges or

opportunities. While such decisions are made difficult by the uncertain

nature of the operations’ environment and of the outcome of actions,

missed opportunities to re-plan constitute sources of failure (Woods

and Shattuck, 2000). Mishaps in the nuclear industry have also

exemplified the pattern by showing the dangers of “rote rule following”

(ibid.). In all of these cases there was a failure to re-plan when the

conditions experienced fell outside of the boundaries the system and

plans were designed for. Some characteristic sub-patterns are:

• Oversimplifications (Feltovich, Spiro and Coulson, 1997).

• Failing to revise current assessment as new evidence comes in (Woods and

Hollnagel, 2006; Rudolph, 2009).

• Failing to revise plan in progress when disruptions/opportunities arise

(Woods and Hollnagel, 2006).

• Discount discrepant evidence (e.g., precursors to Columbia, Woods,

2005a).

• Literal Mindedness, particularly in automation failures (Woods and

Hollnagel, 2006).

• Distancing through differencing (Cook and Woods, 2006).

• Cook’s Cycle of Error (Cook et al., 1998).

The three basic patterns define kinds of adaptive traps. A reflective

adaptive system should be able to monitor its activities and functions

relative to its changing environment and determine whether it is likely

to fall into one or another of these adaptive traps. The three basic

patterns can be used to understand better how various systems are

vulnerable to failures, such as systems that carry out crisis management,

systems that respond to anomalies in space flights, and systems that

provide critical care to patients in medicine. In the next section, we test

the explanatory value of these three basic patterns by re-visiting a recent

analysis of critical incidents (Branlat et al., 2009) that provided markers

of both resilience and brittleness (Woods and Cook, 2006). Urban fire-

fighting provides a rich setting to examine aspects of resilience and

brittleness related to adaptation and coordination processes. Incident

command especially instantiates patterns generic to adaptive systems

10 Resilience Engineering in Practice

and observed in other domains or at other scales (Bengtsson et al.,

2003; Woods and Wreathall, 2008).

The Basic Patterns Are illustrated in Urban Fire-

fighting Critical Incidents

High uncertainty and potential for disruptions, new events, and

surprises all pose challenges for fire-fighting operations. The fire-

fighting organisation needs to be able to adapt to new information

(whether a challenge or opportunity) about the situation at hand and to

ever-changing conditions. For example, consider this case from the

corpus (Branlat et al., 2009):

Companies arrive on the fire scene and implement standard

operating procedures for an active fire on the first floor of the

building. The first ladder company initiates entry to the apartment

on fire, while the second ladder gets to the second floor in order to

search for potentially trapped victims (the ‘floor above the fire’ is

an acknowledged hazardous position). In the meantime, engine

companies stretch hose-lines but experience various difficulties

delaying their actions, especially because they cannot achieve

optimal positioning of their apparatus on a heavily trafficked street.

While all units are operating, conditions are deteriorating in the

absence of water being provisioned on the fire. The Incident

Commander (IC) transmits an ‘all hands’ signal to the dispatcher,

leading to the immediate assignment of additional companies.

Almost at the same time, members operating above the fire

transmit a ‘URGENT’ message over the radio. Although the IC

tries to establish communication and get more information about

the difficulties encountered, he does not have uncommitted

companies to assist the members. Within less than a minute, a

back-draft-type explosion occurs in the on fire apartment,

engulfing the building’s staircase in flames and intense heat for

several seconds, and erupting through the roof. As the members

operating on the second floor had not been able to get access to

the apartment there due to various difficulties, they lacked both a

Chapter 10 11

refuge area (apartment) and an egress route (staircase). The second

ladder company was directly exposed to life-threatening conditions.

The three basic patterns can all be seen at work in this case:

• Decompensation. The situation deteriorated without companies being

able to address the problem promptly. The Incident Commander

(IC) recognised and signalled an ‘all hands’ situation, in order to

inform dispatchers that all companies were operating and to

promptly request additional resources. As there were no

uncommitted resources available, the fire companies were unable to

respond when an unexpected event occurred (the back-draft) which

created dangers and hindered the ability of others to assist. As a

result, team members were exposed to dangerous conditions.

• Working at cross-purposes. Companies were pursuing their tasks and

experienced various challenges without the knowledge of other

companies’ difficulties. Without this information, actions on the

first floor worked against the actions and safety of operators on the

second floor. Goal conflict arose (1) between the need to provide

access to the fire and to contain it while water management was

difficult, and (2) between the need to address a deteriorating

situation and to rescue injured members while all operators were

committed to their tasks.

• Getting stuck in outdated behaviour. The ladder companies continued to

implement standard procedures that assumed another condition

was met (water availability from the engine companies). They failed

to adapt the normally relevant sequence of activities to fit the

changing particulars of this situation: the first ladder company

gained access to the apartment on fire; but in the absence of water,

the opened door fuelled the fire and allowed flames and heat to

spread to the rest of the building (exacerbating how the fire

conditions were deteriorating). Similarly, the unit operating on the

second floor executed its tasks normally, but the difficulty it

encountered and the deteriorating situation required adaptation of

normal routines to fit the changing risks.

12 Resilience Engineering in Practice

Urban Fire-fighting and the Dynamics of Decompensation

During operations, it is especially important for the Incident

Commander (IC) constantly and correctly to assess progress in terms of

trends in whether the fire is in or out of control. To do this, the IC

monitors (a) the operational environment including the evolution of the

fire and the development of additional demands or threats (e.g.,

structural damages or trapped victims) and (b) the effort companies are

exerting to try to accomplish their tasks as well as their capacity to

respond to additional demands. Based on such assessments, the IC

makes critical decisions related to the management of resources:

redeploying companies in support of a particular task; requesting

additional companies to address fire extensions or need to relieve

members; requesting special units to add particular forms of expertise

to handle unusual situations (e.g., presence of hazardous material).

ICs are particularly attentive to avoid risks of falling behind by

exhausting the system’s capacity to respond to immediate demands as

well as to new demands (Branlat et al., 2009). The ‘all-hands’ signal is a

recognition that the situation is precarious because it is stretched close

to its maximum capacity and that current operations therefore are

vulnerable to any additional demands that may occur. The analysis of

the IC role emphasised anticipating trends or potential trends in

demands relative to how well operations were able to meet those

demands (see also Cook’s analysis of resource crunches in intensive

care units; Cook, 2006). For urban fire-fighting, given crucial time

constraints, resources are likely to be available too late if they are

requested only when the need is definitive. A critical task of the IC

therefore corresponds to the regulation of adaptive capacity by

providing ‘tactical reserves’ (Klaene and Sanders, 2008, p. 127), i.e., an

additional capacity promptly to adapt tactics to changing situations.

Equivalent processes also play out (a) at the echelon of fire-fighters or

fire teams, (b) in terms of the distributed activity (horizontal

interactions) across roles at broader echelons of the emergency

response system, and (c) vertically across echelons where information

about difficulties at one level change decisions and responses at another

echelon.

Chapter 10 13

Urban Fire-fighting and Coordination over Multiple Groups and Goals

Fire-fighting exemplifies situations within which tasks and roles are

highly distributed and interdependent, exposing work systems to the

difficulty of maintaining synchronisation while providing flexibility to

address ever-changing demands. Interdependencies also result from the

fact that companies operate in a shared environment.

Several reports within the corpus described incidents where

companies opened hose-lines and pushed fire and heat in the direction

of others. These situations usually resulted from companies adapting

their plan because of difficulties or opportunities. If the shift in activity

by one group was not followed by a successful resynchronisation, it

created conditions for a coordination breakdown where companies

(and, importantly, the IC) temporarily lost track of each other’s position

and actions. In this context one group could adapt to handle the

conditions they face in ways that inadvertently created or exacerbated

threats for other groups. Another example in the corpus was situations

where companies’ capacity to fulfil their functions were impeded by

actions of others. One groups actions, though locally adaptive relative

to their scope, introduced new constraints which reduced another

company’s ‘margins of manoeuvre’ (Coutarel, Daniellou and Dugué,

2003). This notion refers to the range of behaviours they are able to

deploy in order to fulfil their functions, therefore to their capacity to

adapt a course or plan of action in the face of new challenges. Such

dynamics might directly compromise members’ safety, for example

when the constrained functions were critical to egress route

management. In one case, a company vented a window adjacent to a

fire escape which had the consequence of preventing the members of

another company operating on the floor above from using the fire

escape as a potential egress route, should it have been needed.

Goal conflicts arise when there are trade-offs between achieving

the three fundamental purposes of urban fire-fighting: saving lives,

protecting property and ensuring personnel’s safety. This occurs when,

for example, a fire department forgoes the goal of protecting property

in order to minimise risk to fire-fighters. Incidents in the corpus vividly

illustrate the trade-offs that can arise during operations and require

adaptations to on-going operations. Under limited resources (time,

14 Resilience Engineering in Practice

water, operators), the need to rescue a distressed fire-fighter introduces

a difficult goal conflict between rescue and fire operations. If members

pursue fire operations, the victim risks life-threatening exposure to the

dangerous environment. Yet by abandoning fire operations,

momentarily or partially, team members risk letting the situation

degrade and the situation becomes more difficult and more dangerous

to address. The analysis of the corpus of cases found that adaptations in

such cases were driven by local concerns, e.g., when members

suspended their current operations to assist rescue operations nearby.

The management of goal conflicts is difficult when operations are not

clearly synchronised, since decisions that are only locally adapted risk

further fragmenting operations.

Urban Fire-fighting and the Risk of Getting Stuck in Outdated

Behaviours

As an instance of emergency response, urban fire-fighting is

characterised by the need to make decisions at a high-tempo and under

uncertainty. As fire-fighters discover and assess the problem to be

addressed during the course of operations, replanning is a central

process. It is critical that adaptations to the plan are made when

elements of the situation indicate that previous knowledge (on which

on-going strategy and tactics are based) is outdated. The capacity to

adapt is therefore highly dependent on the capacity correctly to assess

the situation at hand throughout the operations, especially at the level

of the IC. Accident cases show that the capacity of the IC to efficiently

supervise operations and modify the plan in progress is severely

impaired when this person only has limited information about and

understanding of the situation at hand and the level of control on the

fire.

Given the level of uncertainty, this also suggests the need for

response systems to be willing to devote resources to further assess

ambiguous signals, a characteristic of resilient and high-reliability

organisations (Woods, 2006a; Rochlin, 1999). This is nonetheless

challenging in the context of limited resources and high tempo, and

given the potential cost of replanning (risk of fragmenting operations,

cost of redeploying companies, coordination costs).

Chapter 10 15

At a wider temporal and organisational scale, fire departments and

organisations are confronted with the need to learn from situations in

order to increase or maintain operations’ resilience in the face of

evolving threats and demands. The reports analysed resulted from

thorough investigation processes that aimed at understanding limits in

current practices and tools and represented process of learning and

transformation. However, it is limiting to assume that the events that

produce the worst outcomes are also the ones that will produce the

most useful lessons. Instances where challenging and surprising

situations are managed without leading to highly severe outcomes also

reveal interesting and innovative forms of adaptations (Woods and

Cook, 2006). As stated previously, many minor incidents also represent

warning signals about the (in)adequacy of responses to the situations

encountered. They are indicators of the system starting to stretch

before it collapses in the form of a dramatic event (Woods and

Wreathall, 2008). To be resilient, organisations must be willing to

pursue these signals (Woods, 2009a). Unfortunately, selecting the

experiences or events which will prove fruitful to investigate, and

allocating the corresponding resources, is a difficult choice when it has

to be made a priori (Hollnagel, 2007; Dekker, 2008, chapter 3).

Recognising What is Maladaptive Depends on

Perspective Contrasts

The chapter has presented three basic patterns in how adaptive systems

fail. But it is difficult to understand how behaviours of people, groups,

and organisations are adapted to some factors and how those

adaptations are weak or strong, well or poorly adapted. One reason for

this is that what is well-adaptive, under-adaptive, or maladaptive is a

matter of perspective. As a result, labelling a behaviour or process as

maladapted is conditional on specifying a contrast across perspectives.

First, adaptive decision-making exhibits local (though bounded)

rationality (regardless of scale). A human adaptive system uses its

knowledge and the information available from its field of view/focus of

attention to adapt its behaviour (given its scope of autonomy/authority)

in pursuit of its goals. As a result, adaptive behaviour is well-adapted

16 Resilience Engineering in Practice

when examined locally, even though the system can learn and change to

become better adapted in the future (shifting temporal perspective).

Second, adaptive decision-making exists in a co-adaptive web

where adaptive behaviour by other systems horizontally or vertically (at

different echelons) influences (releases or constrains) the behaviour of

the system of interest. Behaviour that is adaptive for one unit or system

can produce constraints that lead to maladaptive behaviour in other

systems or can combine to produce emergent behaviour that is

maladaptive relative to criteria defined by a different perspective.

Working at cross-purposes happens when interdependent systems

do things that are all locally adaptive (relative to the role/goals set

up/pressured for each unit) but more globally maladaptive (relative to

broader perspectives and goals). This can occur horizontally across

units working at the same level as in urban fire-fighting (Branlat et al.,

2009). It can occur upward, vertically, where local adaptation at the

sharp end of a system is maladaptive when examined from a more

regional perspective that encompasses higher level or total system goals.

One example is ad hoc plan adaptation in the face of an impasse to a

plan in progress; in this case the adaptation works around the impasse

but fails to do so in a way that takes into account all of the relevant

constraints as defined from a broader perspective on goals (Woods and

Shattuck, 2000).

Working at cross-purposes can occur downward vertically too

(Woods et al., in press). Behaviour that is adaptive when considered

regionally can be seen as maladaptive when examined locally as the

regional actions undermine or create complexities that make it harder

for the sharp end to meet the real demands of situations (for example,

actions at a regional level can introduce complexities that force sharp

end operations to develop work-arounds and other forms of gap-filling

adaptations).

This discussion points to the finding in adaptive system science

that all systems face fundamental trade-offs. In particular, becoming

more optimal with respect to some aspects of the environment

inevitably leads that system to be less adapted to other aspects of the

environment (Doyle, 2000; Zhou et al., 2005; Woods, 2006a; Hollnagel,

2009). This leads us to a non-intuitive but fundamental conclusion that

all adaptive systems simultaneously are (Woods, 2009b):

Chapter 10 17

• well-adapted to some aspects of its environment (e.g., the fluency

law—‘well”-adapted’ cognitive work occurs with a facility that

belies the difficulty of the demands resolved and the dilemmas

balanced; see Woods and Hollnagel, 2006),

• under-adapted in that the system has some degree of drive to learn

and improve its fitness relative to variation in its environment. This

is related in both intrinsic properties of that agent or system and to

the external pressures the system faces from stakeholders.

• maladapted or brittle in the face of events and changes that challenge

its normal function.

This basic property of adaptive systems means that all forms of

linear causal analysis are inadequate for modelling and predicting the

behaviour of such systems. Adaptive systems’ sciences are developing

the new tools needed to accurately model, explain and predict how

adaptive systems will behave (e.g., Alderson and Doyle, in press), for

example, how to anticipate tipping points in complex systems (Scheffer

et al., 2009).

Working organisations need to be able to see and avoid or

recognise and escape when a system is moving toward one of the three

basic adaptive traps. Being resilient means the organisation can monitor

how it is working relative to changing demands and adapt in

anticipation of crunches, just as incident command should be able to do

in urban fire-fighting. Organisations can look at how they have adapted

to disruptions in past situations to estimate whether their system’s

‘margins of manoeuvre’ in the future are expanding or contracting.

Resilience Engineering is beginning to provide the tools to do this even

as more sophisticated general models of adaptive systems are being

developed.

18 Resilience Engineering in Practice

References

Andersson, K. P. and Ostrom, E. (2008). Analyzing decentralized

resource regimes form a polycentric perspective. Policy Science,

41, 71-93.

Alderson, D. L. and Doyle, J. C. (in press). Contrasting views of

complexity and their implications for network-centric

infrastructures. IEEE Systems, Man and Cybernetics, Part A.

Bengtsson, J., Angelstam, P., Elmqvist, T., Emanuelsson, U., Folke, C.,

Ihse, M., Moberg, F. and Nyström, M. (2003). Reserves, Resilience

and Dynamic Landscapes. Ambio, 32(6), 389-396.

Branlat, M., Fern, L., Voshell, M. and Trent, S. (2009). Coordination in

Urban Firefighting: A Study of Critical Incident Reports. Proceedings

of the Human Factors and Ergonomics Society 53rd Annual Meeting, San

Antonio, TX.

Brown, J. P. (2005). Key themes in healthcare safety dilemmas. In M. S.

Patankar, J. P. Brown, & M. D. Treadwell (Eds.), Safety Ethics: Cases

from Aviation, Healthcare, and Occupational and Environmental Health

(pp. 103-148). Adelshot, UK: Ashgate.

Committee on the Future of Emergency Care in the US (2006).

Hospital-based Emergency Care: At the Breaking Point. National

Academic Press, Washington, DC.

Cook, R. I. (2006). Being bumpable: consequences of resource

saturation and near-saturation for cognitive demands on ICU

practitioners. In D. D. Woods & E. Hollnagel (Eds.), Joint Cognitive

Systems: Patterns in Cognitive Systems Engineering. (pp. 23–35). Boca

Raton, FL: Taylor & Francis/CRC Press.

Cook, R. and Rasmussen, J. (2005). “Going Solid”: A model of system

dynamics and consequences for patient safety. Quality and Safety in

Health Care, 14, 130-134.

Cook, R. I., Woods, D. D. and McDonald, J.S. (1991). Human

Performance in Anesthesia: A Corpus of Cases. Cognitive Systems

Engineering Laboratory Report, prepared for Anesthesia Patient

Safety Foundation, April 1991.

Chapter 10 19

Cook, R. I., Woods, D. D. and Miller, C. (1998). A Tale of Two Stories:

Contrasting Views of Patient Safety. Chicago, National Patient Safety

Foundation. (available at http://csel.eng.ohio-

state.edu/blog/woods/archives/000030.html )

Coutarel, F., Daniellou, F., & Dugué, B. (2003). Interroger

l'organisation du travail au regard des marges de manoeuvre en

conception et en fonctionnement [Examining Work Organization

in Relation to Margins of Maneuver in Design and in Operation].

Pistes, 5(2).

Csete, M.E. and Doyle, J.C. (2002). Reverse engineering of biological

complexity. Science, 295, 1664–1669.

Dekker, S. (2008). Just Culture: Balancing Safety and Accountability.

Adelshot, UK: Ashgate.

Doyle, J.C. (2000). Multiscale networking, robustness, and rigor. In T.

Samad and J. Weyrauch (Eds.), Automation, control, and complexity : an

integrated approach. NY: John Wiley & Sons, Inc. New York, pp. 287

– 301.

Feltovich, P. J., Spiro, R. J. and Coulson, R. L. (1989). The nature of

conceptual understanding in biomedicine: The deep structure of

complex ideas and the development of misconceptions. In D.

Evans and V. Patel (Eds.), The Cognitive Sciences in Medicine (pp. 113-

172). Cambridge MA: MIT Press.

Feltovich, P. J., Spiro, R. J., & Coulson, R. L. (1997). Issues of expert

flexibility in contexts characterized by complexity and change. In P.

J. Feltovich, K. M. Ford, & R. R. Hoffman (Eds.), Expertise in

context: Human and machine. Menlo Park, CA. AAAI/MIT Press.

Hollnagel, E. (2007). Resilience Engineering: Why, What and How. In

NoFS 2007 - Nordic Research Conference on Safety, 13-15 June 2007,

Tampere, Finland.

Hollnagel, E. (2009). The ETTO Principle: Efficiency-Thoroughness Trade-Off:

Why Things That Go Right Sometimes Go Wrong. Ashgate.

Klaene, B. J., & Sanders, R. E. (2008). Structural Firefighting: Strategies and

Tactics (2nd ed.). Sudbury, MA: Jones & Bartlett Publishers.

Ostrom, E. (1990). Governing the Commons: The Evolution of Institutions for

Collective Action. New York: Cambridge University Press, 1990.

Ostrom, E. (1999). Coping with Tragedies of the Commons. Annual

Review of Political Science, 2, pp. 493—535.

20 Resilience Engineering in Practice

Perrow, C. (1984). Normal Accidents: Living with High-Risk Technologies.

New York: Basic Books.

Rochlin, G.I. (1999). Safe operation as a social construct. Ergonomics,

42(11), 1549-1560.

Scheffer, M., Bascompte, J., Brock, W. A., Brovkin, V., Carpenter, S. R.,

Dakos, V., Held, H., van Nes, E. H., Rietkerk, M. and Sugihara, G.

(2009). Early-warning signals for critical transitions. Nature,

461(7260), 53-59.

Wears, R. L. and Woods, D. D. (2007). Always Adapting. Annals of

Emergency Medicine, 50(5), 517-519.

Woods, D. D. (2005). Creating Foresight: Lessons for Resilience from

Columbia. In W. H. Starbuck and M. Farjoun (eds.), Organization at

the Limit: NASA and the Columbia Disaster. Malden, MA: Blackwell,

pp. 289--308.

Woods, D. D. (2006). Essential characteristics of resilience. In E.

Hollnagel, D. D. Woods, & N. Leveson (Eds.), Resilience Engineering:

Concepts And Precepts (pp. 19–30). Adelshot, UK: Ashgate.

Woods, D. D. (2009a). Escaping Failures of Foresight. Safety Science,

47(4), 498-501.

Woods, D. D. (2009b). Fundamentals to Engineer Resilient Systems:

How Human Adaptive Systems Fail and the Quest for Polycentric

Control Architectures. Keynote presentation, 2nd International

Symposium on Resilient Control Systems, Idaho Falls, ID, August 11-13

2009 (https://secure.inl.gov/isrcs2009/default.aspx accessed

September 8, 2009).

Woods, D. D. and Cook, R. I. (2006). Incidents: Are they markers of

resilience or brittleness? In E. Hollnagel, D.D. Woods and N.

Leveson, eds., Resilience Engineering: Concepts and Precepts. Ashgate,

Aldershot, UK, pp. 69-76.

Woods, D. D., & Hollnagel, E. (2006). Joint Cognitive Systems: Patterns in

Cognitive Systems Engineering. Boca Raton, FL: Taylor & Francis/CRC

Press.

Woods, D. D. and Sarter, N. (2000). Learning from Automation

Surprises and Going Sour Accidents. In N. Sarter and R. Amalberti

(Eds.), Cognitive Engineering in the Aviation Domain, Erlbaum,

Hillsdale NJ, pp. 327-354.

Chapter 10 21

Woods, D.D. and Shattuck, L. G. (2000). Distant supervision—local

action given the potential for surprise. Cognition, Technology and

Work, 2, 242—245.

Woods, D. D. and Wreathall, J. (2008). Stress-Strain Plot as a Basis for

Assessing System Resilience. In E. Hollnagel, C. Nemeth and S. W.

A. Dekker, eds., Resilience Engineering: Remaining sensitive to the

possibility of failure. Ashgate, Aldershot, UK, pp. 145-161.

Zhou, T., Carlson, J. M. and Doyle, J. (2005). Evolutionary dynamics

and highly optimized tolerance. Journal of Theoretical Biology, 236,

438-447.