Copyright Avancier Limited. All rights reserved.
We need to take more sophisticated view of
change management than current standard processes for it.
Of course change management processes are needed, but you must understand their costs, and when/where/how to be agile.
Architecture is all about change management. Architects define target configurations; they define changes (large or small, strategic or tactical) to baseline configurations. You will never realize a target architecture if you don’t have processes to manage changes to your existing systems. You cannot produce a useful target architecture description unless you have processes for examining and accepting changes to requirements and solution visions during the architecture development process.
Change management processes are necessary. Unfortunately, managers (in both enterprises and service providers) are often so wary of "scope creep" that they impose heavy-weight change management practices. The effects of these are easily underestimated. They can add hugely to the time and cost of any creative activity. They tend to:
Of course change management processes are needed, but you must understand their costs, and when/where/how to be agile.
First, the mythical man month. Second, the myth that heavier change management is the solution. Third, the naivety of change management standards of the kind you find in a typical quality management system. This paper explores the first two, going some way to explain why change management time and costs tend to reduce productivity in a large projects. It leads to another paper that addresses challenges your change management standards may not even mention.
Configuration items are usually items are related in a structure, often a complex network structure. So, a change to one item can reverberate through many other items. This is what makes change management essential. This table defines key terms in change management.
|
Agile |
Willing and able to speedily respond to change. |
|
Baseline configuration |
A specification or product structure that has
been formally reviewed and agreed upon. The basis for further development.
Can be changed only through formal change management. E.g. a contract, a
requirements catalogue, architecture documentation, or a hardware
configuration. |
|
Change Control |
The organisation and processes needed within
change management to: ·
Monitor the potential
sources of change ·
Record change requests ·
Perform impact analysis ·
Decide which changes
should be made. |
|
Change management |
The organisation and processes needed to both
exercise change control to a baseline, and perform configuration management. |
|
Configuration Item |
An item in a baseline configuration. Could be a
requirement, a source code component or a hardware device. Can be at any
level of granularity. “Component of an
Infrastructure under the control of configuration management. A configuration
item can range from an entire system (hardware, software, documentation) to a
single hardware component.” ITIL |
|
Configuration management |
The organisation and processes needed within
change management to establish a baseline configuration and apply changes to
that baseline configuration. Involves work to: ·
Identify and document the
characteristics of each item. ·
Define dependencies
between items. ·
Control the introduction
of new versions of items. ·
Report the status of
configuration items and changes to them. |
|
Impact analysis |
Analysis of the effects of a change (perhaps a
new requirement or deliverable) to find the effects of that change. How does
it impact what has been done so far? How does it constrain what is planned
for the future? Leads to an impact analysis report. |
|
Request for Change |
“Form used to record details of a request for a
change to any Configuration Item within an Infrastructure or to procedures
and items associated with the Infrastructure.” ITIL |
You can probably manage a configuration of a 100 items in your head - hold the structure in mind and remember where the inter-item dependencies are.
On the other hand, you probably cannot do this with a configuration of 10,000 items.
My rule of thumb is that one architect can hope to manage a configuration of up to 1,000 items.
What to do when the required structure grows larger than one person can handle? Managers commonly respond by:
There is a huge step change from a configuration managed by one person, to a configuration managed by just two or three. Suddenly, a lot of oral and written communication is needed, where previously everything was contained in a single mind. And delays are introduced while one person waits for another to complete a task.
As your team size grows, the mythical man month comes into play. As Wikipedia tells us:
The Mythical Man-Month: Essays on Software Engineering is a book on software project management by Fred Brooks, whose central theme is that "Adding manpower to a late software project makes it later." This work was republished as an anniversary edition in 1995 (ISBN 0-201-83595-9) with the essay "No Silver Bullet" and commentary by the author.
Assigning more programmers to a project running behind schedule will make it even later, due to the time required for the new programmers to learn about the project, as well as the increased communication overhead. When N people have to communicate among themselves (without a hierarchy), as N increases, their output M decreases and can even become negative (i.e. the total work remaining at the end of a day is greater than the total work that had been remaining at the beginning of that day, such as when many bugs are created).
Group Intercommunication Formula: n(n − 1) / 2
For example: 50 developers -> 50(50 − 1) / 2 = 1,225 channels of communication
Brook’s formula probably understates the problem. Managers add people not only when a project is late, but also where a configuration has exceeded the size and complexity the current team can manage. And configuration management costs also increase disproportionately with the increasing size of a configuration.
Potential
configuration interdependencies: n(n − 1) / 2
For example: 50 items -> 50(50 − 1) / 2 = 1,225 potential interdependencies between items.
For example: 100 items -> 100(100 − 1) / 2 = 4,450 potential interdependencies between items
Formalising change management processes, alongside team size, tends exaggerate the effect of Brook’s law – since it adds overheads and increases group intercommunication while not increasing product creation activity.
This probably explains why, when you
increase the size of a system or project team, there is a disproportionate
increase in the resources and budget needed for change management
Wikipedia says of the mythical man month: “The tendency for managers to repeat such errors led Brooks to quip that his book is called "The Bible of Software Engineering" because "everybody reads it but nobody does anything about it!"
What can managers do about it? Can agile development principles help?
There are systems and descriptions of systems. Architecture descriptions are abstractions. So there are two entirely different ball games
Consider three configurations involved in building a house. There is a vision. The customer wants a five bedroom house, facing east, a dining room at the front, a large living room at the rear with a grand fireplace. The architect produces a requirements configuration. The builder produces a solution configuration.
|
The architecture – the requirements
configuration. |
The built system - the solution
configuration |
|
The architect provides only abstract drawings. The product is a design and bill of materials. |
The builder buys and makes the necessary concrete items. The product is a completed house. |
|
The drawings are related each other. E.g. The walls of the upper floor rooms must sit on the walls of the lower floor rooms. |
The parts of the house are connected to each other. E.g. The fireplace (for which there is a detailed drawing) should fit in the space marked on another drawing. |
|
The configuration should be internally consistent. |
The configuration must be internally consistent, or the building will fall over. |
|
The customer inspects the drawings and confirms them in conversation with the architect. |
The customer and architect inspect the growing building and discuss requirements and modifications with the builder. |
|
The customer makes changes - introduces requirements not mentioned in the vision. |
The customer and builder make changes. If the fireplace doesn’t fit the wall space, then the builders may change one or the other without telling the architect. |
|
The architect captures changes in the drawings. |
The architect and the drawings become less and less significant as the house is built, and the customer can see it and experience it. Drawings and house may end up out of step. |
|
Changes are to drawings are cheap. |
Changes become more and more difficult as the house grows. |
Analogies like building a house are both appealing and dangerous. They can create false expectations.
Building software systems is a little like building houses. In both cases you expect that:
|
The requirements configuration should be internally consistent. |
Though you may never discover if it is not. |
|
The working system or solution configuration must be internally consistent. |
Client components must call server components via shared interfaces, or else the system will fall over. |
|
The requirements configuration omits details included in the solution. |
A Gartner report said more than 80% of software has no documented requirements. |
|
The solution may depart from the requirements. |
Builders must build a working system whether requirement documents are up to date not. |
But as our paper “Software is not hardware” explains at length, building software systems is not like building houses. In building software, unlike in building houses:
|
The requirements configuration is obscure to customers. |
It can be hierarchical specification pyramid containing thousands of pages of information, some very abstract, some very technical. This is why “signing off” software requirements is often delayed, and sometimes an unrealistic expectation. |
|
The solution configuration is even more obscure. |
The source code is intangible – just more information – even more detailed specification. The customer cannot understand it. The architect cannot easily validate it by inspection. The executable code is even more obscure - readable only by the computer – it cannot be assessed by inspection. |
|
Software configurations (being ‘soft’) are infinitely malleable. |
If a house was like this, then the upper floor could be built before the ground floor. Software solution items and test cases are continually changed. The whole solution may be rebuilt every day or so. |
|
The customer cannot evaluate a solution by looking a description of it. |
The more detailed the information, the closer to code, the less the customer understands it. |
|
Much effort (often 30% to 50%) must be devoted to testing the solution. |
Note that test cases are relatively understandable by the customer. |
IT operations teams manage concrete hardware and network configurations. They need to know which applications (at which versions) are deployed on which machines. IS development teams manage abstract software configurations. They develop and manage fine-grained software components. They store these fine-grained components in an application-centric configuration – using software configuration management tools.
These two configurations are at different levels of abstraction. True, the IT operations need to know which software is deployed where, but they are entirely unaware of an application’s intermal configuration, they know only about coarse-grained deployable files of executables.
Either team, in changing what they see as their configuration, can disable a configuration managed by the other. “Disable” here might mean stop altogether. But it could also mean reduce the performance or availability of existing software below an acceptable level. E.g.
· An IT operations team might remove a server, or upgrade an infrastructure component wherever it appears in the IT estate, unaware that the new configuration will disable an existing application.
· An IS development team may develop using a different version of platform software from that approved by IT operations. Or they may install new software onto existing hardware unaware that the new software will disable some other software already deployed on that hardware.
So, each team has to take care they understand the impact of a change on the configuration(s) managed by other teams. This despite the fact that the two teams use different configuration management repositories.
In theory, you can say that a system and all descriptions of it forms a single configuration, in which every item is related directly or indirectly to every other item. In practice, the architecture of the system is recorded in several configurations - related of course, but maintained relatively independently of each other.
If you look for example at a software system in development, you’ll see several related but distinguishable configurations. E.g.
Obviously these four configurations are related. They are four parallel descriptions of the same system – from different perspectives and at different levels of abstraction. In a sense, they form one large configuration. But they are also distinct in some ways - built using different methods and tools - perhaps by different people at different times.
The larger the project, the more distinct configurations you have to maintain. You may maintain not only the four configurations above, but also several versions of each. At any given time, you may find
You need a process that defines how a configuration moves from abstract to concrete to archived, and back again if need be.
How to record the dependencies between configuration items?
“Every dependency between a new item and those already included in the configuration must be recorded at the point of registration.
“You need not describe the nature of a dependency: it is likely to change during the life of the item and is better discovered by impact analysis under change control. Note however that changes to dependencies are changes to items, and so changes to the whole configuration.
“Record both the name and version number of all dependent items. Dependences are necessarily version-specific and it is essential to cross-relate the various versions of interdependent items as they are changed.
“A complete record of item interdependences is a pre-requisite for effective impact analysis. In its absence, it is extremely difficult to determine accurately whether and how a given change will affect costs and timescales.” From a company’s configuration management standard.
You may choose to record dependencies in one or more of these ways:
You may need to record traceability between requirement and solution configurations, or between any distinct configurations. A standard configuration management process may tell you to maintain traceability records along these lines:
“The traceability process is part of the configuration management processes.
It should be defined in the project/service initiation documents and linked to the solution development/maintenance process.
All items in traceability records should be under configuration management.
Traceability records should be extended as items are approved and placed under configuration management.
Traceability records should be maintained throughout specification, design, code, and testing.
Traceability records can be maintained in spreadsheets, but it is better to use a specialist tool, especially where there are several hundred requirements, and several thousand test cases and solution components.
Traceability records should be visible to all project participants.” From a company’s configuration management standard.
Some standards advise us to record dependencies both within a configuration and in external traceability records. Any such duplication of documentation is a headache. It creates extra work. The two records can easily get out of step. The whole idea is a hostage to fortune. Quality auditors – perhaps because they are unable to judge the quality of the product itself – spend their time looking for discrepancies between traceability records.
This paper has described some change management challenges:
Here is an attempt to explain the cost of change management using a formula:
Change management consumes so much time and cost that you must explicitly and repeatedly decide where you want heavy change management processes, and where don’t. You must balance the costs of applying heavyweight processes against the risks of applying lightweight processes. And to do this, you must understand the various times and places that change management can be relaxed.
Read Agile change management for further explanation of the formula above, and ways to make change management more agile.