Understanding the Vicious Circle of Complexity

"The number of lines of communications grow exponentially
based on the number of people involved in a project."
- Bryce's Law

INTRODUCTION

In an earlier bulletin, I discussed the various types of information resources found in an information system, the average number of each resource, along with the number of design decisions associated with each; see:

No. 10 - Managing Design Complexity

The issue of managing complexity is not simple. As our information systems continue to grow in magnitude, so do the costs associated with maintaining and updating them to suit the
current requirements of the company. Today's systems have grown into such uncontrollable behemoths that companies either elect to outsource them (thereby transferring the headache to someone else), let them run unchecked in end-users departments (whereby data and process redundancies run rampant), or they try to rewrite the system in its entirety (aka, a "Mission Impossible" project).

Compounding the problem of complexity is a vicious circle phenomenon that occurs during development projects. This circle is actually quite simple to understand and explain:

1. First, we start with a simple system.

2. Inevitably, changing user information requirements trigger a need to change the system.

3. To implement the change, more pieces and parts (resources) are required.

4. The change requires new or different people to implement it.

5. Due to inconsistencies in development (lack of standards), each developer is allowed to implement their piece of the puzzle as they see fit. Consequently, communications suffers thereby hindering development time.

6. Poor communications makes the overall system less manageable which adds to the problem of complexity whereby we become dependent on people to maintain different pieces of the system.

This results in a vicious circle whereby complexity is compounded with every development project. Instead of our systems becoming easier to manage, they are becoming much more complicated. So much so, no one person can visualize the system in its entirety.

UNDERSTANDING COMMUNICATIONS

To truly understand how communications compounds complexity, let's begin by understanding the number of lines of communications between people. Interestingly, the number of lines of communications grow exponentially based on the number of people involved. For example:

Number of People: 2
Lines of Communications: 1

Number of People: 3
Lines of Communications: 3

Number of People: 4
Lines of Communications: 6

Number of People: 5
Lines of Communications: 10

As if maintaining the number of lines of communications isn't enough, we must consider the content of the communications. Even if our lines of communications are well maintained, if there are no standards in terms of terminology and work effort, a "Tower of Babel" effect will result whereby developers trip over each other in an uncoordinated manner. Without standardization, systems become more difficult to maintain and modify, thereby compounding the complexity problem.

CONCLUSION

The failure of the ability of one person to handle the relationships of the entire system is due partially to the complexity of the system and partially to our failure to develop concepts which enables us to impose structure on this complexity. Because of this lack of structure, the designer cannot communicate properly with the user in defining requirements or in relating a solution for these requirements. The designer cannot properly communicate with the programmers and ultimately with the people who will be using, modifying and maintaining the system after it has been developed. In fact, the programmer cannot properly communicate with the computer due to the unstructured nature of requirements and the complexity of the processes needed to implement large systems.

Finally, this difficulty in communications manifests itself in the unreliability of each information resource in a system. As we have more and more interconnected resources, so that the failure of any resource results in the failure of the whole system, the reliability of each resource must increase, rather than decrease, in order to prevent the entire system from failing. Needless to say, the more complex the system, the less
likely it is that each of its resources will be more reliable when they are developed from difficult-to-communicate requirements that are implemented in difficult-to-communicate code. In one sense, we are looking for a solution analogous to the Industrial Revolution when there was the transition of a cottage industry to the industrial enterprise. To do so requires standardization of terminology and basic development concepts. Without such standardization, large systems will continue to grow in complexity. But with standardization and some commonsense management, not only can we begin to reduce the level of
complexity, we can turn systems development from an art to a science.