Slide presentation presented at Workshop
Submitted to: OOPSLA'98 Workshop on Modeling Dynamic/Emergent Distributed Object Systems (This background paper for the Business Object Workshop will provide crossfeed to the Workshop focussed on complex adaptive systems.)
Abstract: Concepts in Complex Adaptive Systems (cas) research are relevant to the development of enterprise business object component systems. Many mathematical and computing models have been developed for cas in recent years [CPM 94] and much of this work can be applied, at least conceptually, to Business Object Component Architectures now emerging as the mechanism of choice for building large distributed object systems.
Business entities are often used as examples of complex adaptive systems. The modification time of a business firm is on the order of months or years, about the same amount of time required to enhance its computing systems. Automating business processes renders those parts of the business in software, thus enterprise software systems are examples of cas. A business system has a severely constrained rule set, compared to a typical cas system, like New York City or the U.S. economy. This makes it an ideal test bed for cas concepts. At the same time, flexibility, adaptability, and reusability of these systems can determine the ability of an enterprise to evolve and survive in the marketplace. Since these typical characteristics of cas systems are lacking in most business software, business systems could be significantly more effective if architected with cas concepts.
New discoveries in object technology parallel Holland's analysis of cas. Event driven, distributed object component systems are "interactive systems." Wegner has shown that interactive systems are not Turing machines [Wegner 95]. All interactions in these systems cannot be anticipated because behavior emerges from interaction of system components with the external environment. Such systems can never be fully tested, nor can they be fully specified. Thus contemporary, event driven, business object component systems exhibit emergent behavior, a fundamental feature of cas. And they are clearly complex, to the extent that the running system may be the shortest description of its behavior. This is certainly true of large enterprise systems running in multiple sites with different custom code and hardware platforms at every site.
For some years, the author has described a Business Object Component Framework using the diagram below [Sutherland 98]. Objects are aggregated into components. Components are aggregated into meta-components and a hierarchical structure is built to support enterprise software systems. Holland uses an equivalent diagram to describe a complex adaptive system made of of adaptive agents (see Figure 1.1, Holland 95, p. 6). Work on Business Object Component systems is evolving along the lines of cas, because successful systems are, in fact, instances of complex adaptive systems. Of the 80% of all software projects deemed failures, reasons for failure are often related to the inability of systems to rapidly adapt to changing business needs [BMMM 98]. In fact, the failure to design software systems to support cas behaviors dooms the majority of them to failure and the minority survivors to premature obsolescence.
Figure 1: Business Object Component Framework
It is often taken for granted that as systems evolve over time they tend to become more complex... In single systems it may grow by increases in structural sophistication: the system steadily cumulates increasing numbers of subsystems or subfunctions or subparts to break through performance limitations, or to enhance its range of operation, or to handle exceptional circumstances. Or, it may suddenly increase by "capturing software": the system captures simpler elements and learns to "program" these as "software" to be used to its own ends... A particularly telling example of capturing software is the way in which sophisticated derivatives have arisen and are used in recent years in financial markets... [Arthur 94]In 1993, the author led the "Synchronicity" team at Easel Corporation building a software development environment that would allow creation of software systems from design and support synchronization of design, code, and other software artifacts throughout the software life cycle. The core of the development environment was an object oriented analysis and design facility for building three tier architectures that was seamlessly integrated with automated user interface tools, round-trip engineering facilities to keep design synchronized with code at all stages of the life cycle, and automated object mapping facilities for persistent object storage in relational databases.
The result of this project was the delivery of a set of visual development tools that allowed implementation of business systems six times faster than 4GL systems built with VB or PowerBuilder. And the systems were architected properly for distributing components across system tiers. An interesting side effect was that Bankers Trust used this environment to build the kind of derivative systems mentioned in Arthur's article on cas (see quote above). The resulting increase in market share added tens of millions of dollars of profit to their bottom line in less than a year. Derivative systems needed to adapt on a weekly basis, i.e. a new product should take a week to implement. Stock exchange systems often require a one day turnaround for major enhancements. So production cas systems are alive and well in certain segments of the software industry.
Several major learning experiences for the Synchronicity team relate directly to the intersection of Business Object Components Architectures and cas.
A group of objects is the ideal unit of reuse. Groups of objects behave as a higher level business process and need a clearly specified business language interface. Proponents of cas should note carefully Abelson and Sussman's comments in their classic MIT computer science text [Abelson et al. 96]. The power of computing lies in recursively writing higher levels of language that are supported by lower level languages (thus inducing emergent behaviors).
Business Object Components need to be encapsulated with a protocol that allows efficient communication with other objects on the network. Work on the concept of Ensembles has shown that there is a minimal design specification for a plug compatible component [Love 93, SM 93]. A business object component needs semantics that extend beyond the syntax of COM components and JavaBeans containers. These semantics are covered by the recent OMB BOCA proposal [OMG BOM 98-01-07].
Figure 2: Business Object Component
Consider a typical client/server application like an order entry system. This core component of this system takes a Purchase Order as input and produces a validated order as output or, alternatively, raises exception conditions. Key issues are (1) internals of this component should be a black box to the external world, (2) externals of this component are a black box to internal objects, (3) external components can register for notification of interesting internal events, (4) internal components can be changed or upgraded without affecting external operations, (4) external components can be changed without affecting the component's operations, and (5) any internal component can be run on any processor and the whole component retains its functional integrity. These objectives are largely achieved by emerging COM+ and Enterprise JavaBeans specifications at the syntax level.
More importantly from a business perspective, the Business Object Component must guarantee proper behavior in response to a set of scenarios that ultimately trace back to use cases. These scenarios must be evoked by messages in a standardized context, including timing and sequencing, to guarantee proper operation of the component in any system that uses the component according to its design specification. While these minimal requirements were worked out years ago by Sutherland and McKenna [SM 93], and pursued by the OMG Business Object Domain Task Force since 1995, they have yet to evolve into any standard approach for building plug compatible business components.
It is critical that standard Business Object Components be available to enable a business system to function as a complex adaptive system. Adaptive means that systems must easily evolve and this requires continuous changing, updating, adding, subtracting, and rearranging components. When changing one component causes ripple effects throughout a system, the cost of rewriting major portions of the system slows evolution to a snails pace. This is the state of the 80% of business systems that are viewed as unsuccessful implementations in industry today [BMMM 98].
This punctuated equilibrium effect has been
observed by teams working in a component based environment with adequate
business process engineering tools. Use of the SCRUM development process
accentuates the effect [Schwaber
98]. For example, Figure 3 shows the conceptual context of a component
based system [Sutherland
Figure 3: View of a Business Object Component System
A project domain can be viewed as a set of
packages that will form a release. These are what the user perceives as
pieces of functionality. Packages evolve out of work on topic areas. Topic
areas are business object components. Changes are introduced into the system
by introducing a unit of work that alters a component (Figure 4). The unit
of work in a SCRUM has been called a Synchstep.
Figure 4: Firing a Synchstep
After one or more Synchsteps have gone to completion, forcing some refactoring throughout the system, or often simply providing new functionality to existing components, a new package of functionality emerges that is observable to the user. These Synchsteps are similar to genetic mutations. Typically, several interrelated components must mutate in concert to produce a significant new piece of functionality. And this new functionality appears as a "punctuated equilibrium" effect to builders of the system. For a period of time the system is stable with no new behavior. Then when a certain (somewhat unpredictable) Synchstep completes, the whole system pops up to a new level of functionality.
In a Smalltalk environment with object oriented analysis and design tools that support round-trip engineering, this process can happen extremely fast, much faster than the rest of the development organization can absorb new functionality. We found it necessary to slow down the process by keeping the number of developers small and outnumbering them with quality assurance and documentation people on the SCRUM development team.
The ability to product a complex adaptive business computing system has been demonstrated, albeit adaptation has to be manually induced. The team organization and tools required to do this are not available to most engineering organizations today. It is, however, easy to visualize an environment with introspective components that can self adapt. All of these capabilities await standardization of the syntax and the semantics of flexible business object component environments before large scale adoption is feasible. As these capabilities become available (as they are in some proprietary environments), the concepts of cas become very useful in thinking about the specification and design of enterprise business object component systems.
Holland, in his Ulm lectures at the Santa Fe Institute [Holland 95], creates a synthesis of seven basic cas concepts, four properties (aggregation, nonlinearity, flows, diversity) and three mechanisms (tags, internal models, buildings blocks). These concepts can organize our discussion of business object systems.
Aggregation (property) - there are two important modes of aggregation in cas systems. Aggregation is a basic mechanism in object modeling and is the basis for identity, a fundamental object concept. Forming components out of objects and enterprise systems from components is higher level aggregation. More important are emergent properties such as intelligence that evolve out of dumb subsystems. This is the basic concept in Minsky's "Science of Mind" [Minsky 88] or Hofstader's analysis of an ant colony [Hofstadter 79]. Meta-agents (an enterprise) are formed of aggregates of agents (enterprise systems) and exhibit emergent behaviors (revenue, profitability, and cash flow, the indices of value creation).
Tagging (mechanism) - this mechanism facilitates the forming of aggregates, from HTML pages to the mechanisms in CORBA or DCOM that allow inter object communication. They facilitate selective mating, i.e. firewalls block certain tagged elements to protect the enterprise. Thus they preserve boundaries between aggregates. They allow us to componentize object models and enable filtering, specialization, and cooperation. They are the mechanism behind the development of hierarchical aggregates that exhibit emergent behaviors like an operating system. The basic mechanisms of evoking operations through messages in object technology are based on tagging strategies.
Nonlinearity (property) - nonlinear systems exhibit catastrophic and chaotic behaviors. Traffic flow on the Internet is nonlinear, leading to predictions of the collapse of the network. Brownouts, system loadings, scalability effects are often nonlinear. The arrival, proliferation, and destruction of viruses on the Internet is a nonlinear phenomenon that can be modeled like predator/prey interactions in biological systems. Even more basic phenomenon like revenue prediction in an enterprise financial system has nonlinear behaviors that are often not recognized. The rate of construction of software itself is a nonlinear phenomenon.
Flows (property) - workflows are examples of flows in action. Message routing is a flow. Tags condition flows which often exhibit nonlinear behaviors and emergent behaviors. Flows typically have a multiplier effect. Money injected into the economy has an effect out of proportion to the amount, similar to email or other message flows on a network. The recycling effect of flows enables the rain forest, as well as an enterprise computer ecosystem. Individual pieces evolve, die, are replaced or reused, constantly changing the characteristics of the enterprise. Living software is software that is constantly changing due to flows, as rivers change their course. Dead software is eventually detritus that is expelled from the enterprise organism.
Diversity (property) - persistence of an individual agent depends on the ecosystem of agents that surround it, whether the agent is an ant in the rain forest or a business object in an accounting system. The evolution of these agents as software changes causes convergence of system architectures. This is the basis of emergent patterns that reappear again and again in widely disparate environments. It is difficult to evolve a single agent to make it more useful in an isolated context. Usefulness in business object systems arises from interactions between diverse agents as in human societies.
Internal models (mechanism) - the utility of complex systems is enhanced if the system can learn from experience and adapt its behavior. The ability of the system to develop and act on internal models that simplify the external world is basic to this mechanism. It allows the system to infer the results of actions before they are taken, and to choose actions that have productive results. The prospects for longevity of software systems depend on this capability, just as in living systems.
Building blocks (mechanism) - reuse is dependent on building blocks used over and over again. It is the basis of Moore's law in hardware production. It could be the basis of dramatic improvements in software productivity. Building blocks are the basis for generation of internal models and are essential to the construction of adaptive enterprise systems.
A systematic analysis of business object system architecture properties with regard to Holland's seven basic cas concepts, four properties (aggregation, nonlinearity, flows, diversity) and three mechanisms (tags, internal models, buildings blocks) would be helpful in both architecting systems and the standardization process for a Business Object Component Architecture. These is a theme for the OOPSLA'98 Business Object Design and Implementation Workshop [OOPSLA 98].
The basic difficulty, to put the matter plainly, is an insufficiency of facts. The complexity theorists do not yet have enough information to carry with them into cyberspace. the postulates they start with clearly need more detail. Their conclusions thus far are too vague and general to be more than rallying metaphors, and their abstract conclusions tell us very little that is really new. [Wilson 98]Since these seven cas building blocks have proved useful in describing cas systems in general, they could be the basis for a taxonomy of business object systems. Evaluating a Business Object Component Architecture with respect to these categories could yield a rating on the flexibility, adaptability, and reusability of distributed object computing systems. Further elaboration and application of these concepts to specific systems is not possible in a short paper.
Engineers working on Business Object Component Architectures would benefit by gaining an understanding of cas concepts and applying them to their work. These concepts are larger in scope and broader in application than some of the Business Object work now under development. Some of the best logical and mathematical minds of our time are gravitating to cas research. We can expect fundamental breakthroughs in our understanding of how to build complex adaptive systems that could translate directly to Business Object Component Architectures if the dialogue and definitions used by Business Object specialists aligned with cas research. Furthermore, the Business Object community could provide some of best testing ground for cas concepts and fill in the major gap in cas research by providing hard data on production systems exhibiting cas qualities.
[Arthur 94] Arthur, W. Brian. On the Evolution of Complexity. In Complexity: Metaphors, Models, and Reality. Cowan, George A., David Pines, David Meltzer (eds.), Proceedings Vol. XIX, Sante Fe Institute Studies in the Science of Complexity. Addison-Wesley, 1994.
[Berreby 98] Berreby, David. Complexity Theory: Fact Free Science or Business Tool. Strategy and Business 10:40-50, First Quarter, 1998.
[BDSSS 98] Beedle, Mike; Devos, Martine; Sharon, Yonat; Schwaber, Ken; Sutherland, Jeff. SCRUM: An extension pattern language for hyperproductive software development. Submission to PLoP: The 1998 Pattern Languages of Programs Conference. August 11-14, 1998 Allerton Park, Monticello, Illinois, USA.
[BMMM 98] Brown,William J., Raphael C. Malveau, Hays W, III McCormick, William H. Brown, Thomas J. Mowbray. AntiPatterns: Refactoring Software, Architectures, and Projects in Crisis. John Wiley & Sons, 1998.
[Booch 95] Booch, Grady. Object Solutions: Managing the Object-Oriented Project. Addison-Wesley, 1995.
[CPM 94] Cowan, George A., David Pines, David Meltzer (eds.). Complexity: Metaphors, Models, and Reality. Proceedings Vol. XIX, Sante Fe Institute Studies in the Science of Complexity. Addison-Wesley, 1994.
[Dennett 96] Dennett, Daniel. Darwin's Dangerous Idea: Evolution and the Meanings of Life. Touchstone Books (reprint edition), 1996.
[OMG BOM 98-07-01] Data Access Technologies et. al. OMG Business Object Domain Task Force BODTF-RFP 1 Submission: Combined Business Object Facility: Business Object Component Architecture (BOCA) Proposal. OMG Document BOM 98-07-01.
[Hofstadter 89] Hofstadter, D.R. Godel, Escher, Bach: An Eternal Golden Braid. Vintage Books, 1989.
[Holland 95] Holland, John H. Hidden Order : How Adaptation Builds Complexity. Addison-Wesley, 1995.
[Levy 93] Levy, Steven. Artificial Life: A Report from the Frontier Where Computers Meet Biology. Vintage Books, 1993.
[Love 93] Love, Tom. Object Lessons : Lessons in Object-Oriented Development Projects. SIGS Publications, 1993.
[Manola 98] Manola, Frank. Towards a Web Object Model. Position Paper for the OMG-DARPA-MCC Workshop on Compositional Software Architectures. Object Services and Consulting, Inc., 1998.
[Minsky 88] Minsky, Marvin. The Society of Mind. Simon and Schuster, 1988.
[Mowbray 98] Mowbray, Thomas A. Secrets of Knowledge for OO Architects. Component Strategies 1:1:18-21, July 1998.
[Odell 98] Odell, James. Agents and Beyond: A Flock is Not a Bird. Distributed Computing, April, 1998.
[OOPSLA 95] OOPSLA'95 Workshop on Business Object Design and Implementation II. 10th Annual Conference on Object-Oriented Programming Systems, Languages, and Applications Addendum to the Proceedings. OOPS Messenger 6:4:170-175. ACM/SIGPLAN October, 1995.
[OOPSLA 96] OOPSLA'96 Workshop on Business Object Design and Implementation II. 11th Annual Conference on Object-Oriented Programming Systems, Languages, and Applications Addendum to the Proceedings. OOPS Messenger, ACM/SIGPLAN, 1997.
[OOPSLA 97] OOPSLA'97 Workshop on Business Object Design and Implementation III. 12th Annual Conference on Object-Oriented Programming Systems, Languages, and Applications Addendum to the Proceedings. OOPS Messenger, ACM/SIGPLAN, 1998 (in press). Download PDF, RTF, Word versions.
[OOPSLA 98] OOPSLA'98 Workshop on Business Object Design and Implementation IV. 13th Annual Conference on Object-Oriented Programming Systems, Languages, and Applications. Vancouver, October 1998.
[Schwaber 95] Schwaber, Ken. SCRUM Development Process. In Business Object Design and Implementation: OOPSLA'95 Workshop Proceedings. Sutherland J., D. Patel, C. Casanave, G. Hollowell and J. Miller (Eds), Springer, 1997.
[Schmidt 98] Schmidt, Marc-Thomas. Building
Workflow Business Objects. Business Object Design and Implementation:
OOPSLA'98 Workshop Proceedings.
[Sutherland 98] Sutherland, Jeff. Objects, Databases, and the World Wide Web. COMDEX/Object World Tutorial, San Francisco, 1998.
[SM 93] Sutherland, Jeff and Jeff McKenna. Ensembles. Easel Corporation, 1993.
[SPCHM 97] Sutherland J., D. Patel, C. Casanave, G. Hollowell and J. Miller (Eds). Business Object Design and Implementation: OOPSLA'95 Workshop Proceedings. Springer, 1997.
[Waldrop 93] Waldrop, M. Mitchell. Complexity : The Emerging Science at the Edge of Order and Chaos. Touchstone Books, 1993.
[Wegner 95a] Wegner, Peter. Interactive Foundations of Object-Based Programming. IEEE Computer 28:10:70-72, Oct 95.
[Wegner 95b] Wegner, Peter. Models and Paridigms of Interaction. OOPSLA'95 Tutorial Notes.
[Wilson 98] Wilson, Edward O. Consilience : The Unity of Knowledge. Random House, 1998.
[Ziv 96] Ziv, Hadar and Debra J. Richardson.
Uncertainty Principle in Software Engineering. ICSE'97, 19th International
Conference on Software Engineering. 23 August 1996.
OOPSLA'98 Business Object Workshop IV