Saturday, February 27, 2010

Evolutionary Complexity

I've been working on a massive project to determine possible evolutionary paths in complex systems development. Some see this as engineering, while others argue that engineering is more deterministic than this characterization permits.

The difference is fundamental. Engineering applies (hopefully) scientific principles to a problem domain at specific points in time. The possiblity space however, extends from the past, through the present, and into the future. However, the evolution of the possibility space, in time, changes the nature of the problem, and therefore its engineering solutions. Therefore, it resembles an ensemble of network paths - an entanglement of path trajectories, some of which will almost certainly turn out to be wrong. From a practical perspective, how can we transition from what we discover is an incorrect path, to a more correct path, without having to "backtrack"? The answer seems to be understanding the evolution of the possibility space, contemporaneously with the paths we have actually taken.

This denies the Markov model (and I contend even hidden Markov models) as only a partial, historic artifact, thus incomplete. The reason is this: The effect of new information upon the prior changes the entire nature of the model - the understanding of its history, the understanding of its present states, and the trajectory of evolutionary "realization" it is on.

What is evolving is not only the system under investigation, but the contexts in which it is evaluated. This demands that engineering, likewise, be an evolutionary flow through possibility spaces, with the realized products and processes becoming artifacts of that evolution.

The term "artifacts" includes myriad errors and omissions along the data, information and knowledge gained.