I've been working on a massive project to determine possible evolutionary paths in complex systems development. Some see this as engineering, while others argue that engineering is more deterministic than this characterization permits.
The difference is fundamental. Engineering applies (hopefully) scientific principles to a problem domain at specific points in time. The possiblity space however, extends from the past, through the present, and into the future. However, the evolution of the possibility space, in time, changes the nature of the problem, and therefore its engineering solutions. Therefore, it resembles an ensemble of network paths - an entanglement of path trajectories, some of which will almost certainly turn out to be wrong. From a practical perspective, how can we transition from what we discover is an incorrect path, to a more correct path, without having to "backtrack"? The answer seems to be understanding the evolution of the possibility space, contemporaneously with the paths we have actually taken.
This denies the Markov model (and I contend even hidden Markov models) as only a partial, historic artifact, thus incomplete. The reason is this: The effect of new information upon the prior changes the entire nature of the model - the understanding of its history, the understanding of its present states, and the trajectory of evolutionary "realization" it is on.
What is evolving is not only the system under investigation, but the contexts in which it is evaluated. This demands that engineering, likewise, be an evolutionary flow through possibility spaces, with the realized products and processes becoming artifacts of that evolution.
The term "artifacts" includes myriad errors and omissions along the data, information and knowledge gained.
Saturday, February 27, 2010
Sunday, November 1, 2009
Dr. Stogatz, Meet Dr. Lawvere
There are interesting ways of categorically mapping the structure, behavior and evolution (morphisms) of dynamical systems ala Steven Strogatz in "Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry and Engineering", by Perseus Publishing. This has been detailed by Category Theory (CT - Eilenberg, Mac Lane, Lawvere, among many more) and the use of functors - a programmatic analog affecting state evolutions.
Rather than the set-theoretic approach, with it's limitations of First Order Logic, CT provides a more accurate logical / analytical framework in which to understand complex, dynamical systems, and categorize (or classify) their patterns of change.
Curious. This particular technique seems grossly under-utilized in Complex Systems Theory, and in understanding complex, dynamical systems, their processes, growth and evolution. Furthermore, it seems that these known techniques are actually ignored.
If someone has a good explanation for this phenomenon, I'd sure like to hear it - or am I not looking for exemplars in the right places?
Rather than the set-theoretic approach, with it's limitations of First Order Logic, CT provides a more accurate logical / analytical framework in which to understand complex, dynamical systems, and categorize (or classify) their patterns of change.
Curious. This particular technique seems grossly under-utilized in Complex Systems Theory, and in understanding complex, dynamical systems, their processes, growth and evolution. Furthermore, it seems that these known techniques are actually ignored.
If someone has a good explanation for this phenomenon, I'd sure like to hear it - or am I not looking for exemplars in the right places?
Tuesday, August 25, 2009
Parallel Paths
First, for all my life I have been a champion of many educational issues. Education is far too important to just to be left in the hands of educators. It is not a commodity. By this I mean, education is both a community AND individual personal responsibility.
The processes of education runs in parallel with processes of training. These are very different functions, although these functions are often confused. Often, training and education are in conflict.
Which brings me to the point of this posting: In software engineering, we are trained to view software either from a serial perspective, or a parallel perspective, or maybe some combination of the two. It is obvious that we can run several serial tasks in parallel, and that parallel paths often consist of individual serial task lanes. Nice, neat, very deterministic.
We are taught to rollout loops and partition data for parallel processing - and that is good thing to learn. But I wonder, is the way we are trained to think about computer science and software engineering - in exclusive terms of serial and parallel fabrics - preventing us from understanding the value other computational fabrics? In other words, has our training overly constrained the purposes of our education?
Many of the systems I deal with exhibit a property called complexity. Often this is considered a "problem" - to be avoided or simplified, rather than considered offering an excellent solution strategy. From a computational point of view, complex networks offer extremely efficient encoding capabilities, and sometimes (but, not always) extremely efficient computational capabilities.
In putting together a proposal for a VLS compute cluster, I see how dissonance between training and education affects the conceptualization of computation on clusters of many highly trained professionals. Is this an educational opportunity? Or does training rule the day?
Stay tuned ...
The processes of education runs in parallel with processes of training. These are very different functions, although these functions are often confused. Often, training and education are in conflict.
Which brings me to the point of this posting: In software engineering, we are trained to view software either from a serial perspective, or a parallel perspective, or maybe some combination of the two. It is obvious that we can run several serial tasks in parallel, and that parallel paths often consist of individual serial task lanes. Nice, neat, very deterministic.
We are taught to rollout loops and partition data for parallel processing - and that is good thing to learn. But I wonder, is the way we are trained to think about computer science and software engineering - in exclusive terms of serial and parallel fabrics - preventing us from understanding the value other computational fabrics? In other words, has our training overly constrained the purposes of our education?
Many of the systems I deal with exhibit a property called complexity. Often this is considered a "problem" - to be avoided or simplified, rather than considered offering an excellent solution strategy. From a computational point of view, complex networks offer extremely efficient encoding capabilities, and sometimes (but, not always) extremely efficient computational capabilities.
In putting together a proposal for a VLS compute cluster, I see how dissonance between training and education affects the conceptualization of computation on clusters of many highly trained professionals. Is this an educational opportunity? Or does training rule the day?
Stay tuned ...
Monday, August 10, 2009
Models of Complexity
OK, I apologize for the implication that models are a paradigm of complexity.
The issue I've been discussing for the past several days with several colleagues is: "What do you mean by a model?" This stems from the fact that the term "model" means something different in science (esp. physics), math and engineering.
The reason for the problems stem from the fact that each domain has excellent reasons to adopting their unique definition. Each definition is right in some aspect, yet each is wrong. I have proposed a definition of a model extending from mathematical concepts. Everyone jumped on me for being "too theoretical". And here I thought I had some very practical, easy-to-use reasons for the math-centric foundation that allowed it to extend easily and map across the domains.
That's what I get for thinking ...
The issue I've been discussing for the past several days with several colleagues is: "What do you mean by a model?" This stems from the fact that the term "model" means something different in science (esp. physics), math and engineering.
The reason for the problems stem from the fact that each domain has excellent reasons to adopting their unique definition. Each definition is right in some aspect, yet each is wrong. I have proposed a definition of a model extending from mathematical concepts. Everyone jumped on me for being "too theoretical". And here I thought I had some very practical, easy-to-use reasons for the math-centric foundation that allowed it to extend easily and map across the domains.
That's what I get for thinking ...
Monday, July 13, 2009
Complexity is Hard Enough
... Don't make it harder by mis-understanding it.
Some colleagues and I have been discussing Bayesian reasoning to refine, and hopefully validate models of systems that exhibit complexity. This lead to the discussion of Taleb's "Black Swans", and ideas about how these may be generated, identified and simulated on a computer. I offer this link to Eliezer Yudkowsky's work that provides an excellent context for starting the discussion.
http://www.singinst.org/upload/cognitive-biases.pdf
It seems that one of the hallmarks of complexity is self-reference between meta-scales.
Some colleagues and I have been discussing Bayesian reasoning to refine, and hopefully validate models of systems that exhibit complexity. This lead to the discussion of Taleb's "Black Swans", and ideas about how these may be generated, identified and simulated on a computer. I offer this link to Eliezer Yudkowsky's work that provides an excellent context for starting the discussion.
http://www.singinst.org/upload/cognitive-biases.pdf
It seems that one of the hallmarks of complexity is self-reference between meta-scales.
Wednesday, June 3, 2009
Non-Deterministic Programs
Most (if not all) massively parallel computer programs are non-deterministic. Normally, this is due to the asynchronous nature of the parallel execution paths, but it is also caused by recursion or recurrence in the execution graph.
This brings me to a point of contention, not just a software constraint waiting for resources, but users waiting on a non-deterministic execution. Massively parallel programs (MPP) are normally breathtakingly fast. The challenge I'm facing is the better MPP "normally" performs, the shorter the user's expectation on the time it should perform. The answer is easily solvable (has an answer) for deterministic problems, but consists a long-tailed probability distribution for the more complex cases.
How long should a user expect to wait for a non-deterministic program to execute?
It helps to know this is not a new problem. See: Weinberg and Hoag, 1957 http://blogs.msdn.com/micahel/archive/2009/03/06/GeraldMWeinbergsFavoriteBug.aspx
The challenge remains however of estimating the probabilities of for discovering solutions at given some amount of time. This begs the question - What are solutions?
Turning the problem on itself, the first challenge is separating non-deterministic problems from their analytical cousins. Normally, simple matrix analysis (LU, Cholesky, SVD) helps categorize these problems. For non-deterministic problems, allow the user to specify limits to fractal or recursive depth, then calculate the probabilities within these limits.
Finally, the lesson from Weinberg is: Show that the program is indeed working, and making some progress toward the goal, not just locked in a loop or forever in contention. Since I don't recall open-ended progression indicators in my toolbox, there needs to be mileposts to measure the fitness of a solution. While these too are open-ended, let the user decide. A graphical / network-topological representation of the problem/solution space helps the user understand the nature of the problem. Seeing information about the problem/solution might help in recasting the problem in humanly understandible terms.
This brings me to a point of contention, not just a software constraint waiting for resources, but users waiting on a non-deterministic execution. Massively parallel programs (MPP) are normally breathtakingly fast. The challenge I'm facing is the better MPP "normally" performs, the shorter the user's expectation on the time it should perform. The answer is easily solvable (has an answer) for deterministic problems, but consists a long-tailed probability distribution for the more complex cases.
How long should a user expect to wait for a non-deterministic program to execute?
It helps to know this is not a new problem. See: Weinberg and Hoag, 1957 http://blogs.msdn.com/micahel/archive/2009/03/06/GeraldMWeinbergsFavoriteBug.aspx
The challenge remains however of estimating the probabilities of for discovering solutions at given some amount of time. This begs the question - What are solutions?
Turning the problem on itself, the first challenge is separating non-deterministic problems from their analytical cousins. Normally, simple matrix analysis (LU, Cholesky, SVD) helps categorize these problems. For non-deterministic problems, allow the user to specify limits to fractal or recursive depth, then calculate the probabilities within these limits.
Finally, the lesson from Weinberg is: Show that the program is indeed working, and making some progress toward the goal, not just locked in a loop or forever in contention. Since I don't recall open-ended progression indicators in my toolbox, there needs to be mileposts to measure the fitness of a solution. While these too are open-ended, let the user decide. A graphical / network-topological representation of the problem/solution space helps the user understand the nature of the problem. Seeing information about the problem/solution might help in recasting the problem in humanly understandible terms.
Monday, May 11, 2009
HyperNEAT for CUDA
It's time for me to give back to the Neuro-Evolution for Augmenting Topologies (NEAT) community. I am starting to port strict HyperNEAT to a stand-alone CUDA implementation. For those who don't know, CUDA is NVidia's Compute Unified Device Architecture.
Unless I hear otherwise, it will be a simple implementation for 1 multi-core CPU (of say 2 on-die cores) and 1 CUDA capable GPU (say a GTX 8800 or a GTX 295). Nothing too fancy, just enough to give insight into how I would implement HyperNEAT on CUDA. The way I'm currently doing this may be novel - I'm using a factory class to generate the actual CUDA code, and I probably should offer my experiment development GUI to help others understand this "magic smoke" a little better.
Unless I hear otherwise, it will be a simple implementation for 1 multi-core CPU (of say 2 on-die cores) and 1 CUDA capable GPU (say a GTX 8800 or a GTX 295). Nothing too fancy, just enough to give insight into how I would implement HyperNEAT on CUDA. The way I'm currently doing this may be novel - I'm using a factory class to generate the actual CUDA code, and I probably should offer my experiment development GUI to help others understand this "magic smoke" a little better.
Subscribe to:
Posts (Atom)
