Learn yourself some innovation

Looking at today's programmer job positions, a lot of them, along with those usual lists of funky acronyms, include a somewhat confusing line: "strong problem solving skills."

To measure "amount of experience" in any concrete technology is already a tricky business, so how does one go with such an abstract thing as "problem solving ability"?

In software development there usually are many problems to solve, and for each of them there are many possible solutions. So it's not just about finding a solution, but rather about consistently picking the right solutions from many possible (and picking the right problems, for that matter). At this point we start talking about creativity and innovation.

The question is: is it possible at all to develop such a skill, in a deliberate way? Is this a personality trait, or intuition, experience, knowledge... or all of these?..

Turns out, people were trying to answer the question for ages. Socrates, Descartes, Poincare and many more researched the topic of creativity, innovation and problem solving, looking for answers to the same question: how good ideas are born in human mind?

The particular research which I've been poking into for a while, is called "Theory of Inventive Problem Solving" (aka TRIZ). The author, Russian inventor Genrich Altshuller, having worked in a patent office (it's interesting that Einstein had similar experience) and examined thousands of technical inventions, embarked on a task of finding some common traits in all these seemingly unrelated innovative ideas.

Essentially, he was reverse-engineering - first the good technical solutions themselves, and then mental processes, which could lead to such solutions emerging as an idea in one's head. And he came up with some core principles, building them later into his methodology, which managed to become quite recognized.

As with any good and recognized methodology which goes mainstream and gets the potential to commercialize (it's claimed to have been used at Xerox, Jonson&Jonson, Ford and other industrial giants), it eventually became somewhat complex and confusing, quite often being diluted with a snake oil (must...not...mention...agile...).

However, the core principles are simple and sound.

The first step is to understand the environment (system) in which problem is being solved. This includes building a mental model of what are the actors in the system, the useful (desired) and harmful properties of the system.

Furthermore, one should try to extend the boundaries of vision and see not just the system itself, but other interacting systems. Then, going even wider and seeing macro-level ("metasystems") and micro-level ("subsystems"). And not just that - all this should also be extended in time to the past and the future.

TRIZ claims that there are certain principles of technical systems evolution, and the need of inventions stems from the fact that some parts of the system are not developed in harmony with some other parts. This tension essentially creates the problems we are trying to solve.

At this point one is supposed to understand which exactly problem is tried to be solved, and thus this problem then gets reduced to a simple model. Essential part of this is about finding so called contradiction between two certain properties (parts) of the system. Note that real-world problems usually are not that straightforward - there can be more than single conflicting pair of system properties, but it's claimed that most of the problems can be decomposed to simpler subproblems which would satisfy that duality requirement.

The next thing to realize is the exact locality of the contradiction in time and space (when and where exactly the conflict does take place) - so called operating zone.

Then, solving the problem is essentially about overcoming the contradiction, but not in just any way - while doing that one should aim for so called ideal solution.

Ideal solution is another central principle in TRIZ. What we are trying to achieve should happen "on it's own", i.e. desired new property of the system should be achieved without modifying the system at all (remember "the fastest code is the one you never run"?..)

Now, it's obviously not always possible, and in practice one would end with some kind of compromise anyway. But the point here is to build a proper mental setup, which would divert mind from aiming at compromises prematurely.

To achieve this ideal solution we try to identify existing system resources and then employ them in some way. If there is no easy way - we may try transforming them first.

Particular resources and their transformations are subject of rigorous classification in TRIZ - there are tables, guidelines, principles which are built as the result of analysis of classes of existing good solutions. Some of those may be directly applied to the the software development as well (even though I am skeptical about attempts to map all of them). But what is more important here is another (rather obvious) principle: borrow existing good ideas.

For that to happen one has to build the awareness (implicit "knowledge base") about what good software ideas are already out there. No, it's not just about Design Patterns, even though they are certainly have some place in this awareness.

This proccess has iterative nature. One should be ready to "restart" the chain of thinking in the case that particular original model does not really lead to the desired solution. In this case it means returning back to analysing the system (subsystems, metasystems) and either building a different model, or even trying to solve different problem. It's also important to constantly reflect on own thinking process and especially on the solutions which are being found.

Now, just for an example of how this kind of thinking could map to the activity of problem solving in software development world. There was this wonderful talk, "Self and Self" by David Ungar, which I highly recommend watching.

He mentions about how the idea of generational garbage collector (today used in some form in the most of modern virtual machines) visited him when solving performance problems in virtual machine of Self.

There are some parallels to observe:

Memory usage/CPU conflict during garbage collection (two useful system properties conflicting with each other)

Exploiting skewed probabilities (identify and use existing system resources)

Optimize by eliminating work (ideal solution)

Optimize what's important (operating zone)

Garbage collection should happen to free the memory, and it should not happen because it is time-consuming (contradiction)

"Generations" - birth, getting old, dying ("little fellas" method)

I dare to speculate that there is a chance that aforementioned mental devices could lead to this idea. Even though it's a pure speculation, it's still interesting.

According to the authors, TRIZ "works" because it helps to chunk the solution space and direct the mind of inventor towards good solutions, shifting the focus away from the "bad" ones.

Metaphorically, if problem solving is a game (so one is picking the "best" move out of many possible), then TRIZ is probably something like alpha-beta pruning with heuristics. It reduces the amount of solutions to examine by cutting away the whole "bad solutions branches" when searching the game tree. Essentially, "the algoritm of invention" is nothing else as an attempt to heuristically optimize the huge solution space search, which happens in human mind:

The conclusion is that in normal chess play, certain types of situation recur - certain patterns - and it is to those high-level patterns that master is sensitive. He thinks on a different level from the novice; his set of concepts is different. Nearly everyone is surprised to find out that in actual play, a master rarely looks ahead any further than a novice does - and moreover, a master usually examines only a handful of possible moves!

The trick is that his mode of perceiving the board is like a filter: he literally does not see bad moves when he looks at a chess situation - no more than chess amateurs see illegal moves when they look at a chess situation. [...] This might be called implicit pruningof the giant branching tree of possibilities. By contrast,explicit pruning would involve thinking of a move, and after superficial examination, deciding not to pursue examining any further.

Does it all mean that one can universally apply TRIZ or similar methodology to creatively solve programming problems?.. I don't really think so. Every person is unique in which mental processes work for them better.

Still, the described principles, such as: refining the problem, finding the contradiction and bending it to extreme, looking for an ideal solution and trying to employ existing resources of the system (possibly transforming them), borrowing the ideas from existing great software solutions, fighting the psychological inertia - all of them can be useful regardless.

I guess the process of developing one's own mental toolset is neverending and happens continuously through the career and life.

And this is just another tool. Who knows, maybe it will find its place.