Apr 13, 2010

Learn yourself some innovation

Looking at today's programmer job positions, a lot of them, along with those usual lists of funky acronyms, include a somewhat confusing line: "strong problem solving skills."

To measure "amount of experience" in any concrete technology is already a tricky business, so how does one go with such an abstract thing as "problem solving ability"?

In software development there usually are many problems to solve, and for each of them there are many possible solutions. So it's not just about finding a solution, but rather about consistently picking the right solutions from many possible (and picking the right problems, for that matter). At this point we start talking about creativity and innovation.

The question is: is it possible at all to develop such a skill, in a deliberate way? Is this a personality trait, or intuition, experience, knowledge... or all of these?..

Turns out, people were trying to answer the question for ages. Socrates, Descartes, Poincare and many more researched the topic of creativity, innovation and problem solving, looking for answers to the same question: how good ideas are born in human mind?

The particular research which I've been poking into for a while, is called "Theory of Inventive Problem Solving" (aka TRIZ). The author, Russian inventor Genrich Altshuller, having worked in a patent office (it's interesting that Einstein had similar experience) and examined thousands of technical inventions, embarked on a task of finding some common traits in all these seemingly unrelated innovative ideas.

Essentially, he was reverse-engineering - first the good technical solutions themselves, and then mental processes, which could lead to such solutions emerging as an idea in one's head. And he came up with some core principles, building them later into his methodology, which managed to become quite recognized.

As with any good and recognized methodology which goes mainstream and gets the potential to commercialize (it's claimed to have been used at Xerox, Jonson&Jonson, Ford and other industrial giants), it eventually became somewhat complex and confusing, quite often being diluted with a snake oil (must...not...mention...agile...).

However, the core principles are simple and sound.

The first step is to understand the environment (system) in which problem is being solved. This includes building a mental model of what are the actors in the system, the useful (desired) and harmful properties of the system.

Furthermore, one should try to extend the boundaries of vision and see not just the system itself, but other interacting systems. Then, going even wider and seeing macro-level ("metasystems") and micro-level ("subsystems"). And not just that - all this should also be extended in time to the past and the future.

TRIZ claims that there are certain principles of technical systems evolution, and the need of inventions stems from the fact that some parts of the system are not developed in harmony with some other parts. This tension essentially creates the problems we are trying to solve.

At this point one is supposed to understand which exactly problem is tried to be solved, and thus this problem then gets reduced to a simple model. Essential part of this is about finding so called contradiction between two certain properties (parts) of the system. Note that real-world problems usually are not that straightforward - there can be more than single conflicting pair of system properties, but it's claimed that most of the problems can be decomposed to simpler subproblems which would satisfy that duality requirement.

The next thing to realize is the exact locality of the contradiction in time and space (when and where exactly the conflict does take place) - so called operating zone.

Then, solving the problem is essentially about overcoming the contradiction, but not in just any way - while doing that one should aim for so called ideal solution.

Ideal solution is another central principle in TRIZ. What we are trying to achieve should happen "on it's own", i.e. desired new property of the system should be achieved without modifying the system at all (remember "the fastest code is the one you never run"?..)

Now, it's obviously not always possible, and in practice one would end with some kind of compromise anyway. But the point here is to build a proper mental setup, which would divert mind from aiming at compromises prematurely.

To achieve this ideal solution we try to identify existing system resources and then employ them in some way. If there is no easy way - we may try transforming them first.

Particular resources and their transformations are subject of rigorous classification in TRIZ - there are tables, guidelines, principles which are built as the result of analysis of classes of existing good solutions. Some of those may be directly applied to the the software development as well (even though I am skeptical about attempts to map all of them). But what is more important here is another (rather obvious) principle: borrow existing good ideas.

For that to happen one has to build the awareness (implicit "knowledge base") about what good software ideas are already out there. No, it's not just about Design Patterns, even though they are certainly have some place in this awareness.

This proccess has iterative nature. One should be ready to "restart" the chain of thinking in the case that particular original model does not really lead to the desired solution. In this case it means returning back to analysing the system (subsystems, metasystems) and either building a different model, or even trying to solve different problem. It's also important to constantly reflect on own thinking process and especially on the solutions which are being found.

Now, just for an example of how this kind of thinking could map to the activity of problem solving in software development world. There was this wonderful talk, "Self and Self" by David Ungar, which I highly recommend watching.

He mentions about how the idea of generational garbage collector (today used in some form in the most of modern virtual machines) visited him when solving performance problems in virtual machine of Self.


There are some parallels to observe:


  • Memory usage/CPU conflict during garbage collection (two useful system properties conflicting with each other)
  • Exploiting skewed probabilities (identify and use existing system resources)
  • Optimize by eliminating work (ideal solution)
  • Optimize what's important (operating zone)
  • Garbage collection should happen to free the memory, and it should not happen because it is time-consuming (contradiction)
  • "Generations" - birth, getting old, dying ("little fellas" method)

  • I dare to speculate that there is a chance that aforementioned mental devices could lead to this idea. Even though it's a pure speculation, it's still interesting.

    According to the authors, TRIZ "works" because it helps to chunk the solution space and direct the mind of inventor towards good solutions, shifting the focus away from the "bad" ones.

    Metaphorically, if problem solving is a game (so one is picking the "best" move out of many possible), then TRIZ is probably something like alpha-beta pruning with heuristics. It reduces the amount of solutions to examine by cutting away the whole "bad solutions branches" when searching the game tree. Essentially, "the algoritm of invention" is nothing else as an attempt to heuristically optimize the huge solution space search, which happens in human mind:
    The conclusion is that in normal chess play, certain types of situation recur - certain patterns - and it is to those high-level patterns that master is sensitive. He thinks on a different level from the novice; his set of concepts is different. Nearly everyone is surprised to find out that in actual play, a master rarely looks ahead any further than a novice does - and moreover, a master usually examines only a handful of possible moves!

    The trick is that his mode of perceiving the board is like a filter: he literally does not see bad moves when he looks at a chess situation - no more than chess amateurs see illegal moves when they look at a chess situation. [...] This might be called implicit pruningof the giant branching tree of possibilities. By contrast,explicit pruning would involve thinking of a move, and after superficial examination, deciding not to pursue examining any further.

    Does it all mean that one can universally apply TRIZ or similar methodology to creatively solve programming problems?.. I don't really think so. Every person is unique in which mental processes work for them better.

    Still, the described principles, such as: refining the problem, finding the contradiction and bending it to extreme, looking for an ideal solution and trying to employ existing resources of the system (possibly transforming them), borrowing the ideas from existing great software solutions, fighting the psychological inertia - all of them can be useful regardless.

    I guess the process of developing one's own mental toolset is neverending and happens continuously through the career and life.

    And this is just another tool. Who knows, maybe it will find its place.

    Apr 2, 2010

    Getting a good habit, the easy way

    I consider myself an extremely unorganized person, which makes me sometimes wonder what the hell am I doing being a programmer.

    In software development, planning is an essential activity for getting the job done - something that I've learned the hard way - and personal traits must be essential in this regard.

    However, I've also learned that some of those traits can be actually gained with a bit of effort. Specifically, when we talk about habits, there are several mind exploits, which can be used to rewire the brain.

    Here's one particularly dead simple, nevertheless useful professional habit, which I managed to get during the last year: using ToDo list for everyday microplanning.


    As with any good habit, there are several things which can help:
    • should be simple to do, non-burdened with time-wasting activities
    • consistent, repeatable in a simple ritual-like steps 
    • providing positive feedback
    • have some triggering mechanisms  
    • be forgiving to yourself, no giving up early because of false starts
    • some kind of public commitment, even artificial one, can also help
    I use AbstractSpoon  ToDoList (it's really nice and free but Windows-only, unfortunately... however there are many Linux alternatives - the particular application does not really matter).

    There are some simple things to make its usage feel streamlined:
    • system-wide shortcut to toggle the application window quickly (Ctrl+Shift+T in my case)
    • it starts on system start and resides as an icon in the system tray
    • auto-saves on window losing focus
    • I use dropbox as a simple way to share the tasklist between different computers.
    I start every working day with a simple ritual:
    • open ToDoList with keyboard shortcut and create the new top-level task named by the current date (e.g. "30 mar").
    • look at the previous day's ("29 mar") undone tasks, and either move them to the new day, or to the special "Maybe Later" top-level task.
    • think about what else is to be done today (usually there is some higher-level issue tracking system, such as Trac or JIRA, which already has a list of issues assigned to me and can serve as a guide), and roughly populate "30 mar" with today's goals.
    • look into the "Maybe Later" box and consider if there's something that can be fetched from there for this day. Also, close those of the entries there which are not actual anymore.
    • the current day's goals should be pretty fine-grained (I feel more or less comfortable when there are around 10 of them, doable during the working day), but it's not necessary to go into all the details now, they can be refined later, when it actually comes to doing them.
    • assign priorities to all the entries (10 being the highest), and they get automatically sorted, topmost being with the highest priority.
    • close the task list and start working on the topmost task.
    During the day the shape of the list can change - and it's fine.
    Some new things may pop in, in which case they get dispatched correspondingly:
    • if it's an urgent emergency task, then it gets right to the top of the current task list, with the highest priority.
    • if it is something which needs to be done, but not necessarily today, then it gets into "Maybe Later" box.
    • if this is just a "nice to have" kind of thing, then it gets into the special "IDEAS" box, which is located below the "Maybe Later" one.
    • some tasks can happen to become obsolete during the day, then they just get crossed out.
    Also, priorities of the tasks can change during the day - I also consider it to be an expected phenomena.

    Whenever this "popping in", or any other kind of distraction (quite usual on the working place, unfortunately) happens, there is always this magical "what was I doing" keyboard shortcut, which helps to switch the context back and focus again at the task at hand.

    It did not work all that well from the first attempt, of course, but eventually I managed to get into this simple everyday rhythm and started to feel comfortable with it.

    There is this inherent sense of satisfaction in seeing several things crossed out at the end of the day, and being able to answer these simple, yet important questions: "what have I done today? is there anything exceptional about it? why not, exactly?"

    I believe that this manner of dispatching tasks somewhat resembles the Getting Things Done method.

    There are few "buckets" of prioritized tasks, which periodically get processed, with the "short term" bucket being a list of today's tasks.

    It's not exactly the same, of course. But perhaps I will try to follow the GTD method more closely, eventually.

    After all, it's just another useful habit to try developing. There is no harm in trying it, and no penalty if it does not work for me.

    Doing the impossible things (such as changing one's personality traits, for example) turns out to be not that hard - with a little bit of persistence and open-mindedness.