Self-organization

A force of life
March 26, 2013

Self-organization is the determining element for agile teams and companies. To think and act accordingly requires the acceptance of evolutionary forces which are an inherent part of any social system, whether we like it or not.

This contrasts with the constructing model, in which a central masterminded instance designs the system and controls it. Broadly, one can say that self-organization relies on evolution and construction relies on command and obedience.

The affirmation of the evolutionary approach is rewarding, but will challenge traditional behavioral patterns even in the microcosm of a single team. For the entire - possibly huge - organization is the paradox of giving up central control in order to make the organization controllable, radical and requires open minds and the willingness to learn and improve - in particular from the leaders of the organization.

Leaders in agile organizations have the important function of actively giving self-organizational forces a direction. This seems to be conflicting with the evolutionary approach and to span the tension field even further, here are two quotes I found in [Malik 2009:137]:

The only things that evolve by themselves in an organization are disorder, friction, and malperformance. Peter F. Drucker

... the only possibility of transcending the capacity of individual minds is to rely on those super personal, 'self-organizing' forces which create spontaneous orders. Friedrich A. von Hayek [Hayek 2011:54]

Past and present

Fredmund Malik is pointing out in his work, that our current organized society developed itself during the last 150 years. In former times there were organizations, too, but they had more of an amplifying function which means that most of the people did more or less the same, e.g. shooting a weapon, transporting bricks, piling one brick on the other. In these organizations there were only some rare specialists like master builders or officers. As a consequence of the visibility of tasks and clear hierarchies the communication patterns were simple. Knowledge did not play a major role and only some commands were needed to move the entire organization: “fire”, “heave ho” and the like. [Malik 2009:8]

Of course, knowledge played a role at any time in history, but we have to see this in relation to the knowledge that is needed nowadays.

With the dawning industrialization a new type of organization arises. Reading and writing are essential skills even for simplest operations. Specialists are everywhere and not many people are doing the exact same things. Nearly everyone is speaking a specialized language. Knowledge and communication are essential and characteristic for this new organization. The world we live in, with engineers, computer scientists, psychologists, neurologists, marketing experts, genetic engineers and the like developed itself during the last 150 years.

But how can this new world be connected to the evolution principle?

The new world is the result of humans acting, but it is not made by a central constructing instance and in this way not the result of human design [Hayek 1969:33,35]

The connection is, 150 years ago as well as today no mastermind is able to overview this world, to construct it and to plan it. The system organized itself in evolutionary steps.

Complexity

A consequence of evolution is the increase of complexity which leads to enhanced regulatory capacity. But the better regulatory capacity also increases chances of misregulation. That makes the steering of complex systems incredibly difficult. But on the other hand, complexity is a prerequisite for outstanding achievements. [Malik 2009:17]

Complex systems function although no one is understanding the entire system. A autombile manufacturer is able to build a complicated and efficient car in serial production with high safety standards, although no one is understanding fully the entire construction or manufacturing process. A hospital is functioning although no single person knows all processes and all treatment procedures.

The central construction of such systems would be limited to centralized knowledge which immediatly would become a bottleneck for the organization.

It is irritating that a complex system can not be controlled by a central instance, still the system is self-sustaining.

Self-organization

Self-organization describes spontaneous - emergent - processes that bring a system into some kind of order and to sustain that order. That includes the reaction to internal and external changes, in other words the adaptability to new conditions [Bendel 2006:2]. The ability to self-organize is a property of the system even without the actors being aware of that ability. The essential feature of the self-organizing capability is given by the interactions between the actors.

Water being wet is an emergent property. A single H2O molecule can not be wet but a lot of interdependent water molecules will develop the property of being wet.

Pedestrians participating in crowded events will, when searching their path for different directions, develop streams in which they can move with the least possible collisions. This happens always although no one is planning this behavior.

The spontaneous development of new system properties will happen while these properties could not be predicted upfront when inspecting the actors or components of the system. The so developed new is not more or less but something different than the sum of the parts. While the actors or components are visible, their interactions are invisible and difficult to measure. In particular these interactions are the determining element in complex and self-organizing systems, they make the systems behavior non-linear and difficult or impossible to reproduce - small causes can have enormous effects.

We can find many examples in nature for strong self-organizational forces. The heart is a complex, self-organizing system. Single heart cells generate a synchronized electrical pulse excitation, leading to contractions in a steady rhythm.

The neurons of the human brain can be described. Even their communication protocols can be analyzed. But who, by looking at these components of the system called brain, could predict that from these neurons and synapses something like thoughts, feelings, music, dreams and consciousness will develop? And isn´t it astonishing that the total number of neurons is determined with our birth - no further will be added. Only the connections between the neurons and by that the possibility of interdependencies will increase during our lifetime.

Language develops in the same manner. The interdependencies within our brains continue in the outer world. Thinking and speaking is connected to each other and speaking - as only being one of many ways to communicate - connects our thinking with the thinking of others. The language we use is not the result of a central construction process with the grammar specified upfront. Language evolves and is able to function without a central constructing instance.

The opponent of the described ordering forces is chaos. In shape of turbulences and swirls chaos brings unpredictable dynamics and the moment of chance into the world. Complex self-organizing systems are situated along the border to chaos.

In human organizations stabilizing rules and routines will be put against the chaos. But the so developed structures exist only as long as they are being reproduced and confirmed by the actors. Too many rules solidify the organization and prevent the adaption to new circumstances [Bendel 2006:2].

Knowledge organizations are not able to survive without extensive degrees of freedom, because otherwise self-regulation, innovation and adaption to changing environments can not run fast enough and with the necessary quality.

The reaction to unforeseeable events affords ad-hoc negotiation, the breakdown of routines and goal-oriented development of new rules to modify current structures.

In a radical way one has to leave behind the idea that everything in the organization can be ruled and controlled [Bendel 2006:2f].

One has to withstand the reflex of responding to uncertainty and ambiguity with ever new rules, forms and regulations. Instead one has to trust in self-organizational forces and the emancipation of employees to take local and autonomous decisions where needed.

Still not all decisions are being taken autonomous on local levels. Self-organization can be accompanied with hierarchic structures. Lower levels build up higher levels and higher levels adapt with a slower pace. Lower levels will get their orientation from higher levels and not the other way around. In self-organizing systems impulse and input from lower levels will be integrated into decision processes. It is not the question, if centralized or decentralized decision patterns are better in self-organizing systems. Instead both patterns can be observed and are needed.

This is theory - but what can we do to give self-organizational forces a direction?

Cooperation and communication

For an organization to handle complexity in a proper way, communication and cooperation processes need to be in a good shape. If informations can not easily find their way and flow to the right actors, a major prerequisite for self-organization is violated which leads to a limitation of possible results. A culture of willingness to communicate and mutual understanding on all levels is essential. This will not happen without trust. Therefore, as an actor in a well functioning self-organizing system, say what you do and do what you say.

Transparency

Self-organization is about inspect and adapt. Try to make the invisible visible and operate on facts. Determine the success criteria of your endeavor and ensure that everyone is operating on the same figures. If you need to make assumptions, make clear what assumptions have been taken and try to verify them as soon as possible. Be realistic and radical in terms of going to root causes, do not play down anything.

Results, feedback and continuous planning

If you know your strategic goal and you have your success criteria, inject sensors into your process. On a regular basis inspect your results with regard to target achievement and to the actors behavior in the social context. These are two sides of the coin: adaption of behavior and target achievement are of equal importance. Only by maintaining these two dimensions, a social group can find its way for an upward movement and improve continuously. Based on the findings of the regular inspections you can plan for your next steps. Do not plan to far into the future, instead measure and inspect in short cycles if the planning leads to desired results and then adapt accordingly. You will find orientation and direction in the strategic goal that has been set before starting the endeavor.

Performance orientation

If the organization has a goal and fulfills a purpose, results have to be delivered. One of the most important tasks for the leaders of the organization is to define what results need to be delivered what a good and what an unacceptable performance is [Malik 2009:293]. Self-organizational forces will get their direction out of this setting.

In a social context often one has two options: routined good behavior or results oriented new thinking. If the organization has to survive in a dynamic environment, routined good behavior is not an option anymore.

The definition of great and unacceptable performance does not mean to set internal stimulation by leveraging incentive systems. Daniel Pink [Pink 2009] points out, that financial incentives will lead to worse and not to better performance when applied in complex environments. Incentives will improve exact the key figure where they are applied to, but everything else falls by the wayside.

Wohland and Wiemeyer explain that internal stimulation with incentives create mediocre results. They argue [Wohland and Wiemeyer 2007:62ff]:

  • Incentives are an element of control. Like all control elements they afford knowledge about the future. In a highly dynamic environment the future is characterized by surprises. As incentive contracts can not be adapted continuously, in dynamic environments they point very likely into the wrong direction.
  • Incentives treat all employees in the same way, without keeping individual talents, chances and risks into account. The result is the average performance that anyone can achieve if he really wants to, in other words, the average. That is less compared to what a talented person could achieve in her area of competence and not enough to compete with high performers.
  • High performance presupposes a culture of trust. High performers will stabilize such a culture by carefully ensuring that organizational and employee interests are compatible with each other. Incentives are a result of mistrustful thinking. In an incentive culture the adding of value has to be achieved and stabilized through control and rewards.
  • Motivation is a quality sign of good leadership. The need for internal stimulation through incentives is a hint that leadership still needs to be learned.
  • Internal stimulation blocks or hinders the resistiveness of the led persons because incentives invite for uncritical pursuing of centrally defined goals. This increases the lack of leadership that initially produced the need for internal stimulation even more. The participation of the led people in the definition of goals does not change this conflict, because participation with this regards is often theater communication.

Doing

Done work has a momentum. Let yourself be permeated by your task until a solution arises. Act, deliver, inspect and improve. In the agile movement this attitude is captured by a collection of nice sayings: “Cut through the noise by taking action” [Beedle and Schwaber 2002:27], “Deliver early and decide late”, “Do, do right, do better”.

This all comes down to, that in this complex self-organizing universe where always more is unknown than known, theoretical models are not enough.

Do the things that produce results now. Repeat.

Divergent-convergent thinking and responsibility

Explore and expand possible solution spaces for a given problem. In other words, think divergent and do it in a team before deciding for a solution. Don´t decide easily for the first and obvious solution. Diversity is something valuable. Bear with different perspectives from different team members for a while and then start to converge.

Possible techniques for divergent-convergent thinking are silent-brainstorming or the dynamics group decision making model by Sam Kaner [Kaner et al. 2007].

Personally I find it helpful to have a responsible person to speak the final word for strategic decisions at the end of such decision courses. In Scrum, for example, there is the Product Owner who speaks the final word for business relevant decisions. The IT-Architect will speak the final word regarding architectural decisions.

Use what is already working

Use what is working. Do not put everything all at once into question. Evolution will also not start from scratch every time.

The networks will follow their task

Our tasks determine our communication patterns. Make the task that has to be fulfilled absolute clear. This will lead to a task-oriented organization where teams will freely and on behalf of their tasks form themselves and keep that order intact. A shared sense of belonging will not arise from social events, like christmas dinners, but from shared goals and tasks [Bendel 2006:5].

Anti-fragility

Nassim Nicholas Taleb coined the term anti-fragility [Taleb 2012]. Anti-fragility does not mean robustness. Robustness is the ability of a system to resist changes without modifying its initial stable structure. Anti-fragility is more. It is the ability to become stronger through drawbacks, impacts or enhanced demands. If you lift 50 kilogram on one day, your body starts preparing to lift 51 kilogram on the next day. You become stronger because you are getting trained. The same is true for self-organizing teams. As long as the task of the team is clear, the team will start to orient itself on the task. The task is allowed to be demanding - but it has to be clear and the team must be emancipated to change structures and adapt.

Reflection

I want to close with a suggestion for a reflection exercise from Jurgen Appelo. As a leader, write down two lists: One list with decisions that your team can make without you and one list with decisions you will make without involving your team [Appelo 2011:118].

Think about. Act. Repeat.

References

[Appelo 2011] Jurgen Appelo, “Management 3.0, Leading Agile Developers, Develop Agile Leaders”, Addison Wesley 2011

[Beedle and Schwaber 2002] K. Schwaber, M. Beedle, “Agile Software Development with Scrum”, Pearson Prentice Hall, 2002

[Bendel 2006] Sylvia Bendel, “Grenzen der Steuerung: Umgang mit Komplexität und Selbstorganisation”, Referat für die KFH-Tagung “Kommunikation von Fachhochschulen” Luzern 2006, http://www.sbendel.ch/fileadmin/user_upload/PDF/IK_Vortrag_KFH.pdf

[Hayek 1969] Friedrich A. von Hayek, “Arten der Ordnung, Freiburger Studien, gesammelte Aufsätze”, Siebeck 1969

[Hayek 2011] Friedrich A. von Hayek, “Law, Legislation and Liberty”, Volume 1: Rules and Order, University of Chicago Press, 2011

[Kaner et al. 2007] Sam Kaner et al., “Facilitators Guide to Participatory Decision Making”, Jossey-Bass 2007, Second Edition

[Malik 2009] Fredmund Malik, “Systemisches Management, Evolution, Selbstorganisation”, Haupt 2009

[Pink 2009] Daniel Pink, TED Talk: “The puzzle of motivation”, TED 2009, http://www.ted.com/talks/lang/de/dan_pink_on_motivation.html

[Taleb 2012] Nassim Nicholas Taleb, “Anti-Fragility”, ZURICH.MINDS 2012, https://www.youtube.com/watch?v=zOUph1JpLlk

[Wohland and Wiemeyer 2007] Gerhard Wohland und Matthias Wiemeyer, “Denkwerkzeuge der Höchstleister”, Murmann 2007