The Dangers of Heroism

f you have been in software development for any length of time you have been there.

The Crisis

A crisis arises. The management announces that the crisis threatens the very existence of the team / department / business.

The team works overtime. The team guru comes up with a ingenious workaround. A patch is rushed through QA. The crisis is resolved and the team celebrates.

But should they be celebrating?

While everyone is rushing around putting out fires and inventing workarounds for the suddenly realized risk,

• No-one is thinking strategically.
• No-one taking advantage of opportunities for process and productivity improvement.
• No-one is developing their skills.
• Compromises are made that damage the long term viability of the project, program or business in crisis in order to resolve the short term crisis.
• Quality, maintainability, automated tests, and manual testing are usually the first things to go.
• Effort and energy that could be used to drive the project forward is instead being used to prevent the project from going backwards.

The Root Cause

The crisis happened for a reason. Unless root cause analysis is undertaken and follow up action designed to resolve the underlying issue is completed the crisis will reoccur. Chances are it already has occurred several times under different guises.

The standard response that is heard time after time to a suggestion of root cause analysis and a permanent resolution to the problem is "We do not have time". Of course the reason they do not have time is that they do not do root cause analysis therefore have to solve the same problems over and over again.

The trouble is that the players involved in this little drama may not be especially motivated to resolve the underlying causes. Though the business has suffered losses many of the actors involved may perceive gains.

Managers are seduced by the illusion of rapid progress. Team members get to be heroes.

 

The Manager

On the management side there is even a term for this pursuit of illusionary progress (management by crisis).

Unfortunately the possible outcomes from this strategy are

• the employees run around in mindless panic undertaking counterproductive activities
• the employees start ignoring management as they have lost all credibility (like the boy who cried wolf)

neither situation is conducive to long term business survival.

The Heroic Team Member

If a team member indulges in an all nighter not only can he/she receive praise but the resulting functionality guaranties future employment as other developers will have difficulty in fixing, extending or even comprehending that area of the code.

Conceived in a flash of creativity and brilliance by a mind suffering from an excess of caffeine and a deficit of sleep this solution is likely to be

• overly complicated,
• inconsistent both internally and
• inconsistent with the rest of the project and
• is unlikely to have unit tests.

Further more mesmerized by memory of the feeling of power and genius that was imprinted by the surge of adrenaline that accompanied the birth of this monstrosity this hero will resist the idea of replacing it with something

• simpler,
• more reliable,
• more maintainable, and
• that has unit tests

The Confession

Now for a confession. I was once a hero, indulging in many of the counter-productive behaviors described above.

However I became tired of the fact that after saving the company, the company would get itself right back in trouble again. Convinced that there had to be a better way, I started thinking more long term.

• I found by fixing things permanently instead of a quick fix half solution every month or so
• I had more time to develop reusable solutions and tools which meant
• I no longer had to spend as much time with repetitive infrastructure, which meant
• I had time to learn techniques like test driven development, which meant
• I spent less time debugging and
• so on in a virtuous circle.

The more time I spent on self improvement and process improvement the more time I had to invest in further improvements.

Over time I found that I had to save the company less frequently.

I have experience with both ways of doing things and I can tell you resisting panicking over a short term a crisis and taking a longer view is better for your health and better for your employers long term prospects.

Update

This is an example of the drama triangle. In this scenario the manager who is managing by crisis is the Victim and the heroic team member is the Rescuer. Their efforts only achieves temporary pain relief, as the underlying cause is not addressed.

Related Posts

• Sympathy Considered- Harmful

A Tale of an Under-Resourced Project

Myrmidon Inc Part 1

dam was feeling the pressure. 12 months ago he was given a large project (Project Beowulf) and it was still no where near being complete. Carla his manager once removed had made a nasty remark about the time taken making it sound like Project Beowulf was the only project he was working on. Adam had repeatedly been pulled off Beowulf to work on higher priority projects.

In response to pressure from Carla (the product manager) , David (Adam's manager) assigned Ethan (a more experienced programmer) to help him. Ethan was told to complete some of the features that were part of Project Beowulf. Ethan reviewed Adam's work and made some design suggestions that he thought if implemented would make it easier for Adam to complete the project. He also identified several sections of duplicate code that if extracted into their own classes could be used for some of the uncompleted features.

Then Ethan found the real problem. Beowulf was supposed to work with Module Grendel. Grendel not have a well designed API. In fact it did not have a API at all. You just found a section of code that did something similar and copied that. Grendel was written by David under intense time pressure and David was considered a Hero for finishing Grendel.

Even though the Interface with Grendel was only a small part of the project, 90% of Beowulf related problems identified by QA where from Beowulf related additions to Grendel. Written with no thought to extensibility or maintainability anyone was going to have a hard time working with Grendel. Worse Grendel had no Unit Tests. Only some of the members of the team used Unit Tests and David was not one of them.

With pressure to complete the features that he had been explicitly assigned to, Ethan started  to give less help and time to Adam. Adam started to fall behind again. David started to question Adam lack of progress. Adam exploded claiming it was unfair that he was singled out. He started attacking Ethan claiming that Ethan's Project Icarus also overran its budget. He did this in Ethan's hearing and in the hearing of the rest of the team.

Ethan stopped helping Adam. He wrapped up the Beowulf features he was assigned to and they passed QA. Beowulf continued to ping pong between QA and Adams desk.

If you were interested in honesty there was plenty of blame to go around. However people interested in assigning blame are rarely interested in being honest. Adam was blamed for the overrun.

Adam was not a new employee, his strengths and weaknesses were well known. Project Beowulf was an order of magnitude larger than anything he had been asked to do before.

Shouldn't responsibility given to :-

• Carla and David for assigning a weak programmer to a task that was obviously beyond his abilities.
  
• David for the poor design of Grendel.
  
• Carla for demanding speed of completion of Grendel above all else.
  
• Carla and David for continually pulling Adam on and off the project.
  
• Ethan for not giving the help Adam needed.
  
• Adam for attacking a team member that was trying to help him.

This team had died and no one had noticed. Somewhere along the line they had become only a group of people who worked together.

I was Ethan in this scenario and I regret getting offended and pulling back when Adam verbally abused me. It wasn't his fault. The team was playing a game of hot potato and everybody knows that when you get the hot potato (blame) you pass it on as quickly as you can.

Feedback given badly

ptimally feedback should be

• given frequently,
• should be balanced between affirmative feedback and adjusting feedback,
• and feedback should be about specific behaviour that you want changed or continued.

Is this what usually happens in the average work place?

No.

Often feedback is

• saved until the annual review or not given at all,
• either all positive or all negative,
• said in anger so that the delivery obscures the message,
• the recipient  is given no information on how they can change their behaviour.

Anger can often get in the way of sound judgement. My father always told me never to show anger unless you are no longer angry. It is difficult to display the right amount of anger unless you are in a calm frame of mind.

Unfortunately the price of getting feedback wrong can be expensive.

Cost of delayed feedback

For ungiven feedback

• the unhelpful behaviour continues until it is given.

In a  company I am familiar with management decided to offer permanent positions to two probationary programmers even though they were unhappy with their performance. They did so because they thought that they did not have time to find and train new programmers. There was an important release coming up. There is always an important release coming up. Do I need to tell you that it did not turn out well?

Cost of unconstructive feedback

Looking back at the long list of incidents that I have observed were feedback was saved up over a long period to be delivered on batch or targeted the recipient’s character, intent or I.Q. I can discern a pattern.

For unhelpful feedback

• Productivity always decreases dramatically
• The recipient never thinks they deserve it (even when they do)
• It creates hostility and ill feelings.
• The recipient often redirect their energies into non-work related activities where they can receive the praise that they do not receive at work.

People react to such negative appraisals in different ways. They always react badly but badly in unique ways. Some reactions I have seen are :-

• spending a lot time at work talking on the phone with other potential employers.
• a nervous breakdown.
• skipping work without calling in.
• writing insults directed at their supervisor into the source code.
• start a computer hardware business on company time.

In one company I am familiar with the ill feeling caused by unproductive negative feedback was so bad that the project manager told the out of favor programmers to have a day off so that they would not 'contaminate' a new employee.

Summary 

My advice is to address any problems as soon as they come up. Do not wait for an annual review. Address the problematic behaviour not the person. Encourage them to overcome the problem giving appropriate assistance if needed. Do not insult them or try to scare them. If it is clear that coaching with appropriate feedback is not helping, and their behaviour is not acceptable you need to terminate them.

Once the working relationship has broken down the situation can become uncomfortable even for bystanders. Mediating between warring co-workers becomes a series of temporary patches that resolve nothing. The situation often drags on and on negatively impacting the team.

Sashimi

he book Agile Project Management with Scrum discusses the concept of Sashimi where software functionality is implemented as tiny packets of micro-features. This struck a cord with me.

The idea is that each small chunk of functionality is completely implemented end to end, including

• tests,
• documentation,
• data migration,
• and deployment mechanism.

Done, done, done in agile terms. Each micro-feature needs to be completed before moving on to the next sashimi slice.

In the bad old days I used to concentrate on getting the mechanics working and leaving cosmetic issues and documentation until later. By the time I finished the mechanics of a cluster of features I would have a long list of lower priority tasks I would have to work through.

Often I would be pulled off in the middle of a project to deal with a high priority task which in turn was interrupted by even higher priority task.

The backlog of partially finished work often became unmanageable. I tried to resolve these issues by breaking tasks into smaller pieces and limiting the dependences between the new functionality and the old so I could switch in the new feature seamlessly.

After reading Lean Software Development's comparison of partially finished work to inventory, I started reducing the number of tasks I had open at the same time and reducing the size of tasks even further. After reading the extreme programming material on Ward's Wiki I redefined my definition of 'Done' and my definition of what was important, to include everything that was necessary to deploy to the end-user.

This did not happen overnight but was an evolution driven by hard won experience and encouraged by exposure to ideas that resonated with that experience.

With the concept of sashimi I have another idea that resonates with my experience, and that helps me bring into sharper focus the direction in which I wish to improve my software development processes.

Random Quotes

have this habit of jotting down quotes that strike my fancy in my teams wiki. No-one else seems to pay much attention to them however I find it useful to re-read them every now and again.

Quotes and Concepts

• 25-to-1 productivity ratio: the observation that there is a 25-to-1 productivity ratio between the best and worst developers. Some sources insist it is 64 to 1 or higher. Fred Brooks?
• 6 degrees of Separation
  the idea that everyone on earth is connected to everyone else by a few intermediary relationships from an experiment by psychologist Stanley Milgram in the late 1960's involving a chain letter passed from Omaha, Nebraska to Boston, Massachusetts
• Chasm between Early Adopters and the Early Majority
  Geoffrey Moore Crossing the Chasm
• Code Smell
  a hint that something has gone wrong somewhere in your code More
• Conway's Law: - commonly stated as ``If you have four groups working on a compiler, you'll get a 4-pass compiler''. The original statement was more general: ``Organizations which design systems are constrained to produce designs which are copies of the communication structures of these organizations.'' We might put it more succinctly as ``The means determine the ends'', or even ``Process becomes product''. More
• Cost of Variety - Industrial cost goes down by about 15% to 25% per unit every time volume doubles, but goes up by 20% to 35% when variety doubles.
  George Stalk
• Empirical vs. defined processes
  Consider making cookies. You have a recipe, and you follow it. If you make another batch, with the same ingredients, in the same proportion, in the same oven, you expect to get the same result. This is an example of a defined process.
  Consider instead the process of creating a cookie recipe. The value comes from its originality. You might try a number of variations, to get just the right result. Iteration is inherently part of the process; this is an empirical process.
  See Methods and Tools Winter 2002 page 10
• Estimates
  • estimates prepared by those who will do the work are better than estimates prepared by anyone else
    Nine Management Guidelines For Better Cost Estimating. Communications Of The ACM 35:2 (Lederer 1992)
  • the people most competent in solving the task should estimate it
    (Jørgensen 1991)
  • Studies have shown that we are best at estimating when comparing things that fall within one order of magnitude
    • Improving Subjective Estimates Using Paired Comparisons. IEEE Software, 18:1 (Miranda 2001)
    • Multi-criteria Decision Making: The Analytic Hierarchy Process (Saaty 1996)
  • There is evidence that an estimate based on a developer's intuition is more accurate than other, more analytical approaches.
    Empirically Guided Software Effort Estimation. IEEE Software 17:6 (Johnson et al. 2000)
  • Popular method of agile estimating is by playing planning poker Planning Poker (Grenning 2002)
  • averaging individual estimates leads to better results (Höst and Wohlin 1998)
  • group discussions of estimates leads to better results
    A Review of Studies on Expert Estimation of Software Development Effort (Jørgensen and Moløkken 2002)
  • we are better at estimating “this is like that” than we are at estimating the absolute size of things
    (Lederer 1998; Vicinanza 1991)
• Form follows function - Louis Sullivan
• Form follows failure: "the form of created objects is always subject to change in response to their real or perceived shortcomings, their failures to function properly" Henry Petroski - The Evolution of Useful Things
• Freezing Design As soon as one freezes a design, it becomes obsolete
  Fred Brooks
• Fundamental Attribution Error
  human beings invariably make the mistake of overestimation the importance of fundamental character traits and underestimating the importance of situation and context.
• Interruptions
  managers are able to work an average of only five minutes between interruptions
  Time Power (Hobbs 1987)
• Parkinson's Law
  Work expands so as to fill the time available for its completion (1957)
• Planning
  • Planning is everything. Plans are nothing.
    Dwight D. Eisenhower
  • No plan survives contact with the enemy.
    Field Marshal Helmuth Graf von Moltke
  • A good plan violently executed now is better than a perfect plan executed next week.
  • If you tell people where to go, but not how to get there, you’ll be amazed at the results.
    General George S. Patton
• Queuing Theory
  • Little's Law
    states that in a stable system, the average amount of time it takes something to get through a process is equal to the number of things in the process divided by their average completion rate
• Reaction to a new methodology
  three-step sequence of reactions that meets the introduction of a new methodological principle: (1) “it’s trivial”; (2) “it cannot work”; (3) “that’s how I did it all along anyway”. The order may vary from Object-Oriented Software Construction by Bertrand Meyer
• Rule of 150
  150 is the maximum number of individual with whom human beings can have a genuinely social relationship. Groups work together more effectively when they are kept under the 150 member limit. From the theories of anthropologist Robin Dunbar
• SNAFU Principle: ``True communication is possible only between equals, because inferiors are more consistently rewarded for telling their superiors pleasant lies than for telling the truth.'' More
• Transactive Memory
  Close knit groups develop joint memory where each member is responsible for remembering a different portion of the joint memory - Daniel Wegner
• Triumph of Truth A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it.
  Max Plank
• User Stories
• Aspects
• alliteration - Card, Conversation, and Confirmation - by Ron Jeffries
• Attributes
• acronym INVEST -Independent, Negotiable, Valuable to users or customers, Estimatable, Small, Testable - by Bill Wake (Extreme Programming Explored)
• Zero One Infinity
  Beware of arbitrary constants or limitations. The maximum size of a collection should be either Zero, One or Infinity - Professor van der Poel