Monday, January 23

More Evidence Does Not Always Help, but Always Schedule Followup

M

y boss called me into his office. He asked me if I thought project Indigo could be completed on time. This was a strange thing to ask. My position on the feasibility of Indigo was well known, I'd had stated it clearly to my boss on more than one occasion. Indigo was due to start next business day. Was he having last minute doubts? or was he afraid I would make trouble either with his superiors or the new hires?

I had found myself in an Asch conformity experiment. I had estimated Indigo as taking 5 years for a team of 7 and both the other senior developer and the project manager estimated that it would take 9 months for a team of 5. Even the 5 year estimate was a product of the Anchoring effect, Yet, despite every skerrick of experience and expertise telling me that this was a multi-year long project, I was starting to doubt myself. Everyone else was supremely confident that it would only take 9 months.

Under the intense gaze of my boss I did something that I later regretted. I suggested a compromise. The project manager had a meticulous and detailed schedule laid out which stated the first module would be completed in a month. If we measured how much work was done during the first month we would have a good idea how long the whole project would take.

My first mistake was suggesting a compromise in the first place. The problem wasn't with there being enough evidence that I was right, after all this was a reboot of Indigo, it had already failed once. So there was plenty of evidence from the original project for how long it would take. The failure of the original project was why we had a project manager in the first place. He was there to rescue the project. 

One of the problems with Indigo was that the team was so focused on convincing higher management to green-light the project that they never stopped to consider whether they should do the project at all. Except maybe my boss was asking himself that question now. If so, I blew my best chance at killing the project. I guess I will never know whether being more assertive would have shifted my boss's position on Indigo. On the other hand, maybe he just wanted to know whether he could count on me to be a 'team player'. Sounds a bit paranoid, however the company was very internally competitive.  

My second mistake was not locking in a review meeting right there and then, with evaluation criteria. I probably didn't have the role authority or influence to swing something like that for project Indigo, but I could, and did, do something similar for later projects. It was an important lesson to learn. If you don't schedule it there and then, it is not going to happen. Later I came across many projects that needed either an architectural spike or a technological spike, and post spike reviews are useful. In one case the review resulted in a major pivot and other cases ended up triggering minor adjustments.

How the meeting with my boss played out, was possibly not the best outcome, however because I hadn't completely caved, the other senior developer as well as the project manager were put in charge of Indigo while I was left in charge of project Teal, the project that Indigo was supposed to replace. It was also not the worst outcome; I could have been put in charge of Indigo. Teal was the ideal position. Close enough to the disaster to learn from it, but far enough away to escape the flames.

After a month I checked the work tracking application; the Indigo team had not completed any stories. I asked the project manager whether Indigo was still on track to be finished by the deadline. He said it was on track. I pointed out that the team hadn't finished anything. He told me they had partially completed many of the stories. I asked how he could say they were on target to meet the deadline when assessing progress on partially completed work was notoriously hard. He said that his years of experience and expertise as a project manager allowed him to know that they would make the deadline. I talked to my boss and brought up our previous discussion and pointed out that the project was not going well. He said is was early days and he had confidence in the project manager. 

After a year the Indigo still had not gotten QA to approve a single story. There were tasks the that had passed QA, but that was a whole other problem. Tasks are not suppose to be tested by themselves, they are suppose to be tested as part of the story they are a part of.

Indigo was experiencing the classic signs of having stories that were too big. Most of Indigo's problems I have seen to a lesser extent in other projects. However the words "lesser extent" are doing a whole lot of work in the previous sentence. There was one difference that was a difference of kind rather than a difference of scale, one mistake that doomed Indigo from the very start. A mistake that caused many of the other problems, and that mistake was in how the estimation was done. There were other projects were I was pressured to reduce my estimates, as if re-estimating would change the real completion time. There was only one project that I have ever been on where I was given the estimate I was supposed to come up with before hand.   

As a consequence the stories were like epics and the tasks where like stories. So I used the completed tasks to estimate progress. This was less than ideal, as the tasks were not comprehensive. There were lots of gaps. I came up with a pretty graph that showed that Indigo would be finished in eight years. Again I was making the mistake of thinking that I needed more evidence. More evidence wasn't going to help. 

I showed the graph to my boss. He said that they had learnt a lot from the first year, and they would do better this year. Also he did not think that it would take eight years. I asked how long he thought it would take. He didn't answer.

At this stage cancelling the project would cause too much embarrassment. I do not think that anyone involved actually ever intended to meet the deadline. They thought that it would slip its deadline by a few months and upper management wouldn't be too upset by a small delay. It ended up overrunning its deadline by more than a few months.

The second year the real scope was reduced as low priority functionality was jettisoned, while on paper the estimated scope expanded, as hidden functionality was discovered, new stories were added. Stories were split into small stories and management became alarmed by the aggressive scope creep. 

No more new stories, and no more story splitting, management ruled. This did not help. The stories became bigger and bigger with more and more tasks, their estimated size kept increasing. The scope creep continued.

The third year management decided that the team should split the stories into smaller chunks and re-estimate to the "real" values. The backlog was greatly improved, however the stories were still too big, they still had trouble getting stories through QA and the number of stories kept increasing.

The project manager was starting to panic, he was flying in from interstate every two weeks and the project seemed like it was never going to end. His reputation for bringing in projects on time was starting to suffer. He was complaining to anyone that would listen. Most of the initial team except for him were long gone.

I left for greener pastures. A few years later I met the final team leader of Indigo for coffee and he gave me the low down on what happened next.

After five years, a prototype was given to beta customers. The customers hated it and gave the team a long list of things that needed to be fixed.

After six years the project was cancelled and the developers where laid off.

I agree with my former boss. There was a lot to be learned from the trials and tribulations of project Indigo and the learning did not stop with the first year.


No comments: