Executive Management at my wife's old company claimed to be doing Six Sigma.  What was actually happening was that people invented measurement data and shuffled paper.  The final reports to Executive Management were almost pure fiction.  

Keep in mind that Six Sigma and CMM Level 5 have similar styles and similar goals.  They are both very expensive, and are really only applicable in very limited circumstances.  Capital-A Agile or CMM Level 3 are both more rational approaches that fit a wider variety of circumstances.  

+++++++++++++++++++++++++++++++++++++++++++++++++++

David

I did Six Sigma for GE in a manufacturing environment, and they were also trying to expand the intiative into what I would call "people  processes". IMHO, it don't work. Sixx Sigma Sixx Sigma

+++++++++++++++++++++++++++++++++++++++++++++++++++

Alan

One aspect of Six Sigma is the determination of the part of the process that produces defects (as in "defects per million opportunities").  When there is no consistent process, i.e., every developer has his own way of doing things, it would be nigh onto impossible to determine what part of the process failed and why.

If an "opportunity" were defined as a line of code (a bad idea), the way any given line of code was developed would be very hard to nail down; to know why that line of code ended up being faulty probably next to impossible.  Typically, it would probably be very hard to find out who wrote it so you could ask what happened.  There are certainly millions and millions of widgets (e.g., lines of code or whatever; there is lots of software) and the defect rate is very high.

I'm in favor of fixing what we have instead of adding fixes to old stuff and then adding new stuff so you can sell the fixes.  I'd rather just buy the fixes, thank you.  An annual maintenance fee of 20% of the original purchase cost seems reasonable to me.

In Next Post

I know nothing of the Six Sigma for Software idea, however, I did work for a systems company that was acquired by a Six Sigma company and I had the opportunity to have Six Sigma training and I implemented a Six Sigma project improving the system readiness at ship-time.  The project was successful at determining why systems were not ready on time and how to track the activities necessary to make sure the system was readier.  (There were many reasons systems were not ready and the project basically reduced the number of things not ready on any given system.)

I did not stay with the company long enough to see the long term results of this project, but the short term results were good.  The "Control" phase (of DMAIC) was in place and it was then up to management to be sure that the measurements were made and the appropriate, specified actions taken.

I expect that there will be reluctance from the software development people for such methods are not consistent with the current software development culture.  The idea of root cause analysis, for instance, is certainly alien to that culture.  If your management team has been grown from software development, it will be a very hard sell.

There is also the problem of false metrics.  Finding meaningful measurements in a hostile crowd is a difficult challenge.  My case was easy.  The metric was the number of systems that were ready for formal acceptance when the customer arrived.  The initial result was 0%.  Sound familiar?  We had one customer that was forced to accept because of limitation of financing.  Story I heard was they took him golfing to make peace with him.  I heard he threw his golf clubs at the relevant manager.  Wish I'd been there, but the strain of not cheering might have been too much for me.  I was demonstrating LAN  failure recovery  to him when the system failed catastrophically.  They were downloading a new image during my acceptance demonstration test.  <sigh>

Another bad example: I worked on a project that used the reported rate of unique Sev 1 and Sev 2 defects.   I found that there was pressure to downgrade the severity ratings and the database was actually changed to reduce the number of such reported defects.  When I asked about why it was changed, I was told, "I didn't know anybody used that."

+++++++++++++++++++++++++++++++++++++++++++++++++++

James Bach

<<One aspect of Six Sigma is the determination of the part of the process that produces defects (as in "defects per million opportunities").  When there is no consistent process, i.e., every developer has his own way of doing things, it would be nigh onto impossible to determine what part of the process failed and why.>>

But Alan, this is exactly what's wrong with Six Sigma thinking: everyone DOES have his own process. To think otherwise is to propose that humans are interchangeable rule-following engines. Maybe brick-laying is like that, I don't know, but software development certainly is not. Nor can it be made into a process like that.

To those with a basic understanding of social sciences, or even just any naïve observer, it certainly is possible to examine many human cognitive activities and discover how they probably went wrong. Somehow, the Six Sigma folks have talked themselves into deep ignorance about how people operate.

I think it may be because they insist on oversimplifying their models of processes to the point where those models become amenable to the simple mathematics they prefer to play with.  What you are apparently saying is "nigh onto impossible" is, to put it bluntly, what you and I actually do nearly effortlessly, every day, as we make our way through life: we interact socially, and draw successful inferences about our fellow humans.

+++++++++++++++++++++++++++++++++++++++++++++++++++

Ward Cunningham

+++++++++++++++++++++++++++++++++++++++++++++++++++

Adam

I tend to agree that Six Sigma is best designed for manufacturing though have no empirical evidence to back up this opinion. What I can contribute is an article I read last night from Business Week about 3M's experiences implementing it to another human-oriented task, Innovation. The article is conveniently online at Business Week A nice section is
<quote>
Indeed, the very factors that make Six Sigma effective in one context can make it ineffective in another. Traditionally, it uses rigorous statistical analysis to produce unambiguous data that help produce better quality, lower costs, and more efficiency. That all sounds great when you know what outcomes you'd like to control. But what about when there are few facts to go on—or you don't even know the nature of the problem you're trying to define?
</quote>

Then there is The Big Idea: Six Stigma

<quote>
It could be that programs such as Six Sigma undermine individual contributions to the company. After all, it's hard to feel as if you're doing something meaningful when the language of the workplace reduces you to a cog in an inhuman profit machine. But increasingly, top-level managers are groaning too, and that's because there is evidence to suggest that the numbers don't stand up to scrutiny.
</quote>
and
<quote>
When a company relies too heavily on meaningless defect counts and quarterly earnings reports, he wrote in Harvard Business Review, it is "not unlike a well-tuned car that heads over a cliff."
</quote>

I'll finish this with a link with the opposite bias. iSixSigma.com "is the world's leading online content provider for the Six Sigma community" and has forums, a print magazine, etc. http://software.isixsigma.com/ is their software six sigma section

+++++++++++++++++++++++++++++++++++++++++++++++++++

Dr. Cem Kaner

The fundamental problem with applying Six Sigma to software is its foundational mathematical model. We can compute meaningful probabilities when we have a stationary probabilistic process--for example, the error rate in manufacturing defects in widgets. We can then test whether a variation in manufacturing (or design or whatever) process has affected error rate.

But in software, we are dealing with design quality control rather than manufacturing quality control. We are not looking for the same bug every build. (Or we shouldn't be.) We are looking for the bugs we missed before, and those requires tests of new combinations of sequences and values. Each failure becomes the result of a unique test. I have read a lot of the software reliability literature and I just do not believe that any of the current probability models describe what we are dealing with, with software errors.

There's a lot of money to be made selling the cargo cult (SickSigma) to people who don't understand applied mathematics. And several of the pricey "green belt" crash courses don't give much foundation in applied math or require much on entry. A colleague of mine, who was instrumental in bringing Six Sigma to Harris Corp (GE/Harris to those of you who understand how much of the theory of six sigma comes from GE) bemoans the mathematical insufficiency of many of the miracle-short-course green belts and black belts in sickSigma. If you can't assess the validity of your underlying probabilistic model, and you can't replace a bad one with a better one (many, many processes are not "normal" (gaussian) and are better modelled with other distributions and other statistics), and you don't know enough about measurement theory to realize that you're whistling in a sewer, then you're not likely to collect, summarize or assess data in a way that is useful (unless "useful" = "impress executive who doesn't understand either").

+++++++++++++++++++++++++++++++++++++++++++++++++++

Matthew Heusser

They was a guy in my graduate cohort that did his capstone thesis on sixx sigma for software.  Basically, his conclusions were similar to Alan's or Ward's: If you try to do six sigma for software IN THAT WAY, you will fail.

He did, however, have a really cool, actually helpful example of six sigma for software.  Basically his take was this:

Let's say you have software that supports a huge number of transactions.  Say, Amazon.com, Google, or a big database. 

Every now and again at Amazon, a checkout fails.  Some of the things may be beyond Amazon's control (a dial-up connection goes caput, a three-year-old unplugs an ethernet cable to see what it does) and some of them are completely under amazon's control. (A bug or a lack of redundancy for a component)

The idea with Six Sigma for Software would be that less than one per million of these logical transactions would fail.

So what you do is count the failures, assign each failure a root cause, list the failures in MS Excel, and click "SORT."  Then you work on the buggiest features first.

You also count the successful transactions and the failed ones to get some idea how you are doing.

----> You see, that's how a computer program works - it does the same thing, every time.  In that way, it is possible that we measure the failed transactions, fix root causes (bugs), and watch the transaction success rate shoot up.  It's a disciplined, decent idea, but it's more like a magazine article than a book.

The problem comes when we are counting bugs in a non-transactional system, because then we have to weigh something like "spelling errors in a dialog box" vs. data corruption vs. application crash vs. system crash.  In my experience (and, I should add, according to Phillip Nguyen's graduate research), that's when SixSix Sigma                              More Discussion>>>