Imagine this scenario...
You and three friends stumble upon a great idea for a tech start-up. Because you're all talented developers and there are no "business" people to constrain you, the only thing that separates you from monumental success is just building the darn application. (seriously, it's a great idea). So you decide to all sit in a room to brainstorm the initial architecture, then you'll divvy up the pieces and get started.
I've lived this scenario a handful times, and have seen it too many to count.
What you'd expect to happen is this: each developer will come to the table with a set of preferences on what the architecture should look like, based on their unique knowledge and experience. All of these individual preferences would then be melded together, and the resulting architecture would represent some sensible compromise or average of the pre-discussion opinions.
What actually happens, however, is the exact opposite. Instead of ideas being pushed toward some average, they get pushed to the extreme. Cutting edge technologies, patterns, and approaches are chosen over relatively stable ones, and before you know it, you're programming the system in Clojure using a NoSQL database built using Gant, none of which anyone has any experience with. Two months into development the team realizes that it spent most of its time learning the new technologies, and very little has been actually produced. At this point the "window" has been missed, and everyone goes back to their old lives.
Some of what's going on here is the law of group polarization:
"Members of a deliberationg group predictably move toward a more extreme point in the direction indicated by the members' predeliberation tendencies" (Sunstein)
This phenomenon has been consistently replicated in a number of studies: conservatives meeting with other conservatives form more conservative beliefs, liberals more liberal, racists more racist, and so on. It's why political parties hold conventions before elections, and why niche discussion boards on the internet end up going completely off the rails. Like-minded peoples' preferences push toward the extreme.
The cause of this phenomenon seem to be two-fold:
First, people want to be perceived favorably by others, and so once they hear what others think, they adjust their opinion accordingly, to fit in with the group. In terms of the original example, who wants to be the guy who suggests to write the new application in old, boring Java? Come on man!
Second, people will share evidence in support of their opinions, which will further influence others in that direction (in which they are already leaning anyway). Essentially, the argument pool will be limited to evidence on only one side, and so this preponderance of evidence will push everyone even further.
Back to software development, group polarization happens often, potentially any time some relatively homogeneous faction (e.g. developers, managers, business owners, etc.) meets in isolation and makes decisions without input from others. The antidote, as would be expected, is inviting diversity...which is one reason that Agile works so well, since it puts developers in the same room with business owners. Although there may be disagreements, the resulting decision is typically some compromise.
Anyway, I find it helpful to be on the lookout for group polarization. And if I ever do a start-up again, I may be that lame guy who suggests Java!
