Ben Northrop


Decisions and software development


A Room Full of Techies


(17 comments)
August 8th 2011


Imagine this scenario...

You and three friends stumble upon a great idea for a tech start-up. Because you're all talented developers and there are no "business" people to constrain you, the only thing that separates you from monumental success is just building the darn application. (seriously, it's a great idea). So you decide to all sit in a room to brainstorm the initial architecture, then you'll divvy up the pieces and get started.

I've lived this scenario a handful times, and have seen it too many to count.

What you'd expect to happen is this: each developer will come to the table with a set of preferences on what the architecture should look like, based on their unique knowledge and experience. All of these individual preferences would then be melded together, and the resulting architecture would represent some sensible compromise or average of the pre-discussion opinions.

What actually happens, however, is the exact opposite. Instead of ideas being pushed toward some average, they get pushed to the extreme. Cutting edge technologies, patterns, and approaches are chosen over relatively stable ones, and before you know it, you're programming the system in Clojure using a NoSQL database built using Gant, none of which anyone has any experience with. Two months into development the team realizes that it spent most of its time learning the new technologies, and very little has been actually produced. At this point the "window" has been missed, and everyone goes back to their old lives.

Some of what's going on here is the law of group polarization:

"Members of a deliberationg group predictably move toward a more extreme point in the direction indicated by the members' predeliberation tendencies" (Sunstein)

This phenomenon has been consistently replicated in a number of studies: conservatives meeting with other conservatives form more conservative beliefs, liberals more liberal, racists more racist, and so on. It's why political parties hold conventions before elections, and why niche discussion boards on the internet end up going completely off the rails. Like-minded peoples' preferences push toward the extreme.

The cause of this phenomenon seem to be two-fold:

First, people want to be perceived favorably by others, and so once they hear what others think, they adjust their opinion accordingly, to fit in with the group. In terms of the original example, who wants to be the guy who suggests to write the new application in old, boring Java? Come on man!

Second, people will share evidence in support of their opinions, which will further influence others in that direction (in which they are already leaning anyway). Essentially, the argument pool will be limited to evidence on only one side, and so this preponderance of evidence will push everyone even further.

Back to software development, group polarization happens often, potentially any time some relatively homogeneous faction (e.g. developers, managers, business owners, etc.) meets in isolation and makes decisions without input from others. The antidote, as would be expected, is inviting diversity...which is one reason that Agile works so well, since it puts developers in the same room with business owners. Although there may be disagreements, the resulting decision is typically some compromise.

Anyway, I find it helpful to be on the lookout for group polarization. And if I ever do a start-up again, I may be that lame guy who suggests Java!

I believe that software development is fundamentally about making decisions, and so this is what I write about (mostly). I've been building software for about 20 years now, as a developer, tech lead, and architect. I have two degrees from Carnegie Mellon University, most recently one in philosophy (thesis here). I live in Pittsburgh, PA with my wife, 3 energetic boys, and dog. Subscribe here or write me at ben dot northrop at gmail dot com.

Got a Comment?


Sign up to hear about the next post!

If you liked this article and want to hear about the next one, enter your email below. I don't spam - you'll only receive an email when there's a new post (which is about once a month, tops). It's all low-key, straight from me.

Comments (17)

August 08, 2011
I'm human.

This resonates with what i have seen in many instances in a recent case the entire team was for using and odata service in order to connect two systems.

Seems like an interesting phenomenon add to this the fact that programmers swear they are logical and it makes for quite a deadlock to have an actual real discussion.

Alfredo
August 08, 2011
This is an interesting idea, but is there any way to mitigate it?

It seems to me that group polarization is just another word for making contrarian, non-mainstream decisions. But you want to encourage that!

The problem is if the startup takes on unnecessary risk due to group polarization. Perhaps:

* Make learning costs/risks visible in your planning process. Somehow keep them realistic.
* Make it part of the group ethos that you never do something complex when something simple is good enough.
Mike Kain
August 08, 2011
Certainly, i believe we techies just want to go for best and the latest.

I was firstly building a webpage project for college using the basic notepad for html and css codes but after seeing how Dreamweaver can ease the things i shifted to it then later i just created a vector website using Fireworks.
Mike Kain
August 08, 2011
Certainly, i believe we techies just want to go for best and the latest.

I was firstly building a webpage project for college using the basic notepad for html and css codes but after seeing how Dreamweaver can ease the things i shifted to it then later i just created a vector website using Fireworks.
Dmitri
August 08, 2011
@Mike Kain: This is not necessarily a good sign. You should have a relatively complete knowledge of technologies at hand - and the best way to accomplish that is to use something like Notepad++. Not quite an IDE, but a step up from notepad (has syntax highlighting + code folding).
August 08, 2011
There's also the geek factor: a lot of techies and geeks always want to be surfing the wave of innovation. You know, Java is so old you need Clojure! Why join? NoSQL! :)

Ruben
Mark
August 08, 2011
I think it an issue with the type of techies you are with and the goals of the project. In my experience the projects where we mitigated unnecessary risk were the projects that developed the fastest and had the most time for testing and polishing the released product. We chose technologies that each developer had the most experience which helped speed things along. It took away a bit of the fun from discovering new and exciting technologies but creating a solid product in shorter time than expected was worth it.
Herp Derp MBA
August 08, 2011
Also remember that a group of programmers left to themselves will typically create the answer to the question that nobody asked. I know you all hate business types 'cause they can't speak nerd. And as a combo-MBA & programmer, I understand the frustration translating back and forth.

But you really need someone on the team who actually understands the market you're going after and how to develop a marketing plan and... most importantly....

...to keep you on track, efficiently making ONLY WHAT YOU NEED.
Dirty D
August 08, 2011
That is a long way to spell "circlejerk".
August 08, 2011
Hey! I just wrote my web app in Clojure :) It was a good experience and actually gave me good velocity compared to other options, in that my code matches nearly perfectly to my goals with no extra cruft.

Otherwise, this is a really interesting article. I am always thankful when I can be aware and keep these sorts of biases and tendencies mind. Now that I have a name for it, I can point it out in my own behavior and when working with others.

Thanks.
Alex
August 08, 2011
I don't understand this tendency at all. As a student, all of my programming has been done in Java. If I need anything else besides C/C++ or Java, I will let you know.
August 08, 2011
This has held true for me even when building things on my own. I'll always find this "cool new technology" that I'll want to incorporate and I'll spend so much time learning that I get bored with the original idea and go back to regular life.

The solution for me is to stick to what I know on projects, and learn new things in my free time. That way I'm always fairly close to the cutting edge, but I'm not wasting time on projects "learning".
Fred Fnord
August 08, 2011
Always invite a curmudgeon along to these meetings. And if he says you're being an idiot, listen to him. And if he says that the team could write the damn thing in perl and mysql in two weeks and it could handle 1 million unique visitors a day, or the team could write it in erlang and nosql in three months and it could handle 10 million unique visitors a day, then possibly 'up to a million users in the hand is worth 10 million in the bush'?
Ben
August 09, 2011
Thanks for all the comments!

@Neil - "Is there a way to mitigate it?". Yes. Beyond just encouraging diversity of thought, it turns out well established groups appear to be less susceptible to group polarization (perhaps, because the more you know people, the less you give a damn what they think about you! i.e. the social mechanism is less strong) Also, the more people identify with the group, the more they want to conform. For example, the more I see myself is a talented software developer, the more I'll want to do cool developer-type things (like use cutting edge technologies).

@Fred - Exactly. When I was young, I used to hate the curmudgeon - he was always such a downer, getting in the way of doing something fun. Now, I am that curmudgeon. :)

In general, the post was less about the merits/risks of using new technologies, and more about the phenomenon where like-minded people end up with more extreme beliefs when they all discuss together. It's not limited to techies either - I've seen a room full of "business" folks make decisions equally polarized, in the opposite direction - "just get 'er done!".
August 17, 2011
Good post. Especially the last line. I too am the lame guy who would suggest Java!
ps: how's it going?
Ben
August 29, 2011
Thanks Doug! Good to hear from you.
Grant
September 20, 2012
Boy, this couldn't ring more true for me!

I consider myself fortunate that I've often had a person who would not be shy about pointing out how ridiculous some of my ideas were (and how they provided little value to the problem being solved) when I was younger. Having that influence has been a great way to create an internal balance also (sometimes that group polarization effect occurs within me also!)

It's painful to hear the feedback, but ultimately it's necessary to converge on a good decision.

Nice post!

-Grant