Unclosed Loops and Developer Frustration


(7 comments)
October 16th 2017


Tolstoy once wrote, "all happy development teams are alike; every unhappy development team is unhappy in its own way". And after 20 years as a programmer, I can attest to at least one sure-fire route to unhappiness:

It's the beginning of the sprint. Everything's been planned, estimated (if that's how you roll), and assigned. You have a few features on your plate, and are eager to get going. Starting with the most critical one, you dig in for a day, get a plan together for how to tackle it, and begin coding.

Another day passes and you get about halfway through when the project manager swings by your desk. "Hey", he says, "I know you've already started, but a business priority has changed, and we need you to work on this different feature instead. Finish this new one off now, and then you can get back to your original work".

I'm not sure about you, but this drives me freaking crazy (at least it does when it happens often). But should it? I mean, I'm being paid by this company, and it's up to them to set their own priorities, right? If they decide there's more value in a different feature, that's their call. Further, I'll get back to the original feature eventually, and if I don't, who cares - I learned a few things in the process of digging in, and probably have a better grasp of the application as a result. What reason do I have do be frustrated? And yet I am...

This internal tension has plagued (or at least puzzled) me forever. Why do I hate leaving tasks unresolved so much? And I think its not just me - most of my colleagues seem to get just as frustrated when it happens to them. But why? Recently, while reading the book Irresistable, I found an explanation.

In psychology, there's a well known cognitive bias called the Zeigarnik effect. It states, simply, that an incomplete or interrupted task will be remembered better than a complete one. The phenomenon was coined by Bluma Zeigarnik, a psychologist who noticed that waiters had better recollections of orders that were unpaid. As soon as an order was completed, the waiter effectively flushed it from memory. In her paper Finished and Unfinished Tasks, she explains:

"When the subject sets out to perform the operations required by one of these tasks, there develops within him a quasi-need for completion of that task. It is like the occurrence of a tension system which tends towards resolution. Completing the task means resolving the tension system, or discharging the quasi-need. If a task is not completed, a state of tension remains and the quasi-need is unstilled."

This hits the nail on the head for me. When I start development of a new feature, there's a non-trivially strong force within me that wants to drive it to completion. If I'm asked to table that feature in favor of something different, my brain can't completely let go of the original task and so I can't fully focus on the new task at hand. The details of the original task are still churning in some background process in my brain.

Further, my guess is that the Zeignarnik effect is probably cumulative as well. The more tasks interrupted, the greater the tension and frustration. And for some, this is the rule rather than an exception. You get into work intending on finishing off some feature, but then get pulled into a meeting to discuss something different, and then get pulled out of that meeting to work on a production defect, and so on until the day ends and almost nothing was fully resolved. Ultimately frustrating.

Adjustments can (and probably should) be made in an organization to avoid this effect, since the effectiveness, mental clarity, and possibly even sanity of their developers depend on it. First, of course, we should strive to limit interruptions and let developers close open loops before forcing new ones upon them. Many teams have already started doing this through the adoption of Kanban and work-in-progress limits. Additionally though, in environments where interruptions are rampant, perhaps tasks could be designed to be as fine grained as possible, allowing developers to finish something before being pulled off to work on the next. Lastly, it's possible the best solution might just be to strive to overcome our innate mental wiring - when you get frustrated over switching away from a task, just acknowledge it's just the Zeignarnik effect, and do what you need to do.

Anyway, I'd be interested to hear about your experiences. Is the Zeigarnik effect real, and does it cause frustration? What do you or your team do to address it?

I'm an "old" programmer who has been blogging for almost 20 years now. In 2017, I started Highline Solutions, a consulting company that helps with software architecture and full-stack development. I have two degrees from Carnegie Mellon University, one practical (Information and Decision Systems) and one not so much (Philosophy - thesis here). Pittsburgh, PA is my home where I live with my wife and 3 energetic boys.
I recently released a web app called TechRez, a "better resume for tech". The idea is that instead of sending out the same-old static PDF resume that's jam packed with buzz words and spans multiple pages, you can create a TechRez, which is modern, visual, and interactive. Try it out for free!
Got a Comment?
Comments (7)
Trish isn't my real name
October 16, 2017
Sometimes I fantasize of being deaf and working from home.

That way, all bug reports and feature requests would have to be written down, documented, and prioritized.

And no, boss, your news blurb about a slick deal on Amazon is not more important than whatever I'm trying to focus on.
October 19, 2017
Unfortunately, this is the all too real everyday experience of many software workers.

There are some good words to describe the problem and some solutions in the lean software movement. "Lead time" is the time that it takes between an idea being conceived and its delivery as a real fully-functioning feature in production to a customer. "Waste" is any activity not promoting advancement towards delivering to the customer. One example of waste is "context switching" which is what you've described here. "Flow efficiency" is the measurement of actual time spent advancing towards customer delivery divided by lead time. In other words if lead time is the 10 business days in a typical two-week sprint, and 4 days are spent on actual valuable activity towards the delivery of the feature, then flow efficiency is 40%. In organizations that measure flow efficiency, 40% would be considered pretty good. Most orgs can't hit 15% because there's so much email-checking, pointless status meetings, HR paperwork to fill out, staff surveys, existing maintenance, etc.
Sam Sirry
October 23, 2017
The problem is that in software development you build a mental structure of the system in your memory. This takes time and effort, and when the interruption comes, you have to destroy this structure, then rebuild it again when you get back to the task at hand. This is specifically very unproductive with frequent interruptions to look after other small tasks.
October 23, 2017
@Trish isn't my real name - Ha. :)

@Scott - Great points and nice tie in to Lean principles. Thanks!

@Sam - Definitely agree with the context-switching element...and the inherent frustration there. To me, what's also interesting is that it seems that we're wired in a way that, context switching aside, it's just naturally frustrating to not be able to finish out a task that's already been started. There was a good study I read where there were people who were asked to do some task as part of research, and the time expired, but the participants still wanted to finish the task (despite the experiment being over) just because of the internal drive to complete tasks. Anyway, thanks for the comment!
February 13, 2018
Dear Ben,
We are a small but growing start up in SE PA, U. Michigan Graduates, with a rare opportunity for a Technical developer managing partner that can help us achieve our quarduple growth targets. We have steady and growing customer base and need to scale up our services. This is partly home-based and flexible and ideal for self-starter visionaries. Please check email we just sent you from Ray@kodoom.com
Also a father of 3 energetic boys, who finds our ethical business rewarding and flexible.
001-5216606
February 24, 2019
Hi,

I landed on your site while searching for Agile. I have spent quite a few time on the site and your content helped a lot.

I observed you have linked to https://leankit.com/learn/kanban/what-is-kanban/ from http://www.bennorthrop.com/Essays/2017/unclosed-loops-developer-frustration-and-the-zeigarnik-effect.php. I would like to bring to your attention a resource I recently created – https://www.guru99.com/agile-vs-kanban.html, The content is up to date and very in-depth.

I did be obliged if you link to it.

I would be glad to share your page with our 27k Facebook/Twitter/Linkedin Followers.

Regards,
Alex.

March 08, 2019
Hi,

I am curious to know your thoughts on my proposal below.

I welcome your feedback or rejection ( I pray not : - ) )

===========Original Message========

Hi,

I landed on your site while searching for Agile. I have spent quite a few time on the site and your content helped a lot.

I observed you have linked to https://leankit.com/learn/kanban/what-is-kanban/ from http://www.bennorthrop.com/Essays/2017/unclosed-loops-developer-frustration-and-the-zeigarnik-effect.php. I would like to bring to your attention a resource I recently created – https://www.guru99.com/agile-vs-kanban.html, The content is up to date and very in-depth.

I did be obliged if you link to it.

I would be glad to share your page with our 27k Facebook/Twitter/Linkedin Followers.

Regards,
Alex.