Essence and Problem Solving Blog

January 3, 2021

Build a Better Scrum using Essence and the Heart of Agile

Filed under: Essence and Problem Solving — pemcmahon @ 3:46 am
Tags: ,

If you would like to read this blog, scroll down to where it says “Blog Text Starts Here.” If you would like to listen to the podcast version, click on the link and scroll down to where you can select the podcast.

Blog Text Starts Here:

Today, thousands of companies are conducting agile transformations and according to Forbes 77% of those transformations are Scrum [1].  Yet, despite its popularity, according to Jeff Sutherland, co-founder of Scrum, 58% of all Scrum implementations are late, over budget and have unhappy customers [2]. Given these sobering statistics, a natural question we must ask is:

How can we build a better Scrum increasing our chances of success?

At least part of the answer, according to Sutherland, can be found in the Essence framework [3]. Sutherland tells us he has conducted an exercise with thousands of people in hundreds of companies where the participant teams build their own Scrum using Essence Cards [4] and then the teams presents their Scrum to the other teams as in a science fair.

Sutherland says:

“These Essence Cards flush out what is Scrum, what people are doing with it and how it is working…and we now have extensive data showing which parts of Scrum are implemented well,  which are implemented poorly,  and which are implemented not at all…  And on the average we find that about a third are implemented well, a third poorly, and a third are not implemented at all…. Essence immediately flushes out where the problems are, and then the next question is how can we use Essence to fix it?” [2]  

As a simple example of how Essence can be used to fix these problems Sutherland tell us one common problem you have in a vendor/customer relationship is who is responsible for what… By laying the cards out on the table and asking what the customer is responsible for, what the vendor is responsible for, and what do they jointly have to work on together, success rate is significantly increased.

Sutherland is not the only leader in the Agile movement looking at ways to fix common problems with Agile teams.  Alistair Cockburn, one of the authors of the Agile Manifesto has observed similar issues with Scrum, stating:

Scrum is pretty good.  It has very few rules, but it doesn’t tell you if anything goes wrong how to fix it.”  [5]

Cockburn, who believes that Agile has become overly decorated, has started a new initiative referred to as the “Heart of Agile” (HOA) in which he focuses on just four words—Collaborate, Deliver, Reflect and Improve [6]  The focus of HOA seems to be on the human or emotional side of software development.  Cockburn told me:

“If you see me emphasizing the human component more and more it’s because my experiences show it affects success more than other things.” [7]

The importance of the human dimension in software development success is supported by my own experience and the experiences of others working as SEMAT [8] volunteers on the development of Essence. In one of our Essence working group meetings we heard:

“The reason Scrum works so well is because we have come a long way on the technical side; how to develop requirements, how to design, how to test.  But it is the team side where we have been lacking and where Scrum fills a big hole.” [7]

While Scrum has helped many teams become more successful in areas such as collaboration through practices including Daily Standups and Sprint Reviews, the HOA initiative seems to be poking at a deeper underlying human issue.  One place you can learn more about it is in the first five minutes of a talk Cockburn gave in 2019 at a Lean Conference in France. [9]   

He starts this talk by saying he wants to make one small correction related to Agile, based on what he heard others saying at the conference the day before.  He then says:

“…Others put their energy into processes and tools and documents and plans, whereas the ‘magic’ is actually the way people interact with each other…. The magic comes from the movement of ideas between minds…”

 He goes on to say that the value that many people leave off when they summarize Agile is the very first value of the Manifesto and it is the one he is most proud of

“Individuals and interactions over processes and tools”  [10]

What struck me listening to this talk wasn’t so much what was being said, but rather the implications to successful software development. I was certainly aware of the Agile Manifesto Values, but I hadn’t previously given a lot of thought to the consequences of not emphasizing the values when people try to communicate what is essential to Agile.  

In the Essence language, the Pattern construct is used to reflect Values.  As an example, in Scrum Essentials, the Scrum Values are captured on an Essence Pattern Card. [4] Refer to Figure 1.

Figure 1 Scrum Values Pattern Card   

Those values, as depicted on the card, are:

  • Commitment
  • Courage
  • Focus
  • Openness 
  • Respect 

Note particularly the words on the card preceding the five values:

Successful use of Scrum depends on people living the five Scrum values.

These values are clearly an essential element to the successful use of Scrum, but in my own experience– and I would bet in the experience of many of those involved in the poor Scrum implementations referred to by Sutherland that– the essential values are too often not implemented well.

Reviewing the essential values of agile caused me to reflect on the fact that values must be lived, not just written down. It also caused me to reflect on just what the heart of agile is. 

From the Heart of Agile website [6] we learn that HOA is actually a “community of ideas” and that:

“The community forming around the Heart of Agile is what is interesting these days. People exploring better ways to collaborate, to communicate, to probe, to learn, to reflect, and to have more fun at the same time as having more impact.“

It occurred to me while reading these words that the degree to which teams truly “live” their values often has a significant impact on the success of a software development effort.  This also caused me to reflect on just how HOA might work effectively with Essence and Scrum.  To aid this reflection I recalled something Ivar Jacobson, the primary visionary behind Essence, once said to me, and that is the fact that practices are not just static descriptions. They are what we do! They are alive! 

More specifically, Jacobson recently said:

”…it is high time for a new paradigm. Where practices can be explored, experimented with, adopted and adapted interactively, in the same space where teams plan and execute their work, and continuously inspect and adapt their working practices.” [11]

The value of Essence is that it provides a reference framework that gives users the freedom to compare, and mix practices and patterns to form the best solution for their specific situation. Thinking of Essence this way brings me back to Jeff Sutherland’s exercise and something we all can do, particularly anyone reading this blog who believes they can implement a better Scrum. 

We can all conduct our own ”Build a Better Scrum” exercise similar to the exercise Jeff Sutherland referred to.  If you are one of those organizations that may be implementing Scrum poorly, or failing with Scrum, I suggest you download your own set of Scrum Essential cards [4] and conduct your own Build a Better Scrum exercise.  And when conducting this exercise be sure to pay close attention to the values your team holds giving particular attention to not only the Scrum Essential Values, but also HOA patterns, such as the one that captures the first value of the Agile Manifesto. 

You can use the Heart of Agile along with Essence to help your team build a better Scrum by bringing back the focus on the real essentials—including the values– to successful software development. 

Comments are encouraged. 


[2] SIGSOFT Seminar, April, 2020,, Refer to 14, 26 and 28 minutes into presentation



[5], Refer to 38 minutes into presentation


[7], Refer to Epilogue and Chapter Five.

[8] SEMAT stands for Software Engineering Method and Theory. Refer to for more information.




December 20, 2020

Using Essence to Teach Software Practitioners in Industry

Filed under: Essence and Problem Solving — pemcmahon @ 7:23 pm
Tags: ,

A few weeks after the keynote address I referenced in last week’s blog, I gave a talk at the 5th Essence Education Forum (EEF).  The title of my talk was: Using Essence to Teach Software Practitioners in Industry

In this talk I explain what is different about teaching software practitioners in industry from teaching university students.  I also motivate why university professors might want to consider the approach I use in industry to better prepare our university students for the challenges they are certain to meet.

You can listen to my roughly 27 minute talk using the link below.  As always, feedback is encouraged. 

December 10, 2020

An Industry Perspective on What We Should be Teaching Our Next Generation of Software Practitioners in the Universities

Filed under: Essence and Problem Solving — pemcmahon @ 8:56 pm
Tags: ,

Last month I gave a keynote address at the 1st International Workshop on Essence in Education and Training (WEE&T’20)  conducted as a part of the 32nd IEEE International Conference on Software Engineering Education and Training (#cseet2020).  This conference was conducted virtually from Munich Germany. 

To access a video of my 10 minute talk:

You can learn more about the conference and the workshop at:

August 19, 2020

The single most important software team success factor reinforced by two software engineering giants

The single most important software team success factor, and one I always share relevant stories about when coaching software teams, is the need to create an environment where all team members feel safe in raising any issue that bothers them—even small ones.  This essential for success is captured in two Essence thinking framework checklist items from the Team Alpha, Collaborating State:

  • Communication within the team is open and honest
  • The team is working as one cohesive unit

The importance of this factor was reinforced for me through recent interactions I had with two software engineering giants—Alistair Cockburn and Larry Constantine.  Let me share a little about these experiences.

After I completed the initial draft of my “Shy Boys” book [1] in July, 2019, I asked Alistair Cockburn if he would review the book and let me know if he was okay with what I said. He quickly got back to me letting me know he was fine with all the parts that mentioned him.  I share quite a bit about Alistair and his focus on the people side of software engineering in the book. Later that day while on a plane he read more of the draft book and sent me another email saying:

“It occurs to me that what drives me is this question: ‘What leads a team to a higher likelihood of success?’ If you see me emphasizing the human component more and more, it’s because my experiences show it affects success more than other things.”

Larry Constantine also got back to me after reading the book saying:

“I would like to set the record straight on a couple of matters.

 First, I did not say that Ed Yourdon and I ‘missed the people side’ when we were developing structured design (and later structured analysis). What I did say in Zurich (and on other occasions) was that Ed and I left out design of the human interface (along with database design). It had always been my intention to make the original book comprehensive, but we ran out of time, our hand being forced by the publication of the IBM Systems Journal article and the threat of being scooped by Glen Myers. Moreover, the concepts of interaction design and the user interface were hardly well developed at that point. We also concluded that others were doing a fine job on database theory and architecture, so we went with what we had. Of course, as you know, I did return to interaction design and the human interface later.

 The reality is that, from the beginning, everything I did in the computer field was about ‘the people side.’ My thesis at MIT was on the psychology of computer programming; the conceptual basis underlying coupling and cohesion is rooted in cognitive complexity theory; and the graphic models were obviously all devised for the people doing design and engineering of software, not for the computers. Ed and I had always been highly focused on the people side of software engineering and were instrumental in spreading such key ideas about project organization as Conway’s Law and Mealy’s Law.”

For those who may not be familiar, Conway’s Law states that:

“Any organization that designs a system will produce a design whose structure is a copy of the organization’s communication structure.”

What I find most interesting about this law/principle (and relevant to this blog post) is the fact that two distinct organizations with differing communication structures are likely to have conflicting preferred design approaches and collaboration difficulties.  I observed this phenomenon first-hand on multiple occasions with clients and I have written about it extensively in numerous publications. [2]

Recently, I got thinking again about the challenges of getting people to work well together as I listened to an interview Alistair Cockburn did with Richard Atherton in April, 2019. [3]

About 38 minutes into the referenced interview Alistair shares part of his theory of software engineering and how you can develop explanatory and predictive models using levers you can touch to influence project success.  In the middle of this discussion he hesitates and says:

“And what I love is that the lever you can’t touch is human personality.”

After explaining what he means by this with a few examples, he goes on to say:

“The question in my mind is how do you simplify so its actionable?… Scrum is pretty good.  It has very few rules, but it doesn’t tell you if anything goes wrong how to fix it. SAFe went the other way and added a bazillion rules that are very very hard to follow, but claims to be complete….

So, what I did is reduce all of this to just four words Collaborate, Deliver, Reflect and Improve… This is not going to surprise you…. As people collaborate better, things go better, as people collaborate worse, things go worse…  That’s not rocket science.  It’s astonishing how quickly we forget it.”

What I particularly like about this part of the interview is that Alistair goes on to explain why people quickly forget the importance of collaboration. He does this by connecting collaboration to things we don’t always think of as collaboration, like fear and trust.

Then he asks: “What are people afraid of?”

His answer is:  They’re afraid of looking bad, not getting a raise, getting fired, not getting their idea implemented and on and on…

And he goes on to explain further that the root of the problem often isn’t in process, but it’s in things like Executive bonuses and human resource appraisal systems which often fail to consider effective team collaboration.

This all made sense to me based on my own experiences, but it also caused me to reflect on my own approach to this challenge with my own clients which I will explain in a moment.

Alistair then tells a brief story about a client who said to him:

“Alistair, … Tell me what to tell the bosses to tell the troops.” 

Alistair replies:

“I am not going to tell you anything… I am going to dialogue with you about how to dialogue with your bosses about how to dialogue with the people… We are trying to create a culture of listening… The Execs should dialogue with the people to find out what they know…”

Alistair’s client replies:

“I’ve been trying that but the Execs tell me I’m not here to dialogue with you. I hired you to tell me. You are the expert. Tell me.”

Alistair then says,

“That’s fine. That’s not your client. You don’t have to have all the clients in the world.  If our tone is dialogue based, people will self-include and exclude… Today more and more people in the world are growing up in dialogue, rather than edict.”

This part of the interview particularly resonated with me, as it caused me to reflect on my own thinking when I started my consulting business in 1997.  At that time I wrote the following words that still remain true 23 years later and are still on my website:

“If when you think of a ‘consultant’ you think of someone with real project experience who can come into your organization, understand your key issues, and give you the kind of help that pays high dividends rapidly, then one could refer to our services using this term.  We often don’t use this term because too often it conveys a notion of examining a client’s problem from a distance –and then telling the client what they should do… We work directly with our client’s team first understanding their strengths…We share with our customers practical techniques that make sense in today’s challenging environment.” [4]

The key point here relevant to this blog post is to listen first.  Then I operate more in what I think of as a facilitator of dialogue (rather than a “consultant”) helping the client discover their own best solution.

The reason I take this approach is because when the client discovers their own solution they are far more likely to own it, and believe in it, and implement it leading to a lasting solution. I have also referred to this approach in other publications and training courses as using Essence in “Stealth Mode” [5, 6, 7]

One example of a practical technique I have shared with clients to improve collaboration is to enhance their appraisal system by using inputs from teammates and customers when appraising personnel. When team members know their appraisal and potential salary is tied to how team members view them, their attention to team collaboration often improves rapidly. Whatever is measured gets attention.

These recent reflections have led me to start thinking again about what we should be teaching our next generation of software professionals, in particular with respect to organizational cultures, collaboration and essential team skills.

It seems to me, at a minimum, we should be making them aware of the existence of varying organizational cultures—such as organizations with cultures of “just follow the rules” and those with cultures that encourage  “listening and dialogue.”

These differences, of course, aren’t new.  They existed 50 years ago and although more and more of the world seems to be moving toward the “listen and dialogue” approach as Alistair has observed, I suspect these differences will continue to exist for many years to come.


[2] Numerous stories of conflicts arising when organizations with differing communication structures and cultures attempt to collaborate can be found in my Virtual Project Management book and related articles. Refer to



[5] For an example of  using Essence in stealth mode refer to

[6] For online training in using Essence as a thinking framework with numerous examples of using Essence in “stealth mode” refer to

[7] For more general information on Essence refer to

July 3, 2020

How to get better at transforming a job you don’t like into your dream job

June 25, 2020

How to take back control of your life responsibly and safely during a COVID-19 pandemic

June 16, 2020

How to get better at protecting your privacy, security and safety in an internet-of-things world

June 9, 2020

How big tech companies are trying to control your time and how to get better at fighting back

Filed under: Essence and Problem Solving — pemcmahon @ 6:17 pm
Tags: , ,

June 2, 2020

How “big data” is being used to misinform and how to get better at fighting back

May 26, 2020

Three Essentials To Getting Better At Anything

Next Page »

Blog at