Good Mistakes

For making better decisions, foster the safety to fail

John Lewis was a portrait in courage. Already weathered by his experiences standing up to hatred and racism across the Southern US, he faced his hardest test yet on March 7, 1965 before the Edmund Pettus Bridge in Selma, Alabama. He stood with 600 others on a march to Montgomery when the marchers were attacked by state police. Lewis sustained a skull fracture and many others were injured in an incident to be later known as “Bloody Sunday”.

Recently I watched a documentary about the life of John Lewis. He never stopped fighting the good fight, even up till his passing last July. If there was one theme that could capture the full breadth of his life’s work, it would be summed up by his quote:

“Never, ever be afraid to make some noise and get in good trouble, necessary trouble.”

He understood that to make change means disrupting the status quo. To disrupt centuries of institutionalized racism meant an even greater change in culture, society, and politics. The only way to ensure change builds momentum and leads to an end to racism and injustice would be to confront people and systems intent on maintaining the current order. Good trouble was John Lewis’ way to bring about positive change.

With fifty years of hindsight, we can all agree that the events in Selma was wrong. At the time though, this was not the case. Institutionalized racism in the US was accepted instead of being seen as an affront to dignity and human rights. It took the brave few to walk across the bridge and speak up against unfairness, cruelty, and oppression.

“When you see something that is not right, not fair, not just, you have to speak up. You have to say something; you have to do something.”

Over the past several weeks, there have been several instances of the people speaking up against injustices in the tech world. A well-known AI researcher and ethicist at Google, Timnit Gebru, was abruptly dismissed over the publication of one of her research papers. More recently, GitHub got into hot water for firing an employee posting in a company chatroom comments about the racist ideology of some of the participants of the January 6th riots at the US Capitol. In both incidents, employees spoke up to expose injustices and make change happen.

If the initial mistake was bad, the official response was even more tone deaf. It follows a typical pattern. The first response is to blame the victim. Next is to place fault with the systems. Then it is to issue a tepid apology that avoids directly addressing the concerns raised by the injustice. Last step is to hope everyone forgets the controversy and to avoid any real concerted efforts to change. To GitHub’s credit though, they did eventually take ownership and responsibility for their exceedingly poor judgment.

These are all examples of bad mistakes, the type of mistakes we do not learn from. The reason learning from mistakes is hard because we think of mistakes as one way doors. The blame and embarrassment overshadow the potential for catharsis, reconciliation, and understanding. Our brains are hard-wired to avoid blame and to prioritize short-term desires, preventing us from unwinding our bad decision and apologizing.

In corporate blame culture, we observe the same ritual. A critical app goes down, a database backup goes bad, or a product launch flounders, and we need to find fault in the people or person. There is little appetite for mistakes, so employees and teams avoid risks, suppress change, and engage in elaborate exercises to pass the blame while hoarding the glory.

It would be a mistake to call mistakes one way doors though. The vast majority of decisions we make are never final or irrevocable. Often we make decisions based on incomplete data or experience, only to discover something new and enlightening in the process that allows us to continue on, pivot, or roll back our decision. In that sense, mistakes are really two way doors, or what I call good mistakes.

Good mistakes are ones that we learn from and improve our decision making. Many mistakes can be good mistakes if we operate in a blame-free environments in a culture that values learning and transparency. Even terrible mistakes can be valuable if they lead to better ways of handling future situations.

Do all mistakes have value though? In my talk “Failure at Scale”, I proposed rather than looking at failure as a binary result, it is better to see failure as a scale. On one end, there are failures that result from incompetence, lack of attention, or outright maleficence. On the other are those failures that clarify, provide insight, or expand of our thinking. As an example, some mistakes lead to tragedies like Chernobyl or the Challenger disaster while some lead to surprise innovations like the polio vaccine or the light bulb.

“I have not failed. I’ve just found 10,000 ways that won’t work.”

-Thomas Edison*

Culture in this case plays a significant role. Good mistakes thrive in a generative culture where everyone is aligned to the values of the organization, trust is strong and the default interaction mode of people and teams is collaboration. The more bureaucratic or pathological the culture, the less likely good mistakes will arise, be supported, or advance positive change. This is one reason for the rise of tech cargo cults in organizations which only serve to impede progress.

Another consideration when it comes to mistakes is one of impact. On software engineering teams, there are thousands of seemingly small decisions we make each day, any one of which can have major downstream impacts. Few predicted that two digit years in code would lead to a global Y2K refactoring effort. Then there are other decisions that may seem really important, like whether to use Rust versus Golang or serverless versus microservices, that ultimately do not matter to the success of the product.

The last consideration is speed. Startups are generally faster to act and faster to iterate over larger, more established organizations. When mistakes arise and new insights are revealed, the lack of established process, embedded political structures, and organizational barriers means change happens sooner. This leads to faster innovation, swifter adoption of change, and more nimbleness when taking corrective actions, if enacted fast enough.

Framing decision making in the lens of value, culture, impact, and speed with the understanding that very few decisions are one way doors leads to better outcomes. Good mistakes in that sense are really just experiments towards better understanding that give us the safety to reverse ourselves. The more generative the culture, the more willing they are to accept that mistakes are not errors, but unexpected results that open new doors.

I will talk more in a series of posts on decision making, the biases in decision making that impact technology initiatives, and the role empathy plays. To that point, in his recent Presidential Inaugural address, President Joe Biden asked:

“If we’re willing to stand in the other person’s shoes just for a moment.”

Indeed, if we could spend more time to listen, understand, and empathize with the perspectives of others, how much better would our decisions be? That is the opportunity we have as a species to unlock the vast potential of human ingenuity for the betterment of humanity.

In the meantime, ask yourself whether your organization is open to good mistakes. How do you incorporate learning cycles into your engineering teams in a systematic way and to what extent are engineers allowed to explore good mistakes?

Mark Birch, Editor & Founder of DEV.BIZ.OPS

* The actual quote is not as catchy, read the story of what Edison really said here:

Happy new year! In some ways it feels as if we are living through an extended 2020. I remain hopeful though that the challenges, the heartbreak, and the loss from last year will lead to opportunity, hope, and better days ahead in 2021.

For those of you in the startup world or thinking about jumping into startups, I am presenting on building MVP’s at the B2B Growth Bootcamp sponsored by AWS and HubSpot. My talk is Feb 2nd from 2–4 PM SGT and you can sign up here.

Speaking of startups, through my AWS colleague, I learned of three interesting engineering leadership roles that opened up, based in London.

  • Head of Engineering at Troglo — Looking for an experienced Head of Engineering who can lead all technical aspects of the business from building the technology, discussing the company’s technical capabilities with external stakeholders to developing and delivering on Troglo’s product roadmap to modernising the healthcare industry to reduce the spread of STIs & HIV through our sexual and mental wellbeing app . Read more here.
  • CTO at Pesky — As CTO at Pesky, you’ll be joining the leadership team to define the technical architecture that delivers on our vision. This role will offer sound technical leadership in all aspects of our business and help us deliver digital innovation to the British fishing industry. Click here to learn more.
  • CTO at Salary Finance — We are looking for an innovative and inspirational CTO to lead the Salary Finance technology strategy and team to the next level as a leading global FinTech platform. Reporting to the Co-founder/CEO, you will be contributing to the overall business strategy, representing technology and engineering on the Senior Leadership Team (SLT). Find more details here.

Cloud experience for all of these roles is required and both Pesky and Salary Finance rely heavily on AWS. If you have senior engineering roles you are looking to fill, let me know and I can include in the newsletter.

Want to learn more about DEVBIZOPS and read more hot takes about IT, technology, and working smarter. Receive our weekly newsletter by signing up to our Substack!

Thoughts on developers, digital transformation, enterprise agility, community building & software engineering culture. Author 👉

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store