The machine that makes the machine

A bad system will beat a good person every time.

That has to be one of my favourite quotes from W Edwards Deming.

It seems in many places of work, we put so much emphasis on product delivery, yet we spend very little time looking at how we deliver products.

The last few places I’ve worked have essentially had the same issue, lots of work to do, varying pressures to deliver but actually getting stuff out of the door has been a challenge.

Ironically they’ve both had varying amounts of automated pipelines. With Azure and AWS deployments should be easy. They even support strategies like rolling deploys out of the box in most cases, yet releases are still a big thing.

Often I find there’s a heavy reliance on manual QA. This is naturally slow ( think a day or 3 ) and will invariably discover lots of things like.. There’s a missing connection string. It doesn’t render correctly on the IPad 2 or even:

Frustrating to say the least. We go about fixing the issues, what’s wrong with the connection string.. DB team hasn’t created the database yet, or InfoSec need to unblock the dB port in the firewall. To resolve these issues obviously requires more tickets on different boards.

QA will create a ticket saying “home page returns 500" a developer will investigate, realise it’s a connection string issue. Try to manually connect and find the dB isn’t there. Or is it permissions.. Raise dB ticket to investigate, they look at it, dB wasn’t created.. they sort that, still doesn’t work but works from Devs machine. Another ticket for InfoSec. They need a business justification for this firewall change.. easy enough a day later it’s unblocked and we’re away. But then we ask whether this will happen in production. Yes and it requires new tickets to make the same 2 changes in production, and don’t forget the DR site.

Then there’s problems with your predictability being off, you need to be more realistic in your estimation to account for these things. I can’t help but think of the planning fallacy, a new service goes from, the initial thinking of a few hours, to build a basic skeleton. Too weeks to get it working in the test environment forgetting production.

To these scenarios I say we need to focus less on being realistic in our estimations and more on making it possible for people to actually do their jobs without it feeling like an uphill battle.

It’s easy to cover these issues up with larger estimates but by doing so we’re putting the business at risk, we’re not taking the responsibility for actually improving the system.

There are those that will say:

Or

These utterances drive me crazy, in truth these issues are likely within our control if we were only to say we own them. The dB process or InfoSec process is likely just a historical artifact that has naturally fallen into place. If a team were to engage with infosec and simply express that when developing a new service we need the same ports across all environments, it’s likely they would be very receptive on how to standardise the process. We might even learn what they check for and make the whole process redundant for a whole class of work.

You then have an all hands where the execs are touting the amazing stuff Azure or AWS can offer and how teams can just experiment and build the next amazing thing for the company. You login to AWS and see only EC2, having access to all those cloud goodies is a security risk it turns out.

I think it’s beholdent on us especially if we’re in leadership to resolve these problems.

JFK

Related posts:

Here is a quick tutorial on how to grab some Sui token but first I will introduce you to SUI network. Sui is the first permissionless Layer 1 blockchain designed from the ground up to enable creators…

Making a decision on personal growth and development is certainly a big step. Based on my experience both as a client and a life coach, I have identified 3 traps that can hinder self-development. I…

The availability of 3D shape data significantly increased over the past years due to advances in 3D capturing sensors and 3D modeling tools such as Blender. This naturally leads to a growing interest…