Belt and Suspenders

Riverbank Ruminations; Observations from The Banks of The Technology River

Tom Evans ~  Ashton Engineer Emeritus

Belt and suspenders

A “belt and suspenders approach” is a phrase “commonly used in business negotiations to describe a situation in which two protective strategies are used to minimize the risk that would be present if only a single protective strategy were in place. The origin of the phrase is literal: A belt and suspenders would each likely hold up a pair of pants on their own, but using both at once represents an even safer route.” [1] An example in computing might be having two (or more) different backup plans going to different devices and stored in different locations.

This phrase is usually used in a context recommending a conservative approach to a given situation. As you get more experienced, you come to appreciate the wisdom of being more conservative. You come to realize that things happen and more often than we would like to admit, multiple things happen to cause problems. You are driving to work and the defroster stops working, followed by running out of window washer fluid. Your power goes out at the office and that’s when you find out the battery backup needs new batteries.

I saw this posting one day and my first thought was “first-world problems”. Then I thought how it was representative of how many devices we are putting on the internet just to save a few steps, or allow us to access them remotely, even if we could live quite nicely without the remote access capability.

However, the kind of impact an internet outage could have was brought forward by this article,  which was followed by this one and then this one. Note that problems with one provider (Amazon AWS) affected: Netflix, Amazon Prime, Roku, Ring, Twitch, Zoom, PSN, Xbox Live, Doordash, Quickbooks Online, Hulu, Epic Games, Honeywell, Slack, and others. The third article cited above has the longest list of affected services. That same article quotes a statement that “Amazon has confirmed a power cut at its US-EAST-1 data center is impacting services that rely on the particular ‘Availability Zone.‘

While most of the impact affected entertainment services, Zoom and Quickbooks Online would be two services that could be critical for a business to have available. While cloud-based services have the advantage of availability from anywhere, these outages raise the question “What do you do when they aren’t available?”

You Need A Business Continuity Plan

Business continuity plans are something that cannot be considered optional unless you don’t care if your business can survive. Before the internet, business continuity plans would be more localized in scope. What do you do if there is a fire? What happens if there is a chemical spill? Are the insurance policies up to date? With the advent of computers, other issues were added to the mix. Now you had to worry about backups. Where do you store them? How often do you take them? How do you test them?

When we added the internet to the picture, we now had to deal with security issues and remote access. We have more recently seen the surge in ransomware, data exfiltration for extortion, and even DDoS attacks being used to relieve businesses of cash.

Let’s go back to the belt and suspenders approach to risky situations. If we are talking about literally using a  belt and suspenders, we are looking at a simple situation. We have one pair of pants that we would like to keep in their proper place. If both the belt and the suspenders failed, there is a good chance the pants would still stay up. In the worst case, we could use one hand to hold them up so there is one ‘emergency’ fix that could be applied.

What Is Your Risk?

In business, one of the biggest problems can be determining what the risk is and how to protect against it. When companies had onsite mainframes, a fire in the computer room was a bad thing. Sprinkler heads were one solution but the water could be a disastrous way to put out a fire. Halon systems offered less destructive protection, but at a greater cost. The additional cost could offset the downtime involved in getting replacement computers. It was a cost vs. benefit analysis.

Today, businesses have a broader spectrum of problems to address. Remote workers have all but removed the traditional definition of where the workplace is. The business network used to reside inside the walls of the company. VPNs moved some points outside the walls. Remote work has extended the office network outside the walls to an extent previously unknown to most companies. The risk exposure has increased commensurately.

Cloud-based services have allowed many companies to reap benefits and operational abilities they did not have before. The assumption has been that the belt and suspenders were being used by the cloud provider. The articles cited above should help you realize that even the largest of the service providers are vulnerable to outages. Fortunately, in the cases above, the outages were not lengthy. It is of value to note that among the affected companies was Amazon itself. As the second article noted: “Amazon delivery employees also said they couldn’t access internal apps required to scan packages, access delivery routes, or see any upcoming schedules.”

Don’t Rely on Somebody Else

If you have a business continuity plan, does it account for the loss of services in the cloud? If your internet goes down, how will your remote workers work? How long can you afford to be incommunicado before there are disastrous consequences? The list of things that can go wrong is huge. Some of them may never happen, some of them will happen infrequently but for the sake of your business, you need to consider them, some will be highly likely. It will be beneficial to get someone familiar with the threats businesses face to help you decide which ones to defend against.  You might be surprised to know that most businesses don’t know what they need to protect. They don’t have a complete inventory of what is on their network. They don’t know the patch status of their computers. They don’t know who has access to their network. This is fundamental information that is necessary to formulate an effective business continuity plan. Don’t rely on someone else’s belt and suspenders. It’s your company, you have to protect it.