Raise your hand if you have a Disaster Recovery Plan. Now raise your hand if you practice that plan at least once a year. For many companies, the Disaster Recovery Plan was put together in the past to check the box on RFPs, but if you aren’t practicing with it and updating it in a regular cadence, you may be missing key elements. The wrong time to verify the viability of your plan is when you need it, so let’s look at the things that can blow up your carefully crafted plan.
Let’s start by separating the concepts of disaster recovery and business continuity. Disaster Recovery is a restoration process for IT infrastructure, data, data access, and processing capabilities. Business continuity focuses on what is needed to keep a company operating. Notice that the causes of the disruption and the time frame are not part of the base concept. That is because disruption can come from any direction at any time of the day or night. It’s like Mr. Mayhem on insurance commercials. The best practice is to have a plan for both, not one or the other, so this is a two-part piece.
The cause of the disruption will inform the approach you take as you begin the DR process but remember that it may be something you never imagined. While you may be the victim of an earthquake, hurricane, flood, or fire, it is as likely that your event will be set off by a power line being cut during a construction project 100 miles away or a fiber optic line sliced by equipment 400 miles away, or a gas line cut on the other side of your industrial park. A crane lifting heavy equipment into the building could fail, dropping the weight through your roof. It doesn’t take much imagination to create dozens of scenarios that result in loss of power, internet, water, or access.
Now imagine that you are on the outside and you can’t get inside.
The Obvious DR Elements
To recover, you need to know what you have and your baseline requirements for operation. Begin with a quick check. Is there a written plan sitting on a shelf or in a file on the server? If you can find one, look at what it covers. Even if it is old, it may provide a starting framework for your next step – collecting the information on the current state of hardware, software, customers, products, and the elements that go into the communication manufacturing processes.
If you are already running a cloud-enabled workflow with your mission-critical software running in the cloud, your needs are different from those still operating from a local server room. In either case, you should have a DR plan that identifies every piece of hardware and every software solution. While there is a temptation to make a list from memory and call it done, the best practice is to build a file that identifies each piece of equipment, the vendor and the primary contact, its serial number, its physical location, its power requirements, and what it does. This list must be kept current as equipment is upgraded and replaced. Every software product used should be listed, including the operating system, the vendor, where it fits in the workflow, if it is part of an automation scheme, and the primary contact for problem resolution. If you need to move the operating location, you may need to arrange for a backup license!
The Call Book may exist online or on paper. It must be accessible to senior leadership at any time because in the event you are executing your DR Plan, you will need to reach out to employees, partners, and customers. Talk to any vendor and you will hear tales of DR Plans being activated followed by phone calls to people who now work for different vendors, missing vendor contacts for current hardware and software, and customers who were never notified. The cloud is a great place for your Call Book, but it won’t maintain itself. Someone needs to be accountable for revisiting it on a regular cadence to ensure it will be useful if you need it.
The Hidden DR Elements
After a solid first pass, it’s time to look for the things that will trip you. They will be different in every shop, so it’s harder to offer a practical, comprehensive list. Begin with the products you deliver to customers. Your shop may have a unique color profile for each job, as well as a catalog of profiles related to specific substrates and machines. Where are those files?
Do you print checks? Some companies still print check stock with pre-printed MICR lines or pre-printed logos and formatting. Not all inkjet shops are Automated Document Factories using white paper workflows. If you have stock for specific jobs and you are prevented from entering your facility, what is your plan? If you intend to print with a partner or from a hot site or even another plant you own, will the stock be there for you?
Some companies make extensive use of spreadsheets and whiteboards to keep work flowing and keep the teams informed. Not everyone has a real-time dashboard. Every manual process, every project tracking whiteboard, and every offline spreadsheet represents a hidden element that could derail your DR Plan.
And then, the 800-pound gorilla sitting in the corner – security. Malware, ransomware, and breaches touch our industry every day. Sometimes the touch comes in the form of shutting down servers and blocking access to mission-critical software and data. What’s in your DR plan for a security breach?
Next time the topic is Business Continuity and how it relates to your DR Plan! Have questions? Put them in the comments or get in touch.
Follow all Pat’s posts at the Pat McGrew page. You can also find her on LinkedIn.