Section 3: Developing a Disaster Recovery Plan
Anchor: #i1014087General - What the Plan Involves
Districts and Divisions should develop a written Disaster Recovery Plan appropriate to their own operation. Even minimal preparation can have a significant impact on the success of efforts to recover TxDOT records. Below is information provided by Texas State Library and Archives Commission on disaster recovery procedures which states the following:
- Anchor: #YDKDGCAX
- Create written disaster recovery procedures for accessing vital electronic official records during a disruptive event. Anchor: #UTQEBNPW
- Address vital electronic official records in continuity of operations, business continuity, and/or disaster recovery plans as part of the agency's overall continuity program, as required in Texas Labor Code §412.054 “Title 5. Workers’ Compensation Act, Subtitle A. Workers Compensation Act. Anchor: #CMVRDPEJ
- Require all third-party custodians of official records, holding records on behalf of the state, to provide the state with descriptions of their business continuity and/or disaster recovery plans as regarded to the protection of the state agency's vital electronic official records. Anchor: #HOLTKPQH
- When a disaster occurs where official records are prematurely destroyed or are not identified on TxDOT's approved Records Retention Schedule an RMD 102 Form, Request for Authority to Dispose of Official Records is required to be submitted to Texas State Library and Archives Commission to obtain approval for disposal of records. Contact Records Management to coordinate the submission of the RMD102 Form. Anchor: #KWDLVQAH
- Must be submitted to Texas State Library and Archives Commission for approval for destruction, (due to weather conditions i.e. water-damaged) before the retention period has not been met.
Developing a Disaster Recovery Plan involves:
- Anchor: #WLUNPUUK
- Stockpiling emergency supplies and arranging services. Anchor: #DUVVEOYK
- Establishing a disaster recovery team. Anchor: #TCKVTFHL
- Developing disaster recovery and records salvage procedures. Anchor: #UBQIKIWU
- Contingency planning.
Emergency Supplies and Services
In an emergency, self-reliance is critical. In any major disaster, TxDOT is likely to have to assist with recovery efforts for other agencies and the public while managing its own recovery.
Check supplies and re-confirm vendor relations and service agreements periodically. After a wide-scale disaster, these resources may not be readily available. Suppliers or service organizations may have their own damages to deal with, or they may be helping someone else.
Anchor: #i1014192Disaster Recovery Team
Districts and Divisions should establish in-house disaster recovery teams to resume or sustain operations after a disaster, including handling recovery of their records. The teams should develop and practice disaster recovery and records salvage procedures. Employees should be trained in the processes included in the Disaster Recovery Plan.
Each team member should have two copies of the Districts and Division’s Disaster Recovery Plan, one at the office and another at home.
Anchor: #i1014212Recovery Team Responsibilities
Ideally, a disaster recovery team should consist of at least four members, including a team leader. Each team member should have an alternate. Responsibilities of team members are as follows:
- Anchor: #GOCJFMVQ
- Team Leader: Acts as point of contact for local emergency service agencies; performs overall management of the team's disaster recovery and records salvage procedures; coordinates with other offices; authorizes expenditures for wages, supplies, transportation, and services; and is liaison with TxDOT customers and/or the public. Anchor: #APALRHIM
- Recovery and Salvage Supervisor: Assembles, trains, and directs work crews for recovery and salvage procedures. Controls the flow of work and materials. Anchor: #SAESYJEK
- District/Division Executive Assistant: Coordinates with the District or Division to gather supplies and equipment and may arrange such things as food for work crews. Anchor: #KFLRYAQK
- Records Administrators/Coordinators: Inventories damaged records. Assesses damage and salvage processes required to restore records. Performs or supervises recovery and salvage procedures.
Disaster Recovery and Records Salvage Procedures
Each District and Division is responsible for developing emergency procedures to follow in a disaster and making sure employees are familiar with them.
Include these essential procedures in any Disaster Recovery Plan:
- Anchor: #BJJXRAAG
- Establish
Security and Safety - For security and safety, restrict
access to damaged area(s), allowing only essential personnel to
enter. This minimizes opportunities for further damage to records
and injuries to personnel. Use security guards, sign-in/out registers,
and/or identification badges to restrict access.
Basic safety precautions include the following:
-
Anchor: #PXFHYQKH
- Inspect disaster areas for hidden hazards, such as shorted motors or broken electrical wires. Anchor: #LTUEATJB
- Avoid standing water and wet carpets, which make it dangerous to use electrical equipment. Anchor: #GYHABITN
- Install and use temporary wiring properly. Anchor: #ONYJUBWY
- Handle fire or water-damaged files carefully. Wet records are heavy and fragile. Use care lifting record boxes or opening file cabinets. Anchor: #LRPPEFDL
- Use face masks and protective gloves. Wash or clean clothing as soon as possible to reduce health hazards from fungi and bacteria.
Anchor: #HCKOQMXX - Activate Recovery Team - Once
the building has been declared safe to enter, the team leader should
contact the team and brief them on procedures and priorities to
be met. The recovery team should then:
-
Anchor: #EXPVIAQO
- Assign specific responsibilities to members. Anchor: #WMEBEIVJ
- Set up a communications center and obtain services, supplies, and equipment. Anchor: #NQQGFEME
- Plan to take care of physical needs (food, water, hygiene) of personnel involved in the recovery.
Anchor: #OSEBJUML - Assess Damage - Inspect
damaged areas as quickly as possible. Avoid handling records, if possible.
Take photos to document record damage. Keep the following questions
in mind:
- Anchor: #CHDQWSIE
- How much damage has occurred? Anchor: #IMXWVTBD
- What type of damage has occurred (fire, smoke, soot, clean or dirty water, heat, humidity)? Anchor: #UDWUHDYI
- Is it confined to one area or is the entire building damaged? Anchor: #HPHFRQPS
- How much of the records holdings are affected? Anchor: #BDETDYID
- What type(s) of records media have been damaged (paper, microforms, photographs, magnetic tapes, diskettes)? Anchor: #KPCKCHSM
- Are the damaged records easily replaced? (Are backup copies stored off-site? Are the damaged records convenience copies? Can they be replaced with copies from other offices?) Anchor: #NLPYBSGI
- If the damaged records are irreplaceable, what is their retention/value? (How valuable/important are these records to the business of TxDOT?) Anchor: #DINDOHWV
- What is the order of priority to recover damaged vital/essential records? Anchor: #WGRKXSJQ
- Can the in-house recovery team salvage
records or is outside help needed?
NOTE: Take steps to stabilize the environment while assessing the damage (see below).
Anchor: #TCMLLSFG - Stabilize Environment -
Take steps to stabilize the environment while the damage assessment is
taking place. Mold can appear on records in 48 hours and is encouraged
by high humidity and temperatures. Quickly reducing the humidity
and temperature can reduce the risk of mold and buy time for recovery.
The following procedures will help stabilize the environment:
- Anchor: #BKXVRTJK
- Keep the air constantly circulating. Anchor: #AFFXANOK
- Arrange fans to expel humid air outside. Anchor: #RHXKUTXO
- Use dehumidifiers if they are helpful. (Dehumidifiers may help in small, enclosed areas. Operating them may raise the temperature, however, and they tend to freeze up in colder temperatures.)
Immediately pump standing water from the area. Use caution -- standing water can conceal hazards. Remove wet debris and carpeting. The following equipment should be readily accessible:
-
Anchor: #VBUXMYAH
- Portable generators, in case of power failure. Anchor: #JJCMQEVM
- Shop vacuums capable of handling water (pumps may be needed to remove large quantities of standing water). Anchor: #MBBHPNUD
- Fans to circulate the air. Anchor: #TBDUDFYH
- Thermometers, hygrometers, or other tools to measure temperature and humidity.
Anchor: #MBYCRDUR - Treat Water-Damaged Records - Water-damaged records are the records most likely to be salvaged. Methods of treating water-damaged records are listed in the “Salvaging Water-Damaged Records” below. Anchor: #RYBFRXDP
- Perform Post-Disaster Assessment -
Assessing the Disaster Recovery Plan is essential. Share the results
with Districts and Divisions and revise the plan if needed. Assess
the following:
- Anchor: #WXPYMTGU
- Effectiveness of recovery procedures Anchor: #DSCDGGUL
- Areas for improvement Anchor: #EGDJCOIN
- Sources and supplies Anchor: #EWIGVWDL
- Any off-site facilities or vendors used
Salvaging Water-Damaged Records
The records most likely to be salvaged after a disaster are those that have water damage. Several options are available for treating water-damaged records. The steps to follow vary according to the record medium and are described below.
Paper - When paper records are slightly damp, air-drying (natural drying) may suffice. If records are soaked, they should be freeze-dried, regardless of quantity.
- Anchor: #YTHFYNLU
- Air-drying: To air-dry paper records, hang the paper on lines. You may want to interleave pages with special blotter paper as a preliminary step before hanging the paper. Anchor: #KRHBRDRY
- Rooms used for drying areas should have good air circulation and low humidity. Use fans, dehumidifiers and/or air conditioning. Handheld hair dryers (set on cool) may be used to speed drying. Anchor: #ERMOESCE
- Freeze-drying: The most effective procedure for stabilizing water-damaged records and archival materials is to blast-freeze them to a temperature of -20 degrees F, and then dry them using a thermal vacuum process.
Freezing allows time to estimate recovery costs, prepare and coordinate subsequent steps in the drying and recovery operation, and clean up the disaster site. In addition, freezing stabilizes water-soluble materials (inks, dyes, etc.), that may disperse during air drying.
Thermal vacuum drying causes water to pass from a frozen state to a vapor without returning to a liquid form. It reduces stains and removes the odor caused by smoke. Determining a source for thermal-vacuum drying services should be part of disaster planning. It is recommended to use available recovery services which comply with state procurement standards.
Microfilm - In recovering microfilm or microfiche, speed is essential to avoid the breakdown of film emulsions and the onset of bacterial growth which destroys images.
- Anchor: #MGOBWYOY
- Immerse the film in clean, cold, preferably distilled water. Send the film to a professional laboratory for cleaning. Generally, film should not be frozen because ice crystals may harm it.
Electronic Media - Off-site storage of backups is the best possible disaster recovery strategy for electronic records. If electronic records on portable media are water-damaged, do not use them until they are thoroughly clean and dry, and their housing or containers have been replaced. This reduces the possibility of damage to equipment, especially for disk drives.
Anchor: #i1014596Diskettes require the following steps:
- Anchor: #LMMHDQHE
- To clean and
dry, do the following:
- Anchor: #CLCVVIWE
- Drain. Anchor: #HCUBYGUX
- Remove from jackets. Anchor: #HSCTHOMY
- Rinse in distilled water. Anchor: #KEECYAHU
- Blot with lint-free cloth. Anchor: #SVYPECFT
- Air-dry approximately eight hours.
Anchor: #ADDWSKYJ - When diskettes are dry, do the following: Anchor: #BJMPOGYJ
-
- Anchor: #AVKCIVLS
- Insert into a new jacket. Anchor: #VANLVITV
- Copy information on damaged diskettes onto new diskettes. Anchor: #UCVUEDKL
- If information copies correctly, discard damaged diskettes. Anchor: #TKQNKRSP
- Clean copy equipment drive heads frequently to prevent permanent damage to the heads.
Magnetic Tape that has become wet has a good chance for recovery through these steps:
- Anchor: #ULAVTMHP
- Hand dry all external surfaces with a soft, lint-free cloth. Anchor: #FBBXGFSP
- Air-dry the tape using a tape cleaner or winder to run the tape from reel to reel. Anchor: #QOXHAYWK
- Consult a company that does magnetic tape restoration.
Compact Disk or other Optical Media require these steps:
- Anchor: #MCSYCXKN
- Clean any dirt or debris from the disk using a disk cleaning solution. Anchor: #MMWRASTG
- Hand dry all external surfaces with a soft, lint-free cloth.
Contingency Planning
Contingency planning combines emergency preparation with routine office procedures to help the office continue operations with as little interruption as possible after a disaster.
The real disaster is failure to prevent avoidable situations.
For effective contingency planning, do the following:
- Anchor: #UQUJOKEY
- Keep your office File Plan up to date. Know which records you are the Office of Primary Responsibility and be ready to carry out disaster recovery and salvage procedures for those records. Anchor: #HLTASKQI
- Be aware of alternate locations for the records. This information can be used to replace records quickly after a disaster. Districts and Divisions that are related by function are often alternate locations for the same record, even though only one office may be designated as the Office of Primary Responsibility. Anchor: #NYAVTGOC
- Part of a Disaster Preparedness Plan could
include:
- Anchor: #UMXCXBNN
- Notation of alternate locations on the File Plan, or Anchor: #DFQRMFGU
- Securing a copy of any procedure manuals or internal office procedural materials that may include information on alternate locations for records.
Anchor: #SWTUNVUX - Keep a current list of contact persons for various types of records. Anchor: #SPSKMYYR
- Keep identification and protections for vital records current. Carry out vital records protection as described in Chapter 12, “Vital Records” in a disciplined manner. Anchor: #EYGQEWGS
- Back up electronic records routinely and frequently. Dispersal of a second backup copy at another, off-site location is a good protective measure.