Simulation Leads to Risk Management Improvements

		At a Glance
	A large-scale emergency at a fab could easily result in millions of dollars in losses. The chances of encountering an emergency large enough to activate several different response groups at once is very low, but such an emergency can have a very great human and financial impact. A large-scale simulation drill was performed at Motorola-Austin to identify the improvements required to effectively handle such an emergency.

To be successful in the semiconductor industry, companies must make every effort to efficiently manage risk, protect their assets and minimize production downtime. Systems must be in place to effectively protect the safety of company employees, the community and the surrounding environment. Companies also must protect the financial investment of a fab, which can be over $1B at full build-out. A large scale emergency at a semiconductor manufacturing facility could easily result in millions of dollars of losses. Most companies also face situations where demand exceeds capacity. This makes it imperative to minimize manufacturing downtime to meet the customer's needs and maintain financial stability.

Companies must plan for a wide range of situations including, but not limited to, fire, chemical releases, workplace and domestic violence, espionage, negative press releases and sabotage.¹ To respond to emergencies like these, Motorola-Austin uses the response groups listed in Table 1.

The chances of encountering an emergency large enough to activate several response groups at once is very low, but the human and financial impact of a large emergency can be great. Though the groups operate effectively on an independent basis, prior experience had shown cross-group communication and coordination needed improvement. To accomplish this, a five-hour, large-scale drill was conducted in May 1997 with all the groups listed in Table 1, excluding the Police Department. Representatives from the Travis County Local Emergency Planning Committee, a local hospital, a medical emergency training company and other Motorola facilities provided input during the planning phase and observed the actual drill.

The drill script

A script was developed to effectively exercise all the emergency response groups. The mission of each group was reviewed and a list of fundamental drill components established (Table 2).

To implement each drill component and maintain some sense of reality for the unlikely event of all components occurring simultaneously, a common thread was written into the script. The simulation team organized the script around a disgruntled employee who was upset because she was not accepted for an internal job opportunity. This employee was used as the force to activate, directly or indirectly, each drill component in the script.

The first action was for the disgruntled employee to place a threatening phone call to the plant's main security console. The employee stated, 'That job should have been mine. That jerk hired his friend. If you don't do something about it, I will!' and then hung up. After 20 minutes, the employee placed a second phone call to security stating, 'I haven't seen anything yet; I warned you!' and then immediately hung up.

Table 1 Emergency Response Groups

Motorola Response Groups

Hazardous Materials Response Team

Medical Emergency Response Team

Crisis Management Team

Occupational Health Resources

Security

Plant Services

Human Resources

Public Relations

Environmental, Safety & Industrial Hygiene

City of Austin Response Groups

Fire Department with Hazardous Materials Unit

Emergency Medical Services

Police Department

The next action was to report a chemical odor in the semiconductor manufacturing area and for six victims to report to the on-site health center complaining of nausea from a strange smell. Then, the script called for three workers to discover the disgruntled employee sabotaging a chemical waste drain line in the sub-fab. Chemicals would begin spilling from the waste drain line, causing heavy fumes. Next, an altercation would leave each of the three workers injured and two unconscious.

When Motorola response groups entered the sub-fab to investigate the odor complaint, they would discover the situation and observe the disgruntled employee holding the three workers hostage. At this point, security would notify the Austin Fire Department's Hazardous Materials Response Unit, Emergency Medical Services and Police Department. If this was not done, the script called for that action to be forced by the simulation team. The final two pre-defined script pieces involved the media and concerned family.

As a result of activating the city of Austin emergency response groups, crews from local television and radio stations would begin to arrive. Within an hour, the incident would be reported on television and radio. An employee's wife would hear the story and become concerned, paging her husband and then, after receiving no response, calling the facility. Her husband would be one of the workers held hostage.

As each drill component was activated through the script, the responders were permitted to take whatever action they deemed necessary. Aside from the pre-defined components, the simulation team was allowed to redirect the script based on the actions of the responders.²

Simulating the script

In addition to testing cross-group coordination and communication, a main goal of this exercise was to simulate reality as much as possible. Each response group regularly performs classroom and field drills to varying degrees. But those drills often are limited because of time constraints, no interaction with other groups, no requirement to actually carry out all response actions and an unrealistically narrow choice of action paths. Dynamic simulation was used to expose the responders to the emergency's true timeline, the pressure and stress of the situation, and the open-ended nature of choosing action paths during an actual emergency.

In dynamic simulation, a pre-defined set of drill components is established to activate groups, and the response is allowed to progress naturally within a specifically designed drill framework. This framework is made to allow for unintrusive, real-time monitoring of the response groups and to anticipate and adjust to all possible response actions. For example, the extent of the chemical spill was determined by the speed and effectiveness of the responders. The main components of the dynamic simulation framework for this drill are outlined in Table 3.

Safeguards for manufacturing

The site on which this drill occurred is a major semiconductor manufacturing campus with five manufacturing facilities, ~5,500 employees and a 24-hours-per-day, 7-days-per-week manufacturing schedule. Most of the drill activity took place in one of the sub-fab areas. There were seven other activity areas including: the manufacturing area, media congregation point, security console, on-site health center, crisis management center, response group congregation point and centralized simulation team command center. Because of the operations at this site and the nature of dynamic simulation, it was critical to put safeguards in place to ensure there were no inappropriate disruptions to normal site operations.

Table 2 Drill Components

Chemical Release

Work Place Violence

Hostage Situation

Multiple Victims

Media Crews On-Site

Concerned Family Members

Two months prior to the drill, the senior simulation team members briefed the operations managers on the main drill components and the simulation system. They emphasized the planned safeguards and obtained approval to move forward. The operations managers chose to be active drill participants and, therefore, were not given any specific script information. After 95% of the drill planning was completed, a pre-drill announcement was distributed to all senior managers and emergency response personnel. The announcement stated that a drill would occur within a three-day window and described the planned safeguards. No information about the script was included.

As an additional safeguard, the simulation team practiced performing the script and using all associated equipment in a 'dress-rehearsal' one week prior to the drill. Potential problems were identified and corrected. On the day of the drill, additional safeguards were implemented. First, the senior simulation team members, along with the operations managers, attended the morning manufacturing meetings to state that the drill would begin within two hours. The safeguards were reviewed and general questions answered. Again, no information about the script was revealed.

Then, the simulation team began to mobilize for the drill and put the final safeguards in place. Though many steps were taken to communicate the existence of the drill and monitor all associated activity, there was still the potential for somebody to mistakenly activate the building evacuation system. Because this would result in a significant financial loss to manufacturing, the building life safety systems were placed in a supervised manual mode. System technicians could then manually confirm a situation before evacuating the building. This is a common practice when troubleshooting the life safety system and would only have added a minor delay in an actual emergency.

Drill marshals, some from other Motorola sites, were used at each activity area to monitor the situation and ensure there was no inappropriate activity. For example, a drill marshal was assigned to the security console to ensure that no drill-related calls were placed to the actual city of Austin 911 system. A simulated 911 system was created for the drill. If an actual emergency occurred, the marshal could identify the event as non-drill-related and security would place a true 911 call. All marshals and other simulation team members wore identifying T-shirts with the logo 'Multiple Emergency Simulation Team.'

Table 3 Dynamic Simulation Components

Pre-defined script for the main drill components with approximated times

Alternate script options for each potential action taken by responders and adjustments made by the simulation team.

Actors playing pre-defiined and impromptu roles as necessary

Drill marshals at each main activity area to observe and to communicate via radio and video camera with the simulation center

Centralized Simulation Team Command Center: a) television monitors providing live links to all seven response areas to allow for real-time monitoring, input and later diagnostic. b) simulation phone operators fielding all drill-related phone calls to allow for real-time monitoring, input and later diagnostic.

Signs stating 'Emergency Drill In Progress,' with simulation team contact numbers, were placed in the lobbies and at entrances of the main spill/hostage scene. The facility's internal phone operator was told that questions about the drill should be forwarded to the simulation team command center.

During the dress-rehearsal, the simulation team noticed the responding entry teams would have to travel a long distance from the response group congregation point to the spill/hostage location. To prevent entry teams from getting lost in the maze of hallways and doors, and from being seen by personnel who might not have been aware of the drill, designated simulation team escorts were assigned to travel with the entry teams. The escorts wore the standard identifying T-shirts and carried a 'Drill In Progress' sign.

Creating the scene

Organizers of this drill wanted to simulate reality for the responders as much as possible. This meant that the simulation team had to create a system that would express the script in a true-to-life manner.

Volunteers were obtained from within Motorola and several city of Austin departments to play the actor roles. Some actors had experience in emergency response or their scripted role, but many had no related experience. All actors were instructed to stay in character during the entire drill to maintain the sense of reality. The simulation team pre-scripted the 15 roles listed in Table 4.

The 911 phone operator actor was a senior member of the Fire Department who was very familiar with standard procedures for emergency phone operators. Also, one of the media reporter actors was a city employee who had extensive experience dealing with the media during emergency events. The other actors were Motorola employee volunteers.

Table 4 Pre-Scripted Actor Roles

One Disgruntled Employee

Three Workers Taken Hostage

One 911 Phone Operator

One Concerned Family Member

Six Chemical Odor Victims

Three Media Reporters

Several acting roles were created or expanded on an impromptu basis as the response developed. These included allowing one chemical odor victim to locate the media and begin speaking negatively about the situation, and adding the roles of the concerned family member's mother and hospital employees contacted by the responders.

The chemical odor victim actors were actual manufacturing employees. They were briefed by professional medical consultants on how to act for their exposure level and were given cue cards containing information on their vital signs. When a medical responder assessed a victim, the actor would state the appropriate information for the situation. As an additional test for the medical responders, one chemical odor victim had only 'sympathy symptoms' with no exposure.

The spill/hostage scene contained most of the props. A broken waste drain line was hung alongside the actual drain lines. Large sheets of plastic were used to simulate the chemical spill to minimize post-drill cleanup. The plastic was stretched from the broken drain line down to the floor. As the event progressed, the plastic was extended further and further along the floor until the source of the spill was eliminated. Chemical vapor cue cards indicated odors and gas monitor readings to entry team personnel.

The disgruntled employee actor had a two-and-a-half foot long industrial wrench that she used as a weapon against her hostages and any approaching responders. The hostages wore 'special effects' make- up applied by the medical consultants as appropriate for each actor's injuries, ranging from minor bruises to a major head laceration. The hostages were briefed on how to act for their injuries and given vital sign cue cards. Other actors also were briefed for their roles and given notes on how to react to different situations.

Fig 1 Part of the drill preparation was to clearly define communication flow.

The arrival of the city of Austin response groups was a major script component and had to be coordinated to appear realistic. Although the city responders were not given details about the script, they were given the anticipated time of the 911 call and the number of response vehicles needed. This allowed them to stage their personnel and vehicles just down the street from the site. When the 911 call was placed, the city simulation team member called for units in a similar manner as a real 911 operator, but he used a cellular phone and called the lead vehicle directly. Then the city responders were able to arrive on site in a realistic time frame without unnecessarily driving at high speeds with sirens.

Monitoring and adjusting the script

To anticipate and complete script changes without disrupting the responders, and to perform post-drill diagnostics, the simulation team had to create an unintrusive system for monitoring all drill activity. First, the team analyzed the communication procedures of each response group. Then, the team strategized on what information, and in what form, would allow the most complete and beneficial diagnostic. The system consisted of video cameras, radios, phones, tape recorders and video cassette recorders. As the drill progressed, marshals monitored the response activity and communicated with the simulation team command center. The responders' action paths were noted and all prop, actor or script changes were made instantaneously.

Each marshal had two handheld radios. One was programmed to a hidden channel for simulation team communication only, and the other was used to monitor the response groups on their dedicated emergency channel. In the simulation team command center, standard tape recorders were used to record radio communication of both the simulation team and the response groups. The recorders ran continuously, and the tapes were replaced as needed.

Video cameras were installed at each of the seven response areas and wired back to large televisions in the simulation team command center. This allowed command center personnel to observe the actions and sounds of all the responders instantly and simultaneously. It also provided a one-way backup communication system for response area marshals to the command center. A combination of VHS and security video cameras were used. Video cassette recorders were connected to each television in the command center to tape the activity. The VHS cameras also taped the activity as a backup.

Phones are a major communication tool during an emergency. Because of the size of the facility and the number of responders and available phones, the simulation team concluded it would be extremely difficult to monitor all phone communication in a completely unintrusive manner. Though not realistic, a simulation phone center was established to monitor this communication in the least intrusive way possible. A phone number was dedicated to this use and was communicated to all responders prior to the drill. The responders were instructed to funnel all phone communication through this number.

Two operators staffed the simulation phone center. Multiple-line speaker phones were programmed to roll over all calls from the dedicated number. Each phone was equipped with an in-line tape recorder. When a call came in, the operator would start the tape recorder, answer the call, then ask who was speaking and to what number they would like to be connected. As appropriate, the operator would connect them to internal numbers, outside numbers or to an actor phone station in the simulation team command center.

Information on each call was immediately logged by the operators to provide an up-to-date summary for the marshal. The operators were instructed to provide no information about the script or current response activity. When connecting the caller to his or her party, the operator would then either release himself or herself from the call or continue monitoring the conversation via the handset or speaker. The decision to continue monitoring was based on the impact of the call on the drill activity.

The simulation team

The simulation team consisted of 42 members. This team had a small core group that worked together for several months and a large support group that came together just before the drill. The core personnel were mostly experienced emergency responders from various Motorola facilities. The city of Austin hazardous materials engineering manager also was a member of the simulation team. He provided the city's perspective during the planning and diagnostic phases, and coordinated the involvement of the Fire Department and Emergency Medical Services.

During the drill's development, the simulation team was organized into two sub-groups: a script group and a logistics group. The script group developed the fundamental drill components and wrote the pre-defined script and alternate script options. The logistics group arranged for all the equipment and personnel needed to support the script. They developed logistics planning sheets for each activity area, which were used as checklists during pre-drill planning and drill-day setup.

Table 5 Results Scope From Conducting This Drill

~80 Emergency Responders Participated

~20 Hours of Video Tape

~6 Hours of Audio Tape

9 Debriefing Sessions

16 Follow-Up Debrief Summaries

15 Filmed Post-Drill Interviews

For the implementation of the drill, the simulation team was reorganized. A marshal was assigned to each activity area and was responsible for all equipment, personnel and script adjustments in that area. Additional personnel were assigned as necessary to be actors, camera operators, camera assistants, escorts and general runners. A drill director also was established for overall orchestration of the event. The simulation team communication flow is shown in Fig. 1.

A simulation team reference book also was developed. It contained a schedule of events, phone and radio contact lists, a site map with details of all activity areas, logistics planning sheets, the script and follow-up debrief forms.

Results

After the drill was completed, each marshal held a debriefing session at his or her activity area. Simulation team and response group leader debriefings were held the next day. Several weeks later, post-drill interviews were filmed with many of the drill participants. Extensive notes were recorded at each of these sessions. In addition, an extensive review of the audio and video tapes was conducted. Table 5 illustrates the scope and complexity of the results from conducting this drill.

Lessons learned

After compiling the results of the drill, a thorough diagnostic was completed and an action plan created. The action plan covered the response groups as well as the simulation team. In general, all individuals involved felt that the drill was a positive experience and that the simulation system was effective. Many positive items were noted including: the simulation system kept up with the changes in activity; everyone was calm and professional; the nurses handled multiple patients well; senior Motorola managers were kept informed as required; and all response checklists and pre-defined procedures were followed.

Improvement opportunities for response groups included continuing to enhance cross-group communication, refining checklists and procedures, updating contact lists, conducting additional training with the city of Austin and adding to existing equipment. Improvement opportunities for the simulation team included clearly labeling props or communicating ahead of time what props will be used, eliminating the identifying T-shirts for the actors since this is not realistic and improving communication ahead of time on how to use the simulation phone center.

Conclusion

A large-scale, multiple emergency drill using dynamic simulation was successfully completed involving Motorola and city of Austin response groups. The key components of this drill were writing an effective script, providing safeguards for manufacturing, creating a realistic scene and monitoring and adjusting the script. The drill generated a large volume of data to review and identified many strengths as well as opportunities for improvement. Annual drills of this magnitude and style will be institutionalized as part of how each group manages risk and protects its employees and other assets.

Acknowledgements

The authors wish to thank Doug Keitz, Rick Sivan, Motorola ESIH and Plant Services, and the city of Austin Fire Department. The authors would also like to especially recognize the efforts of all the simulation team members and the response group personnel.

References

1. Thomas M. Campbell, 'Decreased Workforce, Increased Risk,' CP&M, pp. 22-24, February 1997.
2. Stacy and Stamieszkin, 'Hoping for the Best, Preparing for the Worst,' Disaster Recovery Journal, pp. 57-60, Fall 1996.

Todd McCay is currently the environmental manager for the Motorola-Oak Hill facility in Austin, Texas. He has been with Motorola for eight years and has held positions as an environmental, safety & industrial hygiene (ESIH) engineer in the Oak Hill MOS 11 factory and the Ed Bluestein facility's ESIH Department. He received a Bachelor of Science degree in chemical engineering from the University of Texas, and is a certified hazardous materials specialist and incident commander. Phone: 512-895-3685 Fax: 512-895-3222 Email: todd.mccay@motorola.com

Gary DePinto is currently a fab operations manager for Motorola in Austin, Texas. During his 12 years at Motorola, he has held positions as a process and device engineer, engineering section manager, process and equipment engineering manager and director of the Motorola Facilities and Environmental, Safety and Industrial Hygiene Operations in Austin, Texas. He was a member of the technical staff at Motorola for six years. He has been in the semiconductor industry for 16 years and also has held process engineering positions with Texas Instruments and Advanced Micro Devices. Phone: 512-933-6352 Email: r10268@email.sps.mot.com