One of the most interesting challenges of working as Chief Information Security Officer in a utility company is the variety of infrastructure types that supports the business process. I refer here to the infrastructure that supports real-time management systems for generation transmission and distribution of energy and the system that are responsible for coordinating the pumping of water to individual households and industries.
The implementation of a information security management system that includes this kind of critical infrastructure to the business processes provides a number of interesting challenges which are not covered in the conventional security model for IT processes:
Information Security risks associated to the delivery process of energy and water utility services process, can lead to disruption of both services for a large number of people in a country. If errors in the handling of SCADA equipment have been responsible of cascading effects that collapse most of the electrical system of a country, what if someone is doing an identity theft in the energy SCADA system and performs tasks such as increasing the rotation of the generation turbines, increasing the energy flow exceeding the capacity of a transmission line or simply turning off the turbines of a power plant? Imagine the chaos that would plunge a country or region.
What if in the water tanks of a city begins to overflow its maximum level and the pressure causes the pipes bursting in the streets? Imagine scenarios like the following in every city: http://www.youtube.com/watch?v=kbz_zxsJCfgfeature=related
The cost of repairing damage of any of the above scenarios is enormous. If we add the inability of the company to generate money for generation, transmission and distribution of energy, how much time passes before the company cease to exist?
SCADA systems have a very particular operating environment. Because they are real-time systems, data monitoring and orders sent to the RTU should arrive in the shortest time possible, since an additional delay of even 10 ms can mean a massive blackout by activation of the protections of a substation. Similarly, suppliers of these systems tend to provide support on these only on a specific configuration, which is usually not too safe and lacks basic security controls such as security patches, data encryption, authentication and non default configurations.
The architecture for a SCADA system is as follows:
The components are:
Remote Terminal Unit (RTU): The RTU is defined as a communication device within the SCADA system and is located at the remote substation. The RTU gathers data from field devices in memory until the MTU request that information. It also process orders from the SCADA like switch off a transmission line.
Master Terminal Unit (MTU):The MTU is defined as the heart of a SCADA system and is located at the main monitoring center. MTU initiates communication with remote units and interfaces with the DAS and the HMI.
Data Acquisition System (DAS):The DAS gathers information from the MTU, generates and store alerts that needs attention from the operator because it cancauseimpact on the system.
Human Machine Interface (HMI):The HMI is defined as the interfacewhere the operator logs on to monitor the variables of the system. It gathers information from theDAS.
Due to its criticality, SCADA operators are reluctant to implement any type of information security controls that can change the operating environment for the system. How to implement a security scheme that does not interfere with the functionality needed for the business process? We took the following items specified in the standards of North American Reliability Corp (NERC) Critical Infrastructure Protection (CIP) to implement controls for an Energy SCADA:
Project 2008-06 Cyber Security Order 706
CIP0022 Critical Cyber Asset Identification
CIP0032 Security Management Controls
CIP0042 Personnel and Training
CIP0052 Electronic Security Perimeter(s)
CIP006-2a Cyber Security Physical Security
CIP0072 Systems Security Management
CIP0082 Incident Reporting and Response Planning
CIP0092 Recovery Plans for Critical Cyber Assets
For point number two, we took the same table to classify information assets for the corporate information security management system and applied it to the energy processes:
a) Generates loss of confidentiality of information that can be useful for individuals, competitors or other internal or external parties, with non-recoverable effect for the Company.
b) Generates loss of integrity of information internally or externally with non-recoverable effect for the Company.
c) Generates loss of availability of information with non-recoverable effect for the Company.
a) Generates loss of confidentiality of information that can be useful for individuals, competitors or other internal or external parties, with mitigated or recoverable effects in the long term.
b)Generates loss of integrity of information internally or externally with mitigated or recoverable effects in the long term.
c)Generates loss of availability of information with mitigated or recoverable effects in the long term.
a) Generates loss of confidentiality of information that can be useful for individuals, competitors, or other internal or external parties, with mitigated or recoverable effects in the medium term.
b) Generates loss of integrity of information internally or externally with mitigated or recoverable effects in the medium term.
c)Generates loss of availability of information with mitigated or recoverable effects in the medium term.
a) Generates loss of confidentiality of information that can be useful for individuals, competitors, or other internal or external parties, with mitigated or recoverable effects in the short term.
b) Generates loss of integrity of information internally or externally with mitigated or recoverable effects in the short term.
c)Generates loss of availability of information with mitigated or recoverable effects in the short term.
a) Generates loss of confidentiality of information that is not useful for individuals, competitors or other internal or external parties.
b) Generates loss of integrity of information internally or externally with no effects for the company
c)Generates loss of availability of information with no effects for the company.
From the previous table, we assigned controls to implement and ensure the security level for the asset. For point 3 and 4 we adopted all definitions from the corporate Information Security Management System. See all the required controls here: http://www.nerc.com/files/CIP-003-1.pdfand http://www.nerc.com/files/CIP-004-2.pdf.
The biggest issue here was authentication and clear-text traffic. Many devices from our SCADA system did not support authentication and also information was sent using cleartext protocols. Every time we tried to introduce a VPN or crypto level-2 devices, the network latency increased and functions of the system were degraded, which is why we had to remove those controls. When we askedour vendor for those controls as native functions for the system,we receiveda request to purchase the next version of the SCADA System.
The corporate antivirus didn't work because it consumed all the resources of the DASand the HMI. Same happened with the Host IPS. The solution we found for the problem was SolidCore S3 product(http://www.solidcore.com/products/s3-control.html), as it was non-intrusive, did not add extra layers and virtual devices to the operating system and controlled very good the zero-day problems.
For configuration changes, we established a weekly maintenance schedule in which the service of the SCADA system would stop for three hours changing the operation mode to contingency, so the IT operators could perform screening for viruses, install security patches and modifying security baselines. If the change was not successful and the system is degraded, the changes were removed and tried again the following week. This was not an easy task, because the vendor would not support us and we had to learn a lot on how the system components worked.
For point 5, We tried to redraw the SCADAnetwork socritical traffic would not mix with other type of traffic. For wireless devices, we managed to implement 802.1X authentication.Wedivided theSCADAnetwork into the following perimeters:
CiscoFirewall Service Module inside Catalyst 6509 with VSSsupervisors (VS-6509E-S720-10G) gave us the required bandwith and no disruptions were presented within the SCADA environment. It also have IPS (IDSM-2) that sends the alerts along with the log firewalls to our RSAenvision correlator.
For point 6, all the place has armored doors, CCTV, biometric authentication and security guards patrolling around the physical perimeter.
Now we are able to managethesecurity controls inside the corporate IT network and the SCADA systems. I still know that I have many things to dototo achieve the other points of NERC, butstill will be an interesting and challenginggoal.
-- Manuel Humberto Santander Pelez | http://twitter.com/manuelsantander| http://manuel.santander.name| msantand at isc dot sans dot org
(c) SANS Internet Storm Center. http://isc.sans.org Creative Commons Attribution-Noncommercial 3.0 United States License.