Automation system security: Supply Chainmail

Between each connected device, from servers to automation components to cell phones, lies a potential cybersecurity risk. The industry could be forgiven for prioritizing performance over impenetrability as it scrambles to adopt technology, but industry experts suggest risk exposure is too often an afterthought. Whether malicious or accidental, internal, external or due to natural disasters, cybersecurity breaches can have a massive impact on operations that are dependent on connected systems.

“The digital revolution has decentralized industrial data and knowledge,” says Mark Stevens, vice president of global services for Digital Guardian, which specializes in data protection platforms. “In the old days, everything was centralized and you had to physically go to the file cabinet in an office. Devices can make people more productive in all areas, but that makes it really challenging for security professionals when everyone now has that file cabinet in their pocket.”

Although hacking and high-profile breaches make headlines, not all cybersecurity risks are malicious or even intentional. In fact, Jason Royes, senior security consultant for Cisco Security Solutions group, says it is impossible to attain 100% security.

“The adoption of mobile solutions and the emerging Internet of Things certainly complicates things,” Royes says. “The goal, then, is to remediate risks and raise the bar as much as possible. All the hopes for growth in most industries are tied to technology and more connectedness. They can’t resist it, but they have to be smart about it.”

To that end, solution providers and end-users have been revisiting the pre-Internet network security concept of AAA, which calls for authentication, authorization and accounting at each access point. If no solution is perfect, the accounting function in particular can help capture the details of a breach and help prevent another in the future.

Interestingly, although the proliferation of devices, automation and connections adds substantial risk, it is also the industry’s best hope for a new security paradigm. Virtualization of operational data can create redundancy and enable quick response to a brownout or flood, and cloud-based security software means companies of any size can benefit from the most sophisticated solutions available.

The best defense is a good offensive team
New solutions are more powerful by the day, but Tony Baker, product manager for network security at Rockwell Automation, emphasizes that security products only get you so far.

“We used to be in the age of point solutions, where you might apply some software every now and then and you’re in good shape. That’s no longer the case,” Baker says. “You can’t just install a new firewall every six months. Products without services don’t make sense anymore, so companies should be thinking about programs, not projects.”

The point solutions of old used to resemble concentric rings, with the crown jewels in the central vault, guards outside the door, castle walls and a moat. The contents of the vault were the usual suspects: intellectual property, pricing data, customer information and the like. But when your castle has Wi-Fi, a guard brings his own cell phone to work, and there’s a sale on moat-width planks, the spaces between each ring become a free-for-all.

In this environment, well-meaning employees can be as disruptive as any Trojan horse. Baker offers the example of a facility with two production lines. One was taken out of commission for service, and a technician set out to update its programming. After mistakenly downloading the program to the active line, production came to a halt. He was authorized, if not authenticated, for the specific task and the accounting function offered a record of the preventable problem.

Given the emphasis on lean, real-time operations, that record will likely reflect the huge cost of downtime for production and the entire organization. Companies continue to pursue tight connectivity between the enterprise layer, an enterprise resource planning (ERP) system, supply chain management (SCM), a manufacturing resource planning (MRP) system, and forecasts, all the way down to the operations level for automation and production. Tightening security to create fortresses around each connection can seem overwhelming, but it’s no longer just a single person’s job.

“It’s not a technological gap, it’s a cultural one,” Baker says. “Information technology folks and operations folks need to be in the same room and aligned around business objectives, not individual objectives. Customers who do that have the most success, and they have almost created a single business as manufacturing IT folks report to the same place as enterprise IT.”

Baker also suggests IT security experts should shift their thinking from exclusive to inclusive. “Some IT folks feel more secure if they keep personal devices and other data hubs separated,” he says. “I would argue that you actually have more control if everything is included under the same umbrella so there are no surprises. Or if there are, you will have the benefit of your accounting practices to monitor what people do in your network.”

Protecting links in the exposure chain
Most industries are playing catch up, Baker says. So many devices are connected to various systems, and there’s not always a good understanding of what they are, what versions of software they’re running and what version of firmware is installed. Even with tight control of its own equipment, a company that does electronic business with any other company assumes some of that entity’s risk.

“You can’t make any assumptions about how a third party will handle and protect data and information,” Royes says. “From a security standpoint, you have to treat them as untrusted, for lack of a better word.”

Once again, these types of connections might divide security specialists, some of whom prefer distance and some of whom prefer to keep their friends close for added visibility into their mutual cyber-enemies. In either case, security is a shared responsibility in the supply chain.

At the opposite end of the spectrum of supply chain risk is the individual semiconductor, where ones and zeros bounce from one electronic circuit to the next. Because they are virtually everywhere, even an uninterruptible power supply (UPS) presents a potential vulnerability.

“Most people’s approach to industrial controls cybersecurity is to ‘bolt on’ a hard shell around the system with IT-centric technologies. In this approach, the controls remain vulnerable,” says Robert Bergman, co-founder and vice president of sales and business development for Bedrock Automation, a member of the Control System Integrators Association. “Our cybersecurity is baked in, starting with encrypted microcontrollers at the core of every system module, extending all the way to standalone power and UPS products. I am not aware of any other UPS or power module that has any level of cybersecurity, much less this degree of embedded cyber security.”

It’s important that every last component is secure, but the traditional concept of concentric rings still has value. In fact, in terms of deliberate attacks, the outermost perimeter of a company’s security posture is still one of the most important layers.

“Five years ago, it was all about the insider, the Edward Snowden. Now it’s all about outsider,” Stevens says. “Cyber-threats are trying to get into the system, get credentials and become a user.”

Generally, the greater the value of sensitive information, the greater the attacker’s effort, but many hackers are simply canvassing for easy victims. It’s relatively easy to put together a million-dollar unlawful operation to go after data, Stevens suggests, and data theft is a lucrative industry. “But those people are also looking for the low-hanging fruit,” he says. “A thief will knock on the door, and if the security is robust, they will move along and knock on the next door.”

Natural disasters
Floods, earthquakes, power outages and political unrest are rarely so passive, and there’s nothing virtual about the destruction of physical assets. However, comprehensive operational data at the controller, facility and network level can ease recovery.

“A best practice is to always use a sort of industrial demilitarized zone between the enterprise layer and the plant floor,” Baker says. “You never pass sensitive data from the floor to the ERP without passing through that DMZ, so if the ERP goes down, your production doesn’t. You might not get real-time data for KPIs, but you’re not stopping production.”

If a plant or supplier are offline because of a natural disaster, visibility at the ERP level and the overall supply chain make it possible to react very quickly. If there is already an approved secondary supplier, Baker says, weeks of response can be reduced to hours. Frequent backups of controller programs can also help recreate identical operations elsewhere, or when the site comes back online.

“If a hurricane hits the manufacturing platform, lots of other systems are affected, and all of those components need to be brought back up,” Baker says. “Real-time systems help retain the state of operations at the time of shut down, and might even indicate the single point of failure that brought the whole system down in the first place.”

More mobile infrastructure directly translates to agility when disaster strikes, Royes agrees. All the same, the fact remains that more technology and more vulnerability go hand in hand. At least for now.

Companies mentioned in this article
• Bedrock Automation
• Cisco Security Solutions
• Digital Guardian
• Rockwell Automation