Management of Change in Automation

I don’t do many product posts here–usually only when I’m at a conference and the host company unleashes a year’s worth of announcements. But I met with Eddie Habibi, the CEO of PAS, while I was in Phoenix last week and got an in-depth look at the company’s latest product. I think this is one of those new products that push the automation envelop.

Management of change is a pain. Once in my career, I was in charge of the entire product change management system. I’ve lived with the results of undocumented change in control systems. I bet most of you have, too. PAS, through its Integrity platform, is trying to automate the system in such a way as to provide extensive tracking along with the ability for users to easily find the changes.

This week PAS announced the release of Integrity iMOC, “the next evolution in electronic Management of Change (MOC) software.” The company maintains that iMOC ensures that all changes made to a plant’s automation systems are detected and reconciled with specific management of change cases.

As Habibi told me, “While most countries have regulations governing management of change for such things as process chemicals, equipment, documents, facilities, and procedures, few have specified requirements for management of change for automation systems. As a general rule, if an automation change affects other items that clearly require change management, then a rigorous MOC process is to be followed. However, critical changes are often made to automation systems with no record of whether they were approved or not, leaving the systems vulnerable as these undocumented changes carry the potential to be destructive. Only by ensuring that all configuration changes (except for the most fundamental changes required to operate the plant, such as setpoints, outputs, and control modes) are included in the MOC process and reconciled against outstanding MOC cases, can companies be certain their automation systems are secure.”

“Undocumented and unapproved changes to automation systems have been identified as contributing factors in a number of process industry incidents and accidents,” said PAS President Chris Lyden. “With the advent of new viruses that affect the interaction of controls with the process, ensuring that all automation configuration changes are detected and reconciled with MOC cases is an essential element of control system security.”

Integrity iMOC provides a graphical environment for the definition of MOC workflows, which includes describing each state within the workflows, as well as the checklists and required transitions for the states. As an Integrity-enabled application, iMOC takes full advantage of the capabilities of PAS’ Integrity software, which maps the configuration of more than 50 different automation systems and tracks all changes to them. Additionally, it reduces the time required for engineers to acquire design information, since it automatically identifies all links and interdependencies for any automation entity under change management.

As part of the installation services for Integrity iMOC, PAS can integrate it with a plant’s existing MOC system, configuring it to act as a subordinate to that system. PAS can also implement an S95-compliant plant asset hierarchy to link MOC cases to specific parts of the plant as is often done in general MOC systems.

Stuxnet Debate and Siemens Response Continues

When Stuxnet first became public, I was impressed that Siemens immediately issued a release with explanations. We all know that the company must have many talented engineers who surely had been working on the problem. I was immediately taken to task by Dale Peterson in his blog for being too soft on Siemens. He believes that suppliers should provide a 100% fool-proof secured system–or so I took his comments to mean.

Well the public releases dried up. Then Siemens started to parse more information out. Here is a release from last week (I’m catching up on thoughts following two straight weeks on the road and getting a magazine out).

From a press release entitled “Update on S7 Vulnerabilities (Status June 13, 2011)”, Siemens stated, “Despite recent news reports, Siemens latest software vulnerabilities are not caused by malware (like Stuxnet), but by a weakness in communication functions of its Programmable Logic Controller (PLC) product, called S7-1200. The vulnerability was discovered by an NSS Labs researcher and resulted in an ICS-CERT security advisory.

“On Friday, June 10, Siemens released a firmware update of its S7-1200 PLC that eliminates vulnerabilities and improves the security and robustness of its S7-1200 product family. To download the firmware and to obtain more detailed information, please visit: www.siemens.com/networkbehavior-S7-1200.

“At this point, Siemens is not aware of any customers affected by the identified weak points found in its S7-1200 PLCs. The company would like to emphasize that it is fully committed to maintaining the highest quality products with the most stringent security standards. Siemens experts have been working closely with ICS-CERT and various user communities to continuously improve the Siemens industrial controller products. Siemens continues to recommend to all its customers that they implement the appropriate security measures (e.g. firewall, secure switches and gateways) in their facilities that are typically separate from the actual PLCs. Find more info at www.siemens.com/industrialsecurity.

A”s a further precaution, Siemens controllers, including the S7-300/400 families, are being tested against the discovered vulnerability scenarios. Today, Siemens can already exclude any vulnerability of the S7-300/400 against the “denial of service” scenario. Ongoing and extensive tests of further security scenarios are currently underway in our R&D labs. Depending on the results of those tests, the company will react accordingly. If any customers have concerns that an unauthorized person has been able to record an online communication between the engineering PC and the PLC, the company recommends an immediate change to the PLC password.”

As fate would have it, I had lunch with noted security expert Eric Byres at the Honeywell User Group on June 14. He had just written a series of blogs here, here and here in which he took Siemens lawyers and public relations counsel to task for not being forthright enough in that statement.

Cyber Security Discussed at Siemens Summit

After that, Siemens announced that it would feature a forum and discussion on cybersecurity at its Automation Summit. And none other than Byres himself is one of the featured speakers. The list includes “a host of Siemens technical and product specialists,” as well as Eric Byres, Byres Security; Mark Chambers, Astec Inc.; John deKrafft, AE Solutions; Joel Langill, SCADAhacker; Howard Page, McAfee, Inc.; Tyler Williams, Wurldtech Security Technologies; and Todd Stauffer and John Cusimano, exida Consulting, LLC.

This was enough to prompt Peterson to ask why Byres would deal with Siemens. Byres responds here.

There is no doubt that automation vendors need to step up and provide more security within their devices. On the other hand, it’s fair to ask how many customers have demanded it. Further, there are other vulnerabilities that are outside the purview of the supplier–policies, enforcement, network vulnerabilities, and more.

So, it’s fair to ask why the Siemens corporate spokespeople aren’t more forthcoming. But it’s also fair to ask why customers aren’t holding it–and all other systems suppliers–more accountable.

When Executives Diss Manufacturing

Sometimes manufacturing people read something from supposedly very smart and experienced people, and it just makes you wonder how companies ever survived at all. You may have heard of Bob Lutz. He is a famous “car guy.” Product development, and especially big, gas-guzzling cars, embody his legacy. I remember vividly his whining about a year ago when the market and regulators were promoting fuel efficiency. He still wanted to build large cars with big engines.

He has written a book, and The Wall Street Journal offered him a forum to promote it. I saw it online here. There are powerful insights, such as when he reveals management weaknesses at GM. Then there are stories that make manufacturing people cringe.

There is a type of manager who focuses on the minutiae and lets “the vision thing” slip. I’ve seen this phenomenon in many guises. I served two terms on a school board. For four years we had a group that focused on the big picture–hiring the right people and focusing on the direction we wanted to go and letting the managers do their thing. Then three new members were elected. They focused on such important details as where floor polish was purchased and who the junior class advisor would be. Needless to say, overall direction of the school district floundered.

Scott Adams captured the supreme caricature of the “micro manager” in his “pointy-haired boss.” Lutz captures it in unfortunate reality. “One of my favorite anecdotes about the long postwar decline of General Motors came from a senior executive in the advertising agency that served Cadillac back in the 1950s and ’60s. At the time, Jim Roche was head of the division. It was time to design the annual Cadillac Christmas card, and Mr. Roche instructed the agency to find something ‘heartland’—down-home American, an original work from a good artist. One painting found Mr. Roche’s favor: a snowy scene with a small boy pulling a sled upon which was tied a Christmas tree. The lad’s destination was a modest cabin on a hill, with a winding road leading up to it.” Lutz continues his story with the tremendous amount of wasted executive time as Roche proceeded to design the card one piece at a time. No wonder GM floundered.

Manufacturing Unimportant

Lutz really hit a raw nerve with me with this paragraph, “A car company, on the other hand, is one enormous, hugely complicated organism that has many moving parts, all closely interrelated and interdependent. Many of the company’s activities are day-to-day: running the plants to produce components and assemble cars, procuring supplier parts, moving the finished vehicles to the dealers, billing same and booking the revenue. The operations portion of the automobile business has been thoroughly optimized over many decades, doesn’t vary much from one automobile company to another, and can be managed with a focus on repetitive process. It is the “hard” part of the car business and requires little in the way of creativity, vision or imagination. Almost all car companies do this very well, and there is little or no competitive advantage to be gained by ‘trying even harder’ in procurement, manufacturing or wholesale.”

Try telling Toyota or Johnson & Johnson that manufacturing doesn’t matter. When manufacturing slips up, it can cost the company tremendously in lost profit and brand image. On the other hand, companies such as the former Toyota or Subaru or many others who operate in a lean manner focusing on continuous improvement prove just how valuable manufacturing can be as a contributor to enterprise profits. I just toured an Audi plant and a Festo plant–each exemplifying a Lean culture and showing how manufacturing contributes to the enterprise. In this issue, I interviewed manufacturing executives from Phoenix Contact, Webasto and Beam Global Wine & Spirits who discussed how manufacturing is a contributor.

Follow this blog

Get a weekly email of all new posts.