Automation Project Survival Guide

AUTOMATION PROJECT

SURVIVAL GUIDE

Ideas to help you land on your feet

BROUGHT TO YOU BY:

TABLE OF CONTENTS

3 12 Points to Consider Before Even Beginning Your Automation Project 5 Tips for Successful Project Development 7 Nine Tips for Automation Project Managers 9 Four IT Standards You Should Understand 10 Four Considerations for Upgrades & Migrations 11 Eight Ideas for Successful DCS Implementation 13 13 Suggestions for Control System Migrations 15 10 Steps to Creating the Perfect HMI 17 Safety: The Lifecycle Approach 19 Control System Security Tips 21 How to Avoid Mistakes with Control System Remote Access 23 Four Tips for Dealing with Wireless Latency and Bandwidth Issues 24 How to Properly Select and Vet a System Integrator AUTOMATION PROJECT SURVIVAL GUIDE

| 2

AUTOMATION PROJECT SURVIVAL GUIDE

12 Points to Consider Before Even Beginning Your Automation Project The first step in any automation project is the most critical one: Define your objectives. The more thorough and detailed this definition is, and the earlier in the process it can be achieved, the greater the likelihood that the project will be completed successfully. 1. Visualize success. Try to visualize what a project would look like if it were a stunning success. Take note of how it will affect all the people involved and write down any others you think it might touch. Take all of these people and put them on a spreadsheet column. Now in rows across,

write down the attributes they need in the machine/process. Use this when evaluating solutions and communicate shortcomings to those affected. Come up with workarounds or throw out the idea if the results won’t be acceptable. 2. What’s driving the project? You need to understand what is the most important motivation for doing this particular project and use that to guide your decision-making. 3. Helping people. Automation can do many things, but one must be aware that its purpose is to do real

WHAT DO THEY REALLY WANT, AND WHY? It’s essential to understand each person’s expectations before a project starts. There are three parts to this definition process: yy What outcomes or desired results does the project team want to achieve? yy What do they want the project experience to be like (for example, no production line shutdowns during the project or communicate updates by email)? yy How will they define quality, such as on time/on budget or increased production volumes or zero downtime, at the end of the project? Different people will have different expectations and they all have to be satisfied.


things in a given ecosystem. Keep in mind that the goal is to have systems engineered to serve humans, not the other way around. 4. Project definition is critical. Without doing true engineering work, everything you learned in school and in your career up to this point, you are not doing any project properly or professionally. By creating definition for the project and then verifying that the project will answer the need, you are on your way to successful project management. It is only the start, but without a properly defined starting point, it is difficult to complete (or defend) a meandering, ill-defined project that is meant to resolve a problem, address a challenge or complement your company’s engineering resources. 5. Start with the objectives. Don’t even begin to select suppliers and service providers until you’ve established a project’s objectives. Make sure everyone on the team agrees on what the project needs to achieve before it starts. If you don’t know where you’re going, you’ll never get there. 6. Get a second opinion. It pays to get a second opinion from an informed outsider like a system integrator or machine builder before finalizing project objectives—they’ll often

| 3

AUTOMATION PROJECT SURVIVAL GUIDE 12 POINTS TO CONSIDER BEFORE EVEN BEGINNING YOUR AUTOMATION PROJECT

do it for free. If you bring them in at this stage, so that they understand the history of the project, they can contribute to decisions that will improve the chances for a successful project. 7. Set rules for communication. Define what communications are expected at the start of the project— what is to be communicated, how it is to be communicated, what the milestones of the project will be and how often things should be communicated. 8. Talk to everyone. Interview the stakeholders from various factory disciplines, such as operations, maintenance, quality control, supply chain, shipping and management. They always have a stake in every automation project.

TECHNOLOGY COMES LAST Never start by defining technology-driven objectives. Use the following order: 1. Business objectives. What will the business gain from this project? 2. Operational objectives. How will this project impact operations— greater efficiency / better quality / compliance, etc.? 3. Integration objectives. Can data generated by this system be used by other systems? 4. People objectives. Skill development, ease in work pressures. Only when all of these have been defined can you establish the technology objectives. across a time matrix. Then that be-

9. Never assume. Don’t make assumptions about the ground rules— spell everything out in advance and define who is responsible for doing what.

comes a calendar for the schedule, sort of a compressed MS Project. If you color the boxes, it becomes a Gantt chart. Putting all your objectives (the completion of functioning subsystems, integration) into one simple chart keeps those objectives clear to the whole team.

10. Create a chart to keep objectives clear. Define the expected performance for each subsystem, and the expected steps to get there. Use Excel to list the task steps, and the hours/$

11. Spell everything out. If you want drawings in portrait vs. landscape mode, for example, or want certain brands to be used, such as for wire or PLCs or other components, state


that up front. If a requirement is not written down, then it likely will not happen. 12. Scope! Nothing is more important than a scope that reflects both the well-defined areas of the project and the gray areas of the project. The gray areas should have a general framework put together by the customer and the implementer, with benchmarks that clearly indicate when project reassessment should occur. This way scope creep can be managed to the benefit of both parties. 

| 4


Tips for Successful Project Development Project development is not an everyday occurrence at batch process facilities. To help ensure you are covering all the major issues involved in these infrequent work scenarios, here are some tips and considerations to facilitate a successful project startup. 1. Clearly identify the project specifications. What do you want to do? What is your existing process? Define operator involvement, quality control issues, interface points with other systems, and the technological capability available in-house. 2. Conduct a job risk assessment (JRA). Performing a JRA before the start of work highlights any hazards that could produce undesirable results to personnel or property. A safety assessment must be completed to ensure that the scheduled work can be performed in a safe manner and to address any hazards that are uncovered as a part of the review process. 3. Operator training is key. The operators must learn how to navigate and operate their process in the new control system. The training must be performed just in time (about two weeks before start-up) so that the information is fresh in their minds. During the instruction, it is critical that


the operators be trained using the operator interface graphics they will encounter. 4. Emphasize communications. Communicating with the site maintenance and operations departments is critical to the success of the project. Maintenance and Operations need to schedule their duties with enough lead-time to support the installation and startup activities. With enough time, maintenance can even contract back-fill support for the duration of the project startup activities. For operations, the work and vacation relief schedule will have to be organized so that enough operators are available to cut-over and startup the plant. This is especially important if a hot cutover is involved. 5. Have a detailed cut-over plan. Planning is crucial to any stage of an automation project. By putting together a detailed cut-over plan, the personnel performing the work will have a clear directive of the activities that need to be completed each day. The cut-over plan will help keep the activities on task and allow the project manager to assess the progress of the work, create workarounds for problematic situations, coordinate with the plant operations, and drive

the project to completion. A cut-over plan, at minimum, should include the I/O to be cutover and tested (including the order in which they are to be tested), any water testing through the process to verify configuration on the live plant, and the actual order of the first products to be run on the unit. 6. Devise a roles-and-responsibilities matrix. Defining the roles and responsibilities of all personnel and contractors involved in the project is key to delivering a successful project. By putting together this matrix and using it as a pre- and post-training reference for all staff, everyone involved will understand their responsibilities and perform the appropriate work. 7. Get management involved. Management at various levels, including upper management, needs to understand what is involved in the startup process and why it is critical to delivering on management’s expectations of the batch process facility’s operations. Communication and internal buy-in throughout the organization are very important aspects to a successful startup, and management’s visible support and connection to the project is critical to these aspects.

| 5

AUTOMATION PROJECT SURVIVAL GUIDE TIPS FOR SUCCESSFUL PROJECT DEVELOPMENT

8. Be thorough in examining outside support. Be sure to determine if outside personnel, such as system integrators, have experience in your industry. Is their knowledge transferable to the project? Evaluate their background and capabilities. What is


the range of services they provide? Are there any commercial issues outstanding? Check references. Consider cost, but understand that the lowest bid is not always the best. A good resource for companies looking to hire control system integrators is the Con-

trol System Integrators Association, www.controlsys.org. This organization not only validates industry expertise, but also supports dependable business practices by its system integrator members. 

| 6


Nine Tips for Automation Project Managers More than technical skills are required to successfully manage an automation project. It also requires communication and organizational skills, along with the ability to motivate a team of people from a variety of disciplines and different departments. Here are a few practical tips for automation project managers: 1. Project management resource. There have been thousands of words written about project management. If you think you need a refresher course, or expect to be assigned to your first project, there’s an organization, Project Management Institute, dedicated to establishing standards, providing training and certifying individuals in project management skills. 2. Welcome the bad news. Every automation project has things that go wrong, but the earlier you find out what the problems are, the easier and cheaper they are to fix. Nobody wants to hear or deliver bad news, but it’s important not to get defensive. Anybody on the team needs to be able to push the stop button if a project has gone off the tracks. Otherwise, you’re just gambling that things will come out all right at the end. 3. Keep simplicity top-of-mind.

complex for non-engineers to deal with. Make sure expectations are established early that will keep the needs of the people who will have to operate and maintain the systems a priority.

have to test it to prove that it works, not once but twice. It’s easy to get started on designing the tests by using a template. Equipment or systems should first be tested at the facilities of

Include people from these functions on the automation team and consult them early in the design and testing stages for new systems and equipment.

the integrator or OEM. This is called FAT (Factory Acceptance Testing), and its goal is to prove that the system design will work. Simulate various scenarios to find out how the system will react. The final testing stage, SAT (Site Acceptance Testing), is done when the system is delivered to the factory floor. Its objective is to prove that the equipment actually does work as designed and is producing product at the level required. Approve the testing plans early in the project so that everyone knows exactly what performance measures they need to achieve. Don’t rush the testing phase; make sure you leave enough time in the project schedule to accomplish the necessary tests. It’s

4. Be ready to adjust. As with any project, unrealistic projections, poor execution and just plain bad design can cause a project to fail. What is important is that when you begin a project, understand that there will be modifications necessary along with way. The final result is rarely as exactly planned. This is not considered a failure; it’s a realistic need to adjust and fine-tune as the project progresses. 5. Establish testing plans early. It isn’t enough to design a system. You

Looking for training? There’s a source to help you learn the ropes of project management or improve your skills.

http://awgo.to/028 Organization: Project Management Institute

Engineers tend to make systems too AUTOMATION PROJECT SURVIVAL GUIDE

| 7

AUTOMATION PROJECT SURVIVAL GUIDE NINE TIPS FOR AUTOMATION PROJECT MANAGERS

Close Lifecycle Methods Waterfall Agile Spiral Design V Other

Internal Kickoff

Requirements Development

Site Acceptance Test Factory Acceptance Test

Traceability

Specification

Design Subsystem

Integration Test

Unit/Module

Unit/Module Test

Development Project management “V” model, courtesy Control System Integrators Association. also important to make sure the right people attend the FAT; that includes the lead operator and maintenance tech, not just the manager. 6. Follow programming standards. Make sure that in-house programmers, system integrators and OEMs use the same PLC programming standards, such as OMAC and PackML. There’s nothing worse than custom code that has to be reworked at the last minute to make it compatible with a plant’s existing systems. Multiple approaches to programming can cost a company millions of dollars.


7. Communicate often. Don’t make decisions without consulting the team. Unilateral decision-making alienates the team, creates confusion and fails to take advantage of the unique expertise of the team members. Foster open communication and communicate frequently, so that everyone on the team understands the issues and is aware of any problems that need to be resolved. Establish a communications roadmap for vendors; check with them soon into the project to make sure it’s working. 8. Don’t be a roadblock. As project manager, it’s your responsibility to respond to information requests and

approve various aspects of the project in a timely fashion. Stay involved and be responsive to prevent delays in the project’s timeline. 9. Make sure you have bench strength. There’s nothing that delays a project more than a team member who gets assigned to another project and no longer has the time to devote to your project. Identify alternative resources early and have them ready to fill in if needed. That same rule applies to the system integrator’s team; make sure they’ve identified people with equivalent skills who can be assigned to the project if required. 

| 8


Four IT Standards You Should Understand Imagine a world without electrical standards, such as 110 V at 60 Hz, or 220 at 50 Hz, or a world where every phone had a different type of connection and required a different type of switchboard. Just as these standards are critical to the basic functioning of electrical equipment, there are also IT standards used daily to ensure optimal functioning of production systems in the process industries. There are four production-related IT standards of special interest to the processing industries:

• Tcontrol; he ANSI/ISA 88 standard on batch 95 standard for MES • Tand he ANSI/ISA ERP-to-MES communication; • The ANSI/ISA 99 technical reports in industrial cyber security; and • The new ANSI/ISA 106 technical report on procedure automation. These standards and technical reports define the best practices for imple-


menting automated and manual control on the systems that reside above the PLC (programmable logic controller) and DCS (distributed control system) level, and which perform the basic control that keeps production running. These four standards all share a common view of a production facility, providing a consistent terminology that makes it easier to compare plants within a company and across companies.

facturing operations management) specifications.

The ANSI/ISA 88 standard defines the most common and effective method for defining control systems for batch operations or for continuous and discrete startups and shutdowns.

Because these standards establish a commonly accepted terminology, functions and process models by which technical professionals are trained, and upon which solution providers develop applications used in batch and process production

The ANSI/ISA 95 standard defines the most commonly used method for exchanging information between ERP systems, such as SAP or Oracle, and the multitude of shop floor systems. It has also become the de facto standard for defining MES (manufacturing execution system) and MOM (manu-

The ANSI/ISA 99 reports define structures and policies for designing effective and secure networked production facilities. The new ISA 106 reports define the procedural control strategy for continuous production during upsets, switchovers, and other types of process changes.

operations (as well as discrete manufacturing), they should be of particular interest to those who are new to the field and those who seeking a refresher on the fundamentals of industrial processes. 

| 9


Four Considerations for Upgrades & Migrations Regardless of whether you want to increase productivity or shorten timeto-market, attaining success in these areas depends on the application of suitable automation technologies in a batch processing plant. Following are the principal steps involved in assessing your plant’s technology to gauge whether a technology upgrade or migration is in order:

1. Consider the full range of aspects that relate to your existing systems, such as: of unplanned plant downtime • Rand isk production stoppages; production or • introduce Ability to expand new products; to integrate with enterprise• Alevel bilitybusiness software and at what cost;

• Ongoing maintenance costs; AUTOMATION PROJECT SURVIVAL GUIDE

• the efficiency and produc• Etivity ffectofonplant personnel. Need for continuing support of the legacy system; and

2. In each case of upgrade or migration, return on investment plays a crucial role. A huge investment in hardware and application software is associated with the installed process control system, as well as the accumulated know-how of the operating, engineering and maintenance personnel. For this reason, the prime objective of any migration strategy should be to modernize the installed base gradually without any system discontinuity and, if possible, without any plant downtimes or loss of production that would negatively affect the investment return. 3. Assess the long-term security of existing investments. This assessment is important in order to maximize the return on assets (ROA). For this reason, every migration should include a robust lifecycle support strategy for the new system that considers not only the availability of the components, but also product warranties, on-site service and ongoing technical support. 4. Obsolescence. When deciding whether to upgrade or migrate to a

new system, there are two aspects of obsolescence to assess. In a migration, it’s important to understand the history of the technologies supported by the company behind the product under consideration. Does this company actively support the long-term lifecycles of products as they are typically employed in a process operation? Do upgrades have significant backwards compatibility? How often are upgrades typically released for this system and what is required for installation? For upgrades, it’s important to understand what the future outlook is for the system under consideration. With the significant maintenance and security issues tied to process control systems, you should always consider your risk of system obsolescence and the associated costs incurred with such a scenario versus the costs of moving to a better-supported system. The good news is that, in the process industries, most vendors are very aware of the long-term use of their systems by end users and thus tend to support their systems for multiple decades rather a single decade, as is more common with office IT systems. As newer automation technologies become core components of process control systems, be sure to talk with your supplier about their support plan for those newer technologies. 

| 10


Eight Ideas for Successful DCS Implementation Implementing a new distributed control system is one of the biggest and most complicated projects in a process control engineer’s career. Doing one successfully requires everything from a well-defined project document to good grounding practices. Here are recommendations for best practices and some pitfalls to avoid. 1. Standardize. Use of standard wiring throughout the system will make it for easier for others to understand and troubleshoot. Use standard, off-theshelf components for ease of stocking and reordering. If possible, have two sources for the products being used or purchase interchangeable brands. 2. Remember the basics. It’s the little things that can trip you up. Make sure you use proper grounding, proper grouping of signals and proper termination of electrical signals. Make sure you understand the supplier’s grounding requirements for your DCS system. Grounding principles need to be clearly understood by all automation engineers, not just the electrical staff. International standards can be misinterpreted. Instruments and the control system need to be grounded separately. Double check the grounding before powering up any DCS system to avoid any short circuits, particularly during


factory acceptance or site acceptance testing (FAT/SAT). 3. Is communication complete? While most automation suppliers have different software versions for communicating with the system, make sure they will transmit all the required information. Many systems only transmit the basic parameters, which means all diagnostic features will not be available. The introduction of the “Control in Field” concept, although not often used, has added some complications and needs to be thoroughly examined when implementing a DCS.

4. Structuring I/O. Since today’s electronics are available with high temperature specs and may be G3 compliant (conforming coating), the I/O structures should be moved to the field, reducing the rack room footprint and cabling cost. Communication links should be used over fiber optic, in a ring configuration to provide some level of redundancy, to interconnect the field I/O structures. Extended I/O terminal blocks (three to four terminals per channel) should also be used to allow field wiring to be connected directly, avoiding marshaling terminal strips with the related space, addi-

DEFINE IN DETAIL. Successfully implementing a DCS project requires that all stakeholders (operations, maintenance, project team, vendor, management, etc.) have a clear definition of what they want from the system. In both upgrading and installing new DCS systems, the best tip is to keep the end in mind. Good up-front engineering pays dividends. Automation technology can only assist us if we know what the needs are. Maintenance must know what reports and information they really require to do their work. Operations must be completely sure how they operate and what is the best way to do it. Don’t assume anything. Write everything down that’s actually required and all the things the technology can do. Be very specific. In the end, the best DCS is the one that best satisfies all the important requirements in the plant. Writing and signing this definition document should be the first step in any project.

| 11

AUTOMATION PROJECT SURVIVAL GUIDE 8 IDEAS FOR SUCCESSFUL DCS IMPLEMENTATION

tional cost, installation cost and the possibility of poor connections. 5. Dual purpose. The purpose of DCS is twofold. Centralized human control and interface to the plant as well as a centralized location for MIS info to the management network. DCS control should not include auto tuning of control loops other than simple on/off or start/stop functions. These should be the function of a local dedicated controller. Use the DCS to update the tuning parameters.


6. Good links. Distributed control systems are only as good as their communications links. Choose a very solid and reliable link between processing units. 7. FAT is where it’s at. Make sure you do a comprehensive and detailed factory acceptance testing (FAT) test before cutover. FAT involves experienced operations people interacting with engineering to validate graphics and verify that instruments in the configuration exist and will remain in service.

8. Use single server. Base the selection of a DCS system on its redundant capability. A single server system is preferred. Pay attention to the hardware license for client and server to avoid delays during a system or hard disk crash. Care must also be taken in selecting appropriate layered switches for communication. Make sure you properly configure trends and history data for future analysis. 

| 12


13 Suggestions for Control System Migrations As anyone who has been involved in a control system migration will tell you, it’s never an easy process. Whether it’s an upgrade, expansion, stepwise migration or rip-and-replace, the bigger and more complex the project, the more fraught with tension and risk. To help you get through the project with your sanity intact, Automation World readers share their recommendations and suggest pitfalls to avoid: 1. Determine strategy. Your migration strategy will depend on which type of automation you’re dealing with: scripts, workflow tools, policybased orchestration, configuration or control systems. The different activities that can be automated (provisioning, maintenance, proactive incident response, production execution, etc.) and the different degrees of automation (automating just a few actions, partial workflows or end-to-end) will determine your strategy in terms of resources, time scale, production stops, etc. 2. Virtualize first. Automation upgrades or migrations need to be scheduled properly in terms of system commission date to extend the warranty or for a vendor’s obsolete notice date. The best practice is to


conduct a virtualization of the new automation system. The future of automation will need virtualized infrastructure and platforms to deal with the IT spectrum, cyber security and better management capabilities. Virtualization has many benefits in terms of technology, investment, maintenance and lifecycle cost. 3. Take it one step at a time. Avoid changing the entire system or manufacturer if you are upgrading. Upgrading to the newer modules or systems of the same vendor provides a bit more reliability, since the basic architecture remains the same. 4. Don’t experiment. While innovation is important, there is a counterargument for doing what you know will work. If rip-and-replace is possible (and that means you have to stop the plant for several days, weeks, or months depending on the circumstances) and you know that it works, that is the best choice. But if you can’t afford a shutdown, then go for a step-by-step migration. Make sure you work with an experienced vendor and proven technology. 5. Three critical migration issues. When doing a migration there are three points to think about: how to update software and whether you

have the right conversion tools; what you need to do to avoid system failure or risk for the migration step; what is the expected lifecycle of the new system. 6. Make no assumptions. Try to foresee every small step in a migration implementation. Don’t assume anything. Every implementation is done to achieve some objective of the operation. The needs could range from some reporting or alarm functions to an action initiated due to alarm. Always visit the site to understand the requirements and the nuances completely. 7. Changing horses adds some complexity. The difficulty of a process migration usually increases when you change DCS suppliers since different brands often don’t have similar functions. Factor that into your timeline and risk assessment when weighing whether to switch vendors. 8. Start with data needs. First you need to understand what data the user will require and how quickly the data is needed. That should be the starting point in developing your migration strategy. The second priority is to determine the impact on the safety and productivity of the plant.

| 13

AUTOMATION PROJECT SURVIVAL GUIDE 13 SUGGESTIONS FOR CONTROL SYSTEM MIGRATIONS

9. Focus on controllers.The best strategy is to first upgrade the controllers, then replace the I/O chassis piece-by-piece going forward. Some I/O changes could be driven by other projects, such as a motor control center(MCC) replacement. 10. Do your homework. Do some up-front analysis to avoid creating problems for yourself by not choosing the right migration path. For example, migrating from one generation of processor to another one may not be a wise choice. Reviewing the instruction sets and information available about


conversions and manufacturer recommendations will give you insight into the difficulty of the conversion. If you do your homework, you might choose a different processor to make the conversion easier. 11. Technology education. It is important to educate everyone on the new technology. Remember, it is easy to use “old” thinking instead of changing practices to take advantage of the benefits of the new technology. 12. Decentralize. The architecture has to be critically reviewed and trans-

formed, keeping in view the improved performance of the local controllers. Your mantra should be to decentralize the controls as far as possible. 13. Aging equipment. Depending on the technology you have installed, when your equipment is more than 10 years old you will need to implement a rip-and-replace. If you are just making some modifications you can upgrade or make an expansion only. Most of the problems that arise during a migration are with the field equipment you have installed and control room facilities. 

| 14


10 Steps to Creating the Perfect HMI When developing HMI screens, realize that you are attempting to capture the essence of the machine or process, not just posting key automation variables and control mechanisms. Operational feedback is vital for efficient HMI screen layouts. Think of yourself as an artist, commissioned by manufacturing operations to create the HMI screens. 1. Less is more. It’s important to keep the HMI simple and with the operator in mind. It’s best when it’s self-explanatory and easily understood. Also, try to make the pages similar and follow the same page layout throughout. Avoid making the display too technical. It’s normal for engineers to try to give the customer everything, but with HMI, less really is more.

and content. If you get it wrong, your operator misses an indication, you lose money, or worse, someone is injured. The ”bad” screen is less than satisfactory: The layout is poor, the plant representation isn’t logical and the screen layout makes it difficult

of colors to allow actual device state and alarms to stand out. For alarming, choose colors that contrast with the normal process view so the operator will notice the change.

to locate the data. Poor selection of colors, excessive use of capitals in a serif font and repetitive use of units with all data values makes this a really difficult screen to read—especially at a glance or from a distance. Avoid colors that could create problems for people with color blindness. Minimize the use

review with a group of plant personnel to discuss any status notifications, events, alerts and alarms that need to be programmed, both from the perspective of an audio-visual action and an operations response. Step through the intended functional system, once as the designer, once as the user and

4. Plant review forum. Hold a design

2. Right-size displays. Don’t try to save money by selecting an HMI display screen that’s too small. It’s also important not to cram too much information onto a screen. Size the display according to the amount of information that is most important for the operator to see. Always discuss requirements with the equipment’s operators well ahead of time, not just with their managers. Operators usually have different needs and the success of your system depends on their usage. 3. Design tips. A good design requires careful use of layout, color


| 15

AUTOMATION PROJECT SURVIVAL GUIDE 10 STEPS TO CREATING THE PERFECT HMI

then invite at least two levels of users who will be interfacing with the HMI. Doing this prior to specifying equipment helps to identify the features that users will want in the HMI station. It also avoids surprises at point of commissioning. 5. Location, location, location. Real estate can be prime in a busy production area. Locate the HMI in a practical place, out of heavy traffic areas but accessible. Be aware of nearfuture projects in the area. Guard the HMI location so others don’t park or configure something else on top of the station. 6. Back up work periodically. Backups are especially important before implementing upgrades or changes. Software such as Norton’s Ghost Im-


age can be invaluable to support and maintain HMI systems. 7. Visualize the process. HMI graphics should illustrate the production process in the plant to provide better visualization to the operators, giving them a sense of the action that’s required. Use hardware that meets minimum requirements and keeps the number of failure points low and assures high availability of the system. 8. Only essential data. Make control and monitoring of the process simpler by selecting only the most essential information from the process database for the historian. This will reduce the load on the system and keep it from stalling or failing. Don’t forget the need for maintenance and make sure you schedule periodic backups.

9. Think about flow. It is essential to have a clear design approach to the HMI. Decide how the display blocks naturally flow and how sections need to be grouped together for the operator. Do not blindly follow P&I diagrams. The S88 functional hierarchy is a good place to start. Make paper-based designs to get a feel for screens, navigation and other requirements, and review with clients prior to designing and making electronic screens. 10. Alarm strategy. Alarming needs to have a well-articulated strategy. Alarms must be used for conditions that require intervention and must have a clear corrective action associated with each one. Anything else should not be an alarm. 

| 16


Safety: The Lifecycle Approach Production safety is generally thought of as a series of steps necessary to ensure safe interaction with industrial equipment. The process of identifying, agreeing upon and delineating those steps is where things tend to get complicated. That’s why international standards groups play such a significant role, as they set the guidelines for all of industry to follow.

a company contributing to projects. In addition, the standard also makes it easy for outside auditors and governmental agencies to follow the process. IEC 61508 can seem confusing at first, because its underlying philosophy is new for safety standards. Older, more conventional safety standards stipulated specific rules and specifications for making processes safe. IEC 61508

For the process industries, IEC 61511 is probably the most widely used safety standard, as it applies to those industries that base their safety systems upon instrumentation. The goal of safety-system design in IEC 61511 is for the process, whatever it may be, to go to a safe state whenever a process parameter exceeds preset limits. A New Way of Approaching Safety Understanding IEC 61511 means that you must know a thing or two about IEC 61508 — a functional safety standard that provides the framework for building industry-specific functional standards. IEC 61511 was created from the guidelines established by IEC 61508. The key point to understand about IEC 61508 is that it is designed to establish an engineering discipline that will generate safer designs and build safer processes. The uniform procedures built on these disciplines are contingent upon appropriate experts within


offshoots actually do—from project conception to maintenance to decommissioning. In essence, the standards specify safety lifecycle activities that need to be followed over the entire life of a production system. Safety lifecycle management provides a method or procedure that enables companies to specify, design, implement and maintain safety systems to achieve overall safety in a documented and verified manner. Four Phases of The Safety Lifecycle The IEC 61511 standard promulgated by the International Electrotechnical Commission specifies twelve steps in the safety lifecycle. These are segmented into four phases: analysis, realization, maintenance and ongoing functions.

and its derivative standards, such as IEC 61511, departed from this approach by being more functional, or performance-based. A principal aspect of this new approach to safety standards is that it leverages two fundamental principles: safety lifecycles and probabilistic failure analysis. Unlike previous standards that claimed to cover the entire lifecycle of a project, IEC 61508 and its

Safety Lifecycle I: Analysis Phase The analysis phase includes the initial planning, identification and specification of safety functions required for the safe operation of a manufacturing process. Specific activities include:

erform hazard and risk analysis: • PDetermine hazards and hazardous events, the sequence of events leading to a hazardous condition, the associated process risks, the requirements of risk reduction and the safety functions required.

| 17

AUTOMATION PROJECT SURVIVAL GUIDE SAFETY: THE LIFECYCLE APPROACH

•

Allocate safety functions to protection layers: Check the available layers of protection. Allocate safety functions to protection layers and safety systems.

requirements for safety • Specify systems: If tolerable risk is still out of limit, then specify the requirements for each safety system and their safety integrity levels.

Safety Lifecycle II: Realization Phase The realization phase not only includes design, installation and testing of safety systems, but also the design, development and installation of other effective risk reduction methods. Specific activities include: engineer a safety • Dsystem: esign and Design system to meet the safety requirements.

and develop other means • Dof esign risk reduction: Means of protection other than programmable safety systems include mechanical systems, process control systems and manual systems.

commission and validate • Install, the safety protections: Install

and validate that the safety system meets the all safety requirements to the required safety integrity levels. Safety Lifecycle III: Maintenance Phase The maintenance phase begins at the start-up of a process and continues until the safety system is decommissioned or redeployed. Specific activities include:


•

Operate and maintain: Ensure that the safety system functions are maintained during operation and maintenance.

and update: Make correc• Mtions, odifyenhancements and adapta-

tions to the safety system to ensure that the safety requirements are maintained.

the defined requirements for each phase of the safety lifecycle. Activities for Phases I to III are typically carried out consecutively, while Phase IV runs concurrently with the other phases. However, like all models, the safety lifecycle is an approximation.

Conduct review • Dand ecommissioning: obtain required authorization before decommissioning a safety system. Ensure that the required safety functions remain operational during decommissioning. Safety Lifecycle IV: Ongoing Functions Certain functions are ongoing. Examples include managing functional safety, planning and structuring the safety lifecycle, and performing periodic safety system verification and safety audits over the whole lifecycle. Specific activities include:

safety, safety • Massessment, anage functional and safety audit: Identify the management activities that are required to ensure that the functional safety objectives are met.

and structure safety lifecy• Plan cle: Define safety lifecycle in terms of inputs, outputs and verification activities.

safety system: Demonstrate • Vby erify review, analysis and/or testing that the required outputs satisfy

Bottom Line: A Requirements Definition Readers should note that the standards define requirements for safety management, rather than system development. Not all safety lifecycle phases will be relevant to every application; management must define which requirements are applicable in each case. The standards do not prescribe exactly what should be done in any particular case, but guide management toward decisions and offer advice. 

| 18


Control System Security Tips Recognizing that the biggest security risk to your control system assets are the operators who interface with the system on a daily basis is the most important step to successfully securing your systems. For a thorough analysis of your risks and setup of reliable control system security technologies and processes, consult an industrial control system security expert such as scadahacker.com, tofinosecurity.com, or industrialdefender.com. Following are the ground level security steps that a batch process facility should implement at a bare minimum: 1. Assess your systems. Compile an accurate list of all the assets in your plant: make, model, and serial number. Where are your computers? Where are your PLCs? It’s difficult to secure something when you don’t know it exists. This should be a high-level assessment in which you go through your plant and figure out what is high risk and what is low risk, which is determined by two key factors: how likely is a problem to occur? How serious is the problem? For example, if something happened to your chlorine tank, it would be really ugly. That chip pile, not so ugly. Get a feel for the significant risks. Where do you have to focus your effort? The answer is going to drive your decisions and your capital allocation.


2. Document your policies and procedures. No company operates in a vacuum. Each company will have a series of policies and procedures for things like safety and performance, reliability, and change management. Lay those out and understand how they impact control systems and security, and then build on that to create a set of additional security requirements. 3. Start training. No one is going to follow policies unless they know about them and understand why they are necessary. All levels of employees that interact with the control system need to understand what an attack looks like and how to respond to one. You should end up with a matrix of training for the various levels of users; it doesn’t have to be onerous, but it has to be done.

4. Understand your traffic flows. You need a diagram that shows all the things that require intercommunication. Smart companies will have a comprehensive diagram showing that the accounting department needs data out of this area, and maintenance needs data out of this area, and so on. 5. Remember that SCADA security is used to control access. Access should be segmented to specific network resources, hardware resources, and HMI. Effective security practices should prevent access to all layers by unwanted external connections. 6. Leverage safety reports. Those responsible for safety, when they do reports and analyses, have done a good deal of the work needed to understand the security risks. 7. Use separate networks. Though this step is becoming less and less practical, some still advocate that the process control network be kept separate from business networks, and also isolated from the Internet. For this approach, which may not be viable in the longer term, utilize operating system (OS) implemented security, with active directory “domain group security” as the preferred approach. 8. Security in the operator interface should be considered broadly. With advanced human-machine

| 19

AUTOMATION PROJECT SURVIVAL GUIDE CONTROL SYSTEM SECURITY TIPS

interface technologies, security can be implemented for individual attributes. HMI should be the only accessible program, with user-specific exceptions, connected to the control operating system at a dedicated user station. All other resources for that particular terminal should be restricted.

ing solution that the port’s disabled for security reasons. When using port security, you can prevent unwanted devices from accessing the network. 11. Administer antivirus protection. Use an antivirus solution that is compatible with the installed SCADA software.

9. Use unique user accounts and passwords. All users should have unique user accounts and passwords to minimize the risk of unauthorized access. 10. Provide port security. With this approach, the Ethernet MAC address connected to the switch port allows only that MAC address to communicate on that port. If any other MAC


address tries to communicate through the port, port security will disable it. Most of the time, network administrators configure the switch to send an SNMP trap to their network monitor-

12. Open and facilitate communications between IT and process control groups. Roles need to be defined and an understanding of what each group needs must be accomplished so true collaboration can take place to begin and continue the process of enabling a fully functional control system with adequate security protection. 

| 20


How to Avoid Mistakes with Control System Remote Access As more operations aspects are tied to Ethernet networks and, therefore, are open to Internet-based access, the potential for greater collaborative operation and a freer work environment increases. But so does the potential for security problems. Following are some basic tips and considerations for achieving secure and reliable remote access:

start implementing various silos—be they applications or products—things get more complex. This is typical of problems that occur when automation products are implemented hastily, without doing proper research, planning, or analyzing current and future goals, or without realizing that implementing remote access monitoring for a facility is just step one of many.

1. Map out your project from the start. When companies fail to map out their projects thoroughly from the start, they often find themselves saddled with applications and automation products that don’t work cohesively as a single system. Once you

2. Anticipate network interactions. When people have installed devices on a proprietary network then try to use something different (e.g., Wi-Fi or another protocol), individual systems may conflict. Or they may just cancel each other out, so that there is no

communication whatsoever. More often you find yourself managing so many different applications, protocols, and systems that you have more work and headaches than you imagined possible. This issue can be avoided if you select a network that is open and allows everything to work together. 3. Understand users and roles. Understanding users and their roles can have a significant impact on how the remote access strategy evolves. In most control systems operations, the roles that may require remote access to control assets may include, but are not limited to:

operators and engineers for • Slocal ystemsystems; and engineers for • Sremote ystem operators systems; • Vendors; • System integrators; specialists and • Smaintenance ystem supportengineers; • Field technicians; • Business/supply chain partners; • Rand eporting or regulatory entities; • M anaged service providers. The roles of users that would require remote access to mission-critical operations can be extensive and the assign-


| 21

AUTOMATION PROJECT SURVIVAL GUIDE HOW TO AVOID MISTAKES WITH CONTROL SYSTEM REMOTE ACCESS

ment of specific access depending on those roles can be complicated. Map out and document all acceptable access policies and procedures related to allowable network access and coordinate this with industrial control system security experts. Any user access that goes beyond simple viewing of data and permits changes to system parameters should be extremely limited.

can be hijacked • Cafter ommunication it has been initiated (does not

4. Know your vulnerabilities. Beginning at the remote user and following the connection to the data or service, remote access can be compromised at any of the following points:

communication soft• Tware he target listening for requests can be

system can be imper• Tsonated he usertoorfool the target system. can use captured or • Tguessed he attacker credentials to impersonate the user. The attacker can intimidate or coerce the user to provide valid credentials, or to perform activities at the attacker’s demand.

•

access device (laptop, • TPDA, he user’s etc.) can be attacked, com-


rely on impersonation) or intercepted during initiation (impersonating both user and target, also known as a man-in-the-middle attack).

can be • Preplayed arts of atocommunication a target, even if the at-

tacker cannot decipher the content (also known as a replay attack).

promised, and used to access the control system network.

system can be imperson• Tated he target by an attacker to fool the user and thus gain credentials or other information from the user system.

can be listened to • Cbyommunication third parties anywhere along the communication chain.

• can have data • Cinjected ommunications into them by an attacker. The communication can be interrupted or jammed.

attacked and potentially compromised.

a valid • Acommunications n attacker can impersonate node and gain access to the underlying communications medium.

attack can hap• Apen denial-of-service to the authentication server (e.g., radius server or RAS).

attack can hap• Apen denial-of-service to the outward communication device (e.g., an outside router for remote access). 

| 22


Four Tips for Dealing with Wireless Latency and Bandwidth Issues More and more, systems engineers are taking advantage of industrial wireless technologies to reduce the amount of cabling in their designs. There are some issues to be aware of, however, when replacing dedicated connections with wireless links: 1. Need latency tolerance. Today’s wired Ethernet connections are full duplex. This means that each end device can both transmit and receive at the same time. On the other hand, wireless technologies such as 802.11a/b/g/n are half duplex. This means that when any one device is transmitting, all other devices must wait. Make sure that your application is designed to be tolerant of the latency introduced due to the half duplex nature of wireless.

device. Wireless access points transmit multicast traffic at a minimal rate to ensure that all listening clients will be able to receive the traffic. This results in low aggregate bandwidth over the wireless AP as it has to lower its transmit rate down from the maximum.

2. Control multicast traffic. When implementing wireless technology in factory automation projects, be aware of any multicast traffic coming from PLCs or producer devices. Multicast traffic is handled differently than unicast traffic by wireless access points. Multiple devices can receive multicast traffic, while unicast is destined for only one

3. Low bandwidth requirements. Make sure that your application’s bandwidth requirements are low enough to be satisfied by the lower rates. Many designers overlook these points and experience problems when moving to wireless solutions. Being aware of the limitations of wireless technology can ensure that your


upfront design will work in a wireless deployment. 4. Don’t take shortcuts with wireless. Consider the entire system design and the support lifecycle of the system before choosing technology and vendors. Time spent up front on site surveys, path loss calculations and fade margin will pay dividends when it comes time for installation. Design in fade margin. Wireless is very reliable when well designed, but if you don’t design in appropriate fade margin you’ll have problems in the future. 

| 23


How to Properly Select and Vet a System Integrator The process of finding a qualified system integrator for your automation project requires effort and attention to the details. Experience, expertise, staff capabilities and financial wherewithal are all crucial factors to consider in finding the right integrator partner. 1. Selection criteria. Search for a system integrator who has a long list of successful projects in the areas you are looking for. Check out any references they provide and find out how long they have been in the field. They should also have a broad range of products they have worked with and have enough staff to handle all the various areas of a project. People who have done a lot of motion control may not have the expertise to handle a complex SCADA project. 2. Be suspicious of over-promises. If during negotiations and setting requirements, a system integrator continues saying, “No problem. That’s easy. We can do all you want”... you can be sure that It will be a problem, it will not be so easy and It will be something that is more complicated than assumed. The integrator should prove that he understood your requirements, didn’t underestimate the project and that he has experience with similar projects. Be especially


DO YOUR HOMEWORK. Extensive planning is complete, timelines and schedules are determined, budgets and ROI calculated and all the textbook preparations and considerations have been met. What could go wrong? Plenty! Always vet your system integrator. Get references, see a system designed and implemented by them in use, visit their factory and, most important, run credit checks and investigate their financial health. Nothing is more destructive than having an integrator run out of money before the project has been completed. careful if you get a much lower price than expected or than others have quoted.

the field. The integrator also needs to have a staff with expertise and domain knowledge in your business area.

3. Familiarity with standards. Find out what partners the integrator works with since no one can do it alone. It’s also important to see how an integrator manages a project and what their code library looks like. Do they follow S88 and S95 methodologies? They don’t need to follow these to the letter, but if they don’t have a methodology and aren’t even aware of the standards, don’t even consider them.

5. Expertise. Focus on their knowledge, techniques and skills. Make sure they have full knowledge of system engineering, as well as sufficient experience to handle your project. A

4. Comfort factor. In addition to reliability and professional capabilities, choose an integrator you feel comfortable with, who understands your process needs and who has experience in

proven track record and references from the projects they have done are essential. 6. Current experience. Prior experience in your discipline is key to the selection of your Integrator. Experience keeps the integrator current on new technologies and new hardware and software. As a result of the recent recession, integrators are not as abundant as before, with many unable to survive the economic turmoil.

| 24

AUTOMATION PROJECT SURVIVAL GUIDE HOW TO PROPERLY SELECT AND VET A SYSTEM INTEGRATOR

Many integrators have reduced staff, minimized technology education opportunities and made other cutbacks. Take the time to assess the strengths and weaknesses of any integrator you consider to ensure that they are capable of delivering the system that you require. 7. Stay involved. Has your system integrator done something similar before? Chances are the pool of talent isn’t all that big. Can you allocate any resources to working with that

integrator on a day-to-day basis? You will have to take ownership of the system, so you will need to know how to modify it and maintain it or you will be tied into a system that might need unallocated cash to make changes. Get involved at the zero level in the planning, simulation, detailed layout, software handling techniques and maintenance requirements as much as you possibly can in order to get the biggest possible benefits and to learn in excruciating detail how it all goes together.

DETAIL THE REQUIREMENTS 1. One of the most important factors in selecting a system integrator is his willingness to develop a good project proposal. Avoid any integrator whose proposal is just one or two pages long. 2. Automation projects must have good system requirements from the customer, and the system integrator must list in his proposal what requirements will be met and what will not. 3. If the requirements and proposal terms are properly defined from the beginning, the result will be a project with no or minimum change orders. 4. Some system integrators take advantage of a poorly written requirements document from a customer and present a very generic proposal, so the price might look attractive at the beginning. When the project is awarded, then the customer has to face a series of change orders because a requirement that might be obvious was not listed in the proposal. The customer ends up paying far more money for the project than originally estimated. 5. Establishing a good project requirement list is not only an essential customer task, but also requires the cooperation of the system integrator.


8. Take a long-term view. Select an integrator with experience in similar systems, preferably of the same make. Tie payments to project milestones. Make sure his services will be available for upgrades and maintenance by signing a separate contract. 9. Problem-solvers. Choose an integrator who has experience in the tasks you need performed. They have probably already solved many of the problems you may face if you choose one whose experience is outside the necessary area of expertise. 10. Ask questions. Choosing a system integrator is the hardest and easily the most overlooked part of an automation project. Ask questions about types of projects they’ve done, vertical preferences and size of projects. Have them include project details, such as were they on time and on or under budget, and what percentage of the time. 11. Experience has its limits. Be aware that most integrators have experience either in a vertical industry or with a certain type of project, such as PLC/HMI programming. Either way, they may lack the capabilities needed to do projects outside of that experience. Many HMI/DCS vendors have a list of endorsed or recommended system integrators on their home page. This is a good place to start. 12. Smart isn’t enough. Choose an integrator as you would choose an employee. Spend time, talk to refer-

| 25

AUTOMATION PROJECT SURVIVAL GUIDE HOW TO PROPERLY SELECT AND VET A SYSTEM INTEGRATOR

ences and know that while every firm out there enlists very smart engineers, you don’t want them cutting their teeth on your project.

integrator’s commercial qualifications: Are they CSIA certified? Do they have insurance? How many years have they been in business?

13. Professionalism counts. Make sure an integrator can confidently provide you with a project plan, with decision points, contingency plans and staffing that will meet your timeline and project goals.

16. Are they open? Select an integrator that is open to your requests and ideas. Beware of someone that constantly pushes back. If you hear

14. Test the team. Verify the integrator’s capabilities by giving a test to the personnel who will perform the work on your project. Make sure those people are listed in the contract, including fallback or substitute candidates. 15. Do they have business skills? Look beyond technology expertise or project experience to consider an


the phrase “nobody does it like that” or “this is how everyone does it,” you might want to consider another integrator that is more open minded. You are paying that integrator to get what you want and need—not just what they are willing to build because it’s easy or they “always do it that way.” Yes, you hired them for their experience and would like their suggestions, but don’t discount your own ideas just because this is your first time. Also allow for the ability to make some changes—especially if your approach is new and unconventional. Be open for changes and tweaks as you go if it makes the end result easier to use and more flexible. You need to stay involved throughout the whole process. Don’t pass up the learning opportunity! 

| 26

LIKED THIS SURVIVAL GUIDE? Then download our playbooks for free.

These are comprehensive PDF e-books jam-packed with tips, pitfalls to avoid, and best practices for implementing automation in the areas of factory and machine automation, continuous process, and batch process.

FACTORY & MACHINE AUTOMATION PLAYBOOK

awgo.to/factory

DOWNLOAD THE PLAYBOOK!

BATCH PROCESS PLAYBOOK

awgo.to/batch

DOWNLOAD THE PLAYBOOK!

CONTINUOUS PROCESS PLAYBOOK

awgo.to/continuous DOWNLOAD THE PLAYBOOK!

Automation Project Survival Guide

Recommend Documents