Modernizing I&C for operations and maintenance, one phase at a time

In 2018, Dominion Energy initiated an extensive program of upgrades at Surry and sister site North Anna—which also has a pair of Westinghouse three-loop pressurized water reactors—to equip the plants for extended operation. The upgrades include a first-of-a-kind engineering program to replace analog control and protection systems with digital instrumentation and controls.

Atkinson

When young interns first see the 1960s-era controls in Surry’s main control room simulator, senior generation program manager for digital I&C systems Robert Atkinson said that a typical reaction is, “What is all this stuff?” Their surprise emphasizes the incongruity of 1960s technology in a modern workplace. That’s what Dominion’s digital upgrade is all about—modernizing valuable assets by executing a handoff between analog and digital controls separated by decades of advancements.

Nuclear News staff writer Susan Gallier talked to Atkinson to learn more about the multiphase project that will see the dials and push buttons in the Surry and North Anna control rooms replaced with large-screen monitors and glass-top controls by 2034.

Can you give me a big-picture overview of the digital modernization project?

Back in 2018 we began looking at what would be involved in a review, replacement, and upgrade of all the major control equipment in the plants, including the reactor protection system (RPS) and engineered safety features (ESFs), as well as a major changeout of the main control room to digital. We wanted to install equipment that would give us common platforms at both Surry and North Anna and make sure that the operating staff—the end users of the systems—have the equipment and insights they need to manage the plant under any conditions.

What is the price tag for this work, and how is it funded?

The cost is public knowledge, and right now it’s $5.45 billion for both stations for all the SLR projects, including digital modernization. The return on investment is to keep the plants operating for 80 years so we don’t have to build costly new generation here in Virginia.

In 2017, the Virginia State General Assembly wanted to make sure that we had a stable, low-cost electric distribution system that was carbon neutral or carbon free. Dominion’s nuclear plants are some of the cleanest, most reliable, and lowest-cost production facilities in the state. The General Assembly decided it was in the state’s best interest to allow Dominion to upgrade and operate the power stations to 80 years. The program is currently fully funded.

A row of communication cabinets (black, foreground) and control cabinets being tested in Surry’s CSTF. (Photo: Dominion)

How would you describe the scope of the modernization project?

We use the term “modernization” for the digital side of things. We are also upgrading other non-I&C systems to support SLR. We’re replacing steam generators, main generators, heat exchangers, feedwater heaters, and so forth—not just the control and protection systems. Digital modernization is the largest and most complex component of this project at both Surry and North Anna.

You could say that replacing a steam generator means taking one chunk of metal out and putting another chunk of metal back in. Replacing analog systems with digital is completely different. We’re performing 19 major system design changes per unit for four units. That’s about 80 major design changes for the entire process. In addition, we are performing supporting infrastructure changes to include a Cyber Secure Test Facility (CSTF), a new simulator building with a second simulator, and an upgraded instrument ground system.

How have you interacted with others in the industry as you plan this project?

We benchmark other nuclear utilities, and we interface with the Nuclear Energy Institute and the Electric Power Research Institute. We aren’t just benchmarking against nuclear. For example, Dominion has some new-build combined-cycle gas plants. The control room for one of these plants is fully digital and one operator sits in front of a series of monitors controlling 1,500 MW of power. It’s a different world!

We approached a chemical manufacturing company that operates huge petrochemical plants and said, “You used to have analog plants too. What did you do to change to digital?” The response that we got was interesting. They said, “Oh, we did that so long ago. I don’t think there’s anybody around that knows anything about it.”

It’s not a secret: The nuclear industry is behind the times. But that means that almost everything that we’re doing has been done someplace else—at nuclear plants or in other industries.

While what we’re doing is no different from what a lot of nuclear plants have already done on a system basis, nobody has done it with the holistic scope that we have for this project. That’s where Dominion is on the leading edge of the digital changeout curve.

How is your team working with Dominion Energy stakeholders?

Both station vice presidents have formally signed on as champions of the project, which is very important. We’ve had a senior shift manager from each station assigned to the project, along with one or two reactor operators. It’s important to have a strong Operations presence in the design change process for digital modernization. Having those individuals has been invaluable because what we’re trying to do is give the Operations staff a better view of the plant. Ops is our number one champion. The Maintenance and Training organizations are also stakeholders. We have procedure writers who are fully engaged, and we also support corporate engineering with revising or creating new calculations. It would be a challenge to find someone at the plant who is not a stakeholder in this project.

Surry’s CSTF currently has about 40 control cabinets undergoing testing. (Photo: Dominion)

Surry is first up for digital modernization. Why does it have the lead on North Anna?

Surry is first because those units are about six years older. We want to get new equipment into the plant so that we can start receiving the benefits of reduced maintenance and operational enhancements. Surry is an older Westinghouse three-loop plant that has a 7100 Control and a relay RPS/ESF protection system that is fraught with well-documented obsolescence issues. North Anna is a 7300/SSPS plant. Parts are becoming harder to get and are prone to early failure. For example, at Surry we had one safety-related relay fail within 30 days of installation.

We will see a major benefit during outages once we replace RPS/ESF systems. Currently, when Surry comes off line, we spend days setting up and performing tests of the relay protection system. The digital replacements will allow this up-front testing to be automated and will cut down on front-end outage testing.

A project of this scale demands strong oversight and informed external partners. How are you communicating requirements?

Replacing a plant component like a steam generator requires a design specification or procurement specification. You send it out to vendors, they bid on it, and you make a product selection.

For this project we did something a little different. We were looking at changing 19 systems per unit at each station. We focused on the need-to-know functions that are being replaced and that vendors would need to provide in a quality response to a request for proposals.

Exactly how many analog controls, switches, and indicators would we need to replace? We did a survey and put all that information for all the systems into one document—it ended up being 3,100 pages long—and sent it out with an RFP.

We selected the successful bidder and will work with our equipment supplier and engineer of choice (EOC) to complete writing the vendor-informed equipment design specifications needed to support design development and implementation.

Who are you working with, and what is their role?

Surry nuclear power station. (Photo: Dominion)

Sargent & Lundy is the primary EOC. Our equipment vendor is Westinghouse, and we’re using Emerson’s Ovation and Westinghouse’s Common Q system for the protection platform. We have regular biweekly and quarterly face-to-face meetings as well as many ad hoc meetings.

There have been rough spots and there have been great spots, as we would expect with such a huge project. We have a number of project managers, and we have weekly phone calls between the project managers at Surry and North Anna. We’ve sent project managers to North Anna outages, and they’ve sent people down to Surry. We’re constantly working as a team.

Dominion’s survey of its analog systems identified over 7,000 individual functions or controls. Is anything not targeted for modernization?

We went through every switch, control, indicator light, meter, recorder, control switch, and push button that the operators use to operate the plant under all circumstances, and we cataloged them. We included a lot of metadata, such as the drawings associated with each device and the other equipment it is connected to. That documentation will eventually help us prove to the Nuclear Regulatory Commission, function by function, that the operators still can control every aspect of the plant. There are some systems not included in the upgrade program, for instance turbine controls, primarily due to those controls having been recently upgraded.

In addition to system functional replacements, we’re also adding some new capabilities. An example of an enhancement is automated steam generator water level control. Right now, taking our steam generators from zero power to 100 percent power is all analog, and the operators have to tweak the generator water levels as we’re coming up in power. It requires a unique talent set. If you’re off a little bit, your unit will trip, and you don’t want that. We’re also adding an enhancement for automatic cooldown using steam dumps. Operators will be able to set the cooldown rate and monitor it as it comes down automatically.

Will these design changes all require license amendment requests (LARs)?

We’ll do 10 CFR 50.59 evaluations to see if we need to do LARs. Changing our current analog reactor protection systems will absolutely require a LAR. If an upgrade means changing our licensing requirements and our technical specifications, we’ll write LARs as appropriate. We are also monitoring what other utilities are doing as well as keeping track of the latest NRC activities. Modernizing the control and protection systems may have positive impact on future operational changes such as risk-informed technical specifications, for example, which would be beneficial to the stations.

Can you tell me how cybersecurity is integrated into the modernization project? I noticed it’s part of your Phase 1 activities.

What we call our CPNI—controls platform network infrastructure—is where our cybersecurity equipment that continually monitors our networks and equipment is attached. The cyber monitoring devices will detect hostile hacking or intrusions. In addition to the system monitoring, every new piece of equipment that goes into the plant must go through a critical digital asset (CDA) assessment.

Housing and testing CDA were identified as critical during our initial development of the project. We realized that we would have many cabinets coming from Westinghouse, and we would need a place to perform our site acceptance test and store the equipment in a secure area. We decided to build a Cyber Secure Test Facility (CSTF)—an 11,000-square-foot facility that is rigidly monitored just like any other cybersecurity area. The CSTF can securely hold about 80 cabinets at max capacity. We’re very proud of that building. Right now, we’ve probably got the Surry CSTF about two-thirds full, and we’re getting ready to finalize site acceptance testing on some of the equipment and move that equipment out and into the plant next year.

How is human factors engineering incorporated into the control room redesign?

Human factors engineering is necessary because we’re changing how operators interface with the equipment in the plant. NUREG-0711 [the NRC’s Human Factors Engineering Program Review Model] is guiding our work, but I want to emphasize that it’s not a requirement, it’s a guideline.

The idea is that operators need to be comfortable with what they’re seeing. They need to be able to quickly get the information to respond to any operating condition, including any accident or transient.

We purchased a full-size glass-top simulator to support human factors engineering and training development. The glass-top simulator has a set of video cameras to capture how operators interact with the controls during validation testing. You can think of it as a testing environment for human factors. We want to validate that operators can perform a procedure, that they know what to do, and they’re comfortable with everything from the colors of the background and the indicators to how close together the controls are.

What training strategies are you putting in place to prepare operators for the transition to the digital systems?

It’s an iterative approach. The changeover will happen in phases, and during successive outage seasons more and more controls will be taken off the bench board and vertical boards in the control room and transitioned to glass-top operator terminals.

One of the things we learned early in the project is that we are going to spend a lot of time changing the actual control room, which means we have to spend a lot of time changing the plant’s ANSI 3.5 certified simulator to reflect what we’ve done in the control room. This interrupts ongoing LORP [low-level operator radiation protection] and new operator training. In addition, throughout the project, operators still need to train on the new controls, so we needed to provide a simulator for their use.

We installed a full-scale glass-top simulator into our training building consisting of the bench boards, vertical boards, and some of the backboards. This allows operators to come out and do just-in-time or catch-up training while we’re either updating the main simulator or LORP training is occurring. The glass-top simulator also allows them to test procedures that will be introduced later and that we need their feedback on.

Training an operator on the older system’s 7,000 devices takes about 18 months. The training group is working with Operations to develop new training products, and they are confident they can reduce the total training time.

Understanding Surry’s main control room will shift from analog to digital controls in phases, has that process begun?

Yes. At Surry, we have one system operating out of our Ovation-based plant computer system. It’s a very small system that controls some pumps and motors located about two miles from the plant at our low-level intake structure. That system—an Ovation installation—has been operating since 2004 and we’re now upgrading that 20-year-old equipment. Eventually our plant computers will devolve and become part of our digital control system (DCS). Additionally, the operators have a newly upgraded Ovation glass-top turbine control system.

Inside Surry’s CSTF, the half-scale glass-top simulator visible at the left is used for testing and by operations staff to verify or develop new procedures. At center right are new digital annunciator windows, and a row of cabinets being tested is visible in the background. (Photo: Dominion)

In what we call Phase 2B, during our outages in 2031 and 2032, we’re replacing the RPS, ESFs, and modifying the main control room at the same time. There’ll be some large-screen displays and some terminals, and the benchboard will have a sit-down area for the operators to work at.

Ultimately, the operators won’t have any hard switches, with one important exception. We will have a set of hard controls for a system-level and diverse actuation system. These controls will allow the operators to bypass the digital system to initiate a reactor trip, safety injection, engineered safeguards, and other controls needed to mitigate an accident. It’s basically an AMSAC [ATWS Mitigating System Actuation Circuitry] replacement. Those hand switches will be there for the operators as a fallback measure as part of ensuring that the plant is always safe.

How are you measuring the project’s success?

The easy answer to that is we succeed when we get the equipment installed on budget and on schedule, turned over, and the operators are using it to safely and reliably operate Surry and North Anna power stations.

Our plan is to do as much as we possibly can with the unit on line. We’re pulling miles of fiber optic cable, we’re installing patch panels in different locations, all set up for success so that when we tie it into the CPNI during an outage, our impact on the outage is minimized.

What are some challenges or lessons learned that you would share with utilities going through similar projects?

I would say it’s very difficult to get qualified people. You just can’t have anybody walk in and design nuclear digital control systems. Companies looking for qualified people—such as EOCs like Sargent & Lundy—are bringing in young engineers fresh out of college and training them in nuclear processes. At Dominion, we are getting our interns involved from the start. Our interns are our future engineers and future subject-matter experts. It’s important to get them involved early and give them hands-on experience with testing and installation.

Conceptual planning is very important. Not just for a system but looking at the entire program. It is very important that you understand the impact on HVAC, power, infrastructure, and other nondigital projects that may be occurring at the same time. Coordination with Operations and with Outages and Planning is critical to minimizing outage and implementation impacts. These impacts should be identified as early in the process as reasonably possible.

For a project this complex you must be flexible. There are going to be times when something comes up and causes a problem. You have to be able to say, “Okay, we have a risk mitigation strategy, and here’s how we want to handle it.” Always keep the end goal in mind when addressing any emergent issues.

Finally, one of the most important things is good communication. You must communicate with your stakeholders. They’ll help you if you let them know what you’re doing. Don’t assume that everybody understands what a particular widget does. We’ve organized lunch-and-learns with our station management where we’ll provide lunch and have the responsible engineer and project manager go through a system, explaining what they’re doing so that the management is comfortable with what’s going on.

Once the modernization is complete, what will be the biggest benefits to the station?

I have two answers to that question. One is that the maintenance and engineering staff will be helped tremendously by the diagnostics built into the new equipment. Ultimately, digital modernization will reduce the amount of affected I&C equipment installed in the plant by about 60 to 70 percent.

But the biggest benefit is for the operating staff. I will keep saying it, but the purpose of this project is really to improve our already safe plant operations by giving the operators better, improved visibility on what’s happening in a unit at any time.

Could this modernization program see Surry and North Anna through another license renewal and potentially to 100 years of operation?

The equipment that we’re putting in (not just digital controls) has the potential to go beyond 80 years. No decision has been made to extend the life of Surry and North Anna beyond 80 years, but I hope they do!