Real-Time Redundant Communication for a Reliable Grid

The electricity grid is a vital resource for today’s world. The reliability of the electrical network is paramount, particularly with the increasing use of distributed generation and the introduction of microgrids that require more local intelligence in terms of monitoring and control. All systems on the grid need to cooperate to guarantee that the network is able to deliver stable and consistent AC through the frequent analysis of voltage, frequency and switchgear status. This requirement points to the use of advanced monitoring and readily available, low-latency communications between control centers and high-value nodes such as substations to ensure that the status of each subsystem is closely monitored.

To help manage grid infrastructure and provide the necessary level of control and communication, the electrical distribution industry has adopted the IEC 61580 standard. Using IEC 61580, digital signals transmit information on power status between substations and electrical subsystems over a network based on Ethernet technology. To cater for the increasingly complex needs of grid operators, the IEC 61580 standard has evolved to embrace a number of time-critical services, such as IEEE 1588 synchronization, the Generic Object Oriented System Event (GOOSE) and Sampled Value (SV) messaging.

Figure 1: Key elements of the IEC 61580 and their relationship to Ethernet.

The GOOSE protocol supports publish-subscribe messaging, which is better suited to distributed control applications than the traditional client-server communications model used in many IT systems. IEC 61580 makes client-server messaging available separately to support device management by remote stations and other situations where a node within a substation or microgrid needs to be accessed directly.

GOOSE allows real-time messages on the status of various electrical elements to be relayed to any devices that need to see them. Each relay, for example, publishes information using a single identifier for each different type of object it contains for which it needs to report status. Other relays and devices can subscribe to the stream of status message sent out by the target relay. This is performed using a combination of multicasting across the network and filtering on receiving nodes – the publisher does not need to know who subscribers are or if they receive the messages. For maximum performance, GOOSE does not use the commonly employed TCP/IP protocol. Instead GOOSE messages are put inside Ethernet frames directly. Similarly, SV messaging provides a way of multicasting data for values that change rapidly, such as voltage measurements, to subscribing receivers.

The services provided under IEC 61580 include time synchronization according to the IEEE 1588 standard. The standard distributes timing signals derived from a high-precision grandmaster clock. On many systems in use today, this grandmaster clock uses the time signals transmitted by Global Positioning System (GPS) satellites, each of which contains an atomic clock. The grandmaster clock feeds a tree of slave clocks that are either classed as boundary clocks – these are situated in gateways that can act as temporary masters if the GPS signal fails – or ordinary clocks that sit inside each of the systems that need to be synchronized. Timing information ripples through the tree using a series of message exchanges that need to be repeated at regular intervals to ensure all the systems stay synchronized.

The protocol works by having a master clock send out a sync message to slaves that contains a timestamp. Shortly afterwards it sends a follow-up that contains a later timestamp. The slave responds with a message asking the master for a delay estimate, which it provides with a further response containing the time it was sent. At the end of this process, the slave will have four timestamps, three of which were sent by the master. Using the timing information in the sequence, the slave determines the alteration needed to its internal clock to match the master as closely as possible.  The protocol results in timing synchronization that, on a local area network, is typically accurate to tens of nanoseconds.

Other key parts of the IEC 61580 infrastructure are redundant data-transmission protocols. Standardized in IEC 62439, the Parallel Redundancy Protocol (PRP) supports dual-port full-duplex Ethernet communication. The protocol provides two streams of redundant packet handling underneath the MAC layer interface seen by upper-layer applications protocols such as GOOSE and SV. Typically, the Ethernet connections are through different Ethernet switches and assume a star-wiring network topology.

Figure 2: The PRP redundancy protocol assumes a star-wired topology.

An alternative is High-availability Seamless Redundancy (HSR), also defined in IEC 62439. HSR employs a ring topology, with each packet being duplicated and forwarded in opposite directions around the ring towards the destination. The ring architecture avoids the need to employ additional switch/routers at the cost of increased delay if packets need to pass through multiple nodes to reach their destination. Advanced communications controllers can help minimize this delay through the use of cut-through forwarding. This approach does not require that a packet be fully decoded before forwarding to the destination begins.

Figure 3: The HSR redundancy protocol is intended for a ring topology.

Because of the need for extensive filtering of incoming Ethernet messages, the IEC 61580 requires high-performance processing that can reduce the amount of compute power available to control algorithms. One answer is to offload as much of the network-level analysis as possible so that the host processor only needs to deal with messages that require its attention. This can be achieved on multicore SoCs, some of which contain specialized intelligent network processors. An example is the AM572x Sitara embedded microprocessor made by Texas Instruments. An evaluation board is available that provides easy exploration of its networking functionality.

The AM572x is based on the ARM® Cortex®-A15 processor. The multicore device augments the host processor with a Cortex-M4 that can be used to offload I/O-intensive tasks. Also included is a pair of networking processors together with a digital signal processor based on the C66x architecture for performing data analysis. The PRU-ICSS subsystem on the AM572x provides separate processing in addition to that available on the ARM core. The unit contains two PRUs, each of which contains a 32-bit RISC processor able to run at up to 200 MHz and a networking interface. The availability of two independent intelligent cores provides ready support for PRP and HSR.

The RISC processor in the PRU core does not have a general-purpose architecture. Instead this core is specifically designed for manipulation of the types of packed memory mapped data structures encountered in network frames. It incorporates a number of features to support applications that have tight real-time constraints. Some degree of packet filtering can be performed on the PRU processors. On the AM572x, more headroom for protocols such as IEEE 1588, GOOSE and SV is available from the Cortex-M4.

The Cortex-M4 can be used to analyze all incoming multicast packets and compare their Application ID (APPID) addresses for valid subscriptions that are provided by software running on the Cortex-A15. From this the M4 can determine which messages need to be passed upstream. Other packets can be dropped and cleared from memory.

Figure 4: Shared-memory IPC supports the offloading of IEC 61580 processing to the Cortex-M4 and other processors.

A key consideration in this offloaded-processing architecture is how the individual processors communicate with each other. The AM572x offers shared memory to facilitate the passing of messages from one processor to another. Packets can readily be formed into queues so that they can be written and read in sequence. The key question is the protocol to use. One option is to employ Linux on the Cortex-A15. This allows the use of standard application programming interfaces (APIs) that the operating system offers for inter-process communications, such as remoteproc and rpmsg.

The rpmsg messaging systems works by providing a virtual device that reflects each communication channel linked to a remote process. Channels are identified by a textual name and have a local rpmsg address and remote rpmsg address. When a driver starts listening on a channel, the callback function used for reception is bound to a unique 32-bit rpmsg local address. When inbound messages arrive, the rpmsg core dispatches them to the appropriate driver according to their destination address. The messages are relayed by calling the driver's reception handler while providing the payload of the inbound message. Through this scheme, the filtering code for GOOSE and SV messages can pass messages with specific APPID addresses to different handlers running on the Cortex-A15. Alternatively, they may all be grouped for relaying to a common message processor and sorted on the host processor.

The open-source IPC3.x library available from TI’s Git repository implements the software to support the use of rpmsg between the Linux environment and the TI RTOS that is available for the Cortex-M4 and DSP cores. Providing the same APIs across TI devices, the IPC product abstracts device-specific support for hardware spinlocks, inter-processor mailbox and rpmsg-compatible message passing through the MessageQ API.

Conclusion

Through multiprocessing with message-passing support, devices such as the AM572x can efficiently support complex real-time control protocols such as those devised for the mission-critical job of managing grid infrastructure.