GPS issue affects mobile operators

Data Edge Ltd, manage most of Ireland’s synchronisation systems, including the Telecom operators and Large Enterprises. Paul Phelan (CTO) talks us through the decades worst GPS outage which took place earlier this year, and something that most of us remained blissfully un-aware of.

Problem:

• GPS Satellite SVN23, launched 1990, was detected to have an anomaly at 15:36 UTC on 25th January and decommissioned at 22:00 UTC later that same day.
• Several hours later, other GPS satellites were observed to have timing anomalies.
• For reasons still being investigated the UTC signal on some satellites was off by 13 microseconds.
• Most Synchronisation systems automatically declared GPS unusable and went into ‘Holdover’ relying solely on their internal clock.

Who could have been affected?

• Timing systems that use GPS signalling to lock their atomic clock systems are critical to all modern electronic systems. These include mobile and fixed line telecoms systems, banks, stock markets, air traffic control, road and railway signals, manufacturing and power.

What happened to these systems?

• Early morning January 26^th all of Data Edge’s telecoms/large enterprise clients were informed that a serious GPS Service effecting situation was occurring. Data Edge was being advised by Microsemi re. mitigating/corrective action that could be taken. Microsemi are the manufacturer of 90% of Irelands clocking/synchronisation systems.
• Most business critical systems under contract with Data Edge have backup atomic clocks that go into holdover when GPS timing is un-available. These backup clocks (depending on the quality/type of the clock) can preserve required clock quality for between 2 days and 72 days (Quartz Crystal – low quality, Rubidium, Cesium – highest quality).
• As the situation developed (over the following 48 hours) system log files were analysed by Data Edge & Microsemi technicians and GPS receiver settings were adjusted to cope with the signal degradation. Full GPS service was restored to all clients within 48 hours.

What did we learn?

• Monitor your telecom timing systems.
• Have high quality back up clocks.
• Have a good operations/NOC (network operations centre) process in place.
• Being well informed allowed the backroom technicians to take corrective action before service was effected.

Conclusion

• A GPS problem is rather like high blood pressure – while it is normal, you take it for granted but once it goes wrong, it causes all sorts of unexpected problems. If You Don’t Measure It, You Can’t Manage It.
• Data Edge’s Expert knowledge and quick dissemination of information to our clients, combined with Microsemi’s system inherent resiliency ensured that the problem did not escalate.