Faced with a backlash against its latest outage last week, M1 has taken the rather unusual step of asking an outsider – an independent expert – to review its problematic mobile network.
This third party will examine how its network is connected and further enhance its performance, after several high-profile gaffes in the past year have signalled that recent network disruptions were not one-off events.
More tellingly, M1 said yesterday that the latest outage affecting users for more than five hours last week came about because its network was becoming more complex. That, ironically, was meant to cut out the problems it had faced lately.
Indeed, they point to the increasing technical challenges that face many mobile operators, especially in a densely packed place like Singapore where the user base has been growing fast as its population has expanded in the past 10 years.
The skyrocketing demand for mobile Internet services has not helped, with bandwidth-sucking applications like video on the go and constantly-engaged services such as Facebook always keeping networks busy.
In a bid to cater to higher demand, operators have installed more software and hardware in their networks to better detect what users are doing and allocate resources accordingly. When things go wrong with these additional systems, however, the situation can get bad very fast.
M1’s story last week, according to its early investigations, is one good lesson for operators everywhere.
The problem began when its call processing software failed to let customers authenticate themselves against a customer database when they tried to make or receive calls. These attempts to authenticate quickly sucked up computing and network resources and soon M1’s customer databases were clogged up with requests.
In the operator’s own words: “The congestion cascaded to another network layer, tying up additional network resources and preventing customers from making or receiving calls.”
Questions will surely be asked of M1’s network design. How has it allowed something as critical as its customer database servers to be saturated without stopping the authentication requests that swamped them?
Right now, it says it is investigating the issue with its technology vendor. It will hope that the outcome won’t be nearly as embarrassing as some of its findings last year, when an almost-three-day outage was blamed on contractors who caused a spark in an equipment room and triggered a sprinkler that damaged its network gear.
For that gaffe, M1 was given a record S$1.5 million fine by the government regulator. Goodness knows how many frustrated customers it lost as a result.
This time round, with the Minister for Communications and Information and scores of angry customers already expressing their unhappiness, M1 cannot be having an easy time explaining why its network is down again. Surely, it cannot say the incident was one-off.
“With a more sophisticated network, it has inevitably increased the level of complexity in troubleshooting,” M1 chief executive officer Karen Kooi offered yesterday. Once again, she apologised for the disruption and promised to improve things.
But Singapore’s smallest operator cannot claim that its network is more difficult to manage than its bigger rivals, who surely handle more calls, texts and Internet traffic for a higher number of customers.
While nobody can guarantee a network will never fail, what M1 will have to show now to customers is a period of sustained uptime in the coming months.
Each little error that crops up will shake confidence even more. Its years of goodwill built through its cheerful branding and well-regarded customer service won’t count for much if its basic service cannot be counted on to work.
