Criminal cases, public and police safety put at risk by IT failures, RCMP documents say

Internal reports detailing critical IT failures at the RCMP suggest gaffes and poor service from Shared Services Canada continue to jeopardize police work and the safety of officers and the public.

"We cannot continue to operate like this," Lois Karr, manager of the operational communications centre for the RCMP in B.C., wrote in February 2016.

Her email to Shared Services Canada (SSC) was included in critical issue report describing the unusually high failure rate of replacement telephone sets for 911 dispatchers. The unit had submitted a high-priority request for new phones in 2014. After months of waiting, they apparently received faulty, used equipment. 

• Tech support puts RCMP, public safety at risk: documents
• Chief statistician quits over failure to protect StatsCan
• Border agency warns IT failures pose security risk

"It is believed the equipment is refurbished," the report says. "The first complaint to SSC about this issue was made on 2016-01-07 and RCMP was assured by SSC NWR that the bad sets would be replaced;  however they are being replaced with more refurbished sets."

According to Karr, operators lost active calls, couldn't hear callers, had to repeatedly unplug and reset their phones to reset stuck buttons and struggled with phones that would log out entirely in the middle of their shifts. 

The RCMP demanded SSC provide brand new telephone sets and requested the department end the practice of providing refurbished equipment.

CBC News asked Shared Services Canada for comment about the equipment and several other IT headaches at the RCMP but no one had responded to the request before deadline. 

RCMP had no email during Biden visit

The Mounties are also dealing with repeated email outages. A major failure on Dec. 9 and 10 caused 800 Mounties in Ottawa to go without email access during the official visit of U.S. Vice-President Joe Biden. Not only that, no backups had been conducted since Nov. 20, which meant the Mounties were at risk of losing a lot of important correspondence and information.

According to a source familiar with the failure, a last-gasp fix by an outside technician, who put the failed hard drives in a freezer for an hour, allowed the equipment to come back online just long enough to salvage data. 

That outage followed several other episodes affecting RCMP national headquarters.

"SSC is exposing the RCMP to an unacceptable risk to its operations by not having effective support and monitoring procedures in place for its critical business applications and services," reads a report on an outage in October.

Data backup is another ongoing concern.

A briefing note from Sept. 14, 2016, outlines the serious concerns of the top brass at Alberta RCMP related to the consequences of moving 10.5 terabytes of data from two old Windows Server 2003 computers to another "new" server, which was purchased in 2011 and was "almost at end of life."

"SSC reports that the current state of backups for ALL of our investigations on the (electronic major case management server) is on the verge of complete failure. SSC has stated that it will 'not increase backup capabilities,'" the briefing note says.

"Serious Crimes Branch and Federal Policing Serious and Organized Crimes are constantly adding new investigations and investigators are also adding more data to the investigations. We are now unsure how much data we can add before backups are no longer possible. We must ensure the investigational data is secure and recoverable."

RCMP refuses to pay SSC

The Mounties informed SSC last February that it would not pay another cent for Windows Server 2003 customer support, because it did not believe shared services would actually provide the RCMP with an adequate level of service.

A similarly frustrated officer-in-charge of IT for Mounties in B.C. wrote: "The loss of data on a detachment file server will risk multiple investigations with a significant risk to public and member safety, along with significant embarrassment to the (government of Canada) and RCMP."

The documents also highlight how a blanket approach to procuring services across government doesn't suit the Mounties. Before SSC took over IT services, the RCMP used to negotiate individual data plans with companies such as Bell, Telus, Rogers and Videotron. 

"RCMP in B.C. has been provided a number of invoices from SSC for excessive data usage. One invoice for SMS [mobile text messaging] was $4,400, while the RCMP non-SSC cards would have been $15 for unlimited SMS," reads the national security portfolio issue management intake form.

The documents also describe:

• An April, 2016 complaint from the RCMP's surveillance unit in Halifax that was still waiting, after 10 months, for an increase in Internet bandwidth for ongoing investigations.
• How a failure to renew software licenses for forensic analysis could affect prosecutions, as the courts frown upon the use of unlicensed products for forensic examinations.
• That the remote detachment at Takla Landing, B.C., had by April 2016 waited more than two years for standalone Internet.
• When Mounties at two rural Alberta detachments, with no notice, had to type in codes before dialling any long distance numbers.
• An episode in January 2016, when SSC failed to tell dispatchers that "critical radio infrastructure at the Mainland site was offline" for an entire weekend.

Frustration goes back years

Poor service from Shared Services Canada has long been an irritant to the RCMP. At one point in 2015, Commissioner Bob Paulson informed SSC he would not permit the agency to have any more control over the Mounties' information.

When CBC first reported on the strained relationship last year, shared services said it had assembled a special team in November 2015 to address the force's top IT infrastructure priorities.

The RCMP is by no means alone in its high-level frustration.

Last summer, Statistics Canada chief statistician Wayne Smith resigned over what he said was unacceptably shoddy service from the IT agency. The Border Services Agency and Department of National Defence have also complained about poor service.

When asked if the RCMP could be exempted from SSC, Public Safety Minister Ralph Goodale said Tuesday he is "determined to help (the RCMP) get the kind of communications system they need to protect Canadians and safeguard Canadians' rights and freedoms."

To date, only the Supreme Court and other federal courts have been given permission to opt out of shared services.