Errors Found in the Verizon Data Breach Report 2018Errors Found in the Verizon Data Breach Report 2018

Errors Found in the Verizon Data Breach Report 2018

by Calyptix, April 16, 2018

Verizon’s annual Data Breach Investigations Report is among the most widely referenced cyber security reports in the industry.

But how reliable is the data?

As the authors of the 2018 report note, “One of the things readers value most about this report is the level of rigor and integrity we employ when collecting, analyzing, and presenting data.”

But do all the numbers add up?

In the new 2018 Verizon Data Breach Investigations Report, published last week, one of our writers found a few conflicts in the data. We share them below.

5 Data Breaches – or 302?  

A visually striking graphic is found on page 26 of the 2018 DBIR (figure 28). This data-rich table shows readers the number of security incidents and data breaches across eight industries.

But something seems wrong with one of the cells. Circled in red below, it’s the count of POS breaches in Accommodation.

“Accommodation” refers to the Accommodation and Food Service industries, which are grouped together in the report. They have been targeted by cybercrime for years, partly due to their abundance of payment card data.

So it’s not surprising when the Verizon report notes “the Point of Sale pattern accounts for 90% of all breaches within this vertical,” on page 27.

However, the table shows only five breaches in Accommodation for the POS pattern.

Surely, that can’t be right?

If the total number of data breaches in Accommodation is 338 (page 27), then five breaches is only 1.5%. That’s a long way from 90%.

So what’s wrong?

Breach Data in the Wrong Cell

The table has the answer. In Accommodation, the 302 breaches listed as Privilege Misuse are likely in the wrong spot. They should be in the cell below – for Point of Sale.

The math works out this way. Dividing 302 POS breaches by 338 total breaches yields 0.89349 – or nearly 90% – which fits the report’s analysis.

Great – but what about the five breaches listed in the table as Point of Sale? Where do they belong?

We have no idea.

Errors on Healthcare Errors?

Healthcare is the only industry in the 2018 DBIR where internal actors caused more data breaches than external ones. We highlighted this in our earlier post about the report.

Two pages of the report focus on Healthcare. A table on page 33 shows data highlights (see below).

The table lists the top three “patterns” – which are assumedly the nine incident classification patterns Verizon identified in its 2014 data breach report.

Every Verizon Data Breach Report since 2014 has used the nine patterns in some capacity. The 2018 DBIR discusses and defines them beginning on page 22 and uses them throughout.

So – what’s wrong with the Healthcare table above? The numbers don’t add up for the given patterns.

The table lists 750 security incidents total. Three incident patterns are said to account for 63%: Miscellaneous Errors, Crimeware, and Privilege Misuse.

Going back to the data-rich table from earlier (figure 28 on page 26) we can see the number of incidents for each pattern. Below, we use data from the table to check if the three patterns add up to 63%.

Security Incidents in Healthcare – Calculation 1

Pattern
Incidents
All
Incidents
Percentage
Misc. Errors181/     750     =0.241333
Crimeware 154/     750     =0.205333
Privilege Misuse24/     750     =0.032
TOTAL 47.87%

 

Nope, only 48%. That’s a long way from 63% – so what gives?

Perhaps It Was Espionage!

One way to arrive at 63% is to change one of the patterns.

Instead of Privilege Misuse, let’s use the 138 incidents attributed to Cyber-Espionage and retry the calculation.

Security Incidents in Healthcare – Calculation 2

Pattern
Incidents
All
Incidents
Percentage
Misc. Errors181/     750     =0.241333
Crimeware 154/     750     =0.205333
Cyber-Espionage138/     750     =0.184
TOTAL 63.07%

 

Success!

But wait – did someone enter the wrong value in the big blue table on page 26? Or did someone type the wrong pattern name in the little healthcare table on page 33?

Who knows?

Privilege Misuse is Real in Healthcare

The first sentence of the 2018 DBIR section on healthcare states, “The Healthcare vertical is rife with Error and Misuse” (page 33).

Here’s the good news: even if the data and labeling in the Verizon Data Breach Report 2018 is a little off – this statement has been confirmed in many, many, many anecdotal reports.

Privilege abuse in the healthcare industry is indeed a major problem. So the report’s analysis in the text is sound, even if some of the numbers don’t add up.

Lighten Up Calyptix!

Let’s be clear – we love the Verizon data breach report. We feature it every year. It’s loaded with tons of valuable information. Everyone should read it.

Download it here right now

We like to run calculations on the data in security reports. It confirms we understand the information presented – so we can confidently present it to you, dear reader.

When the data doesn’t add up, it’s almost always because we missed something. We failed to understand the sample size, or we didn’t read the text carefully enough.

But a few times reading the Verizon DBIR 2018, we couldn’t make the numbers work no matter how hard we crunched them. We also found a few other, more trivial examples where the numbers seemed slightly off – but those were easy to dismiss.

Here’s the important point: the analysis provided in Verizon’s 2018 DBIR is solid. Again, it’s a valuable report and everyone should read it. But whatever you take away from it, make sure you confirm your insights elsewhere before taking action.

Note: we contacted Verizon for feedback and did not receive a response.

Did We Miss Something?

When you make an assumption, it’s like burying a landmine in your front yard. It’s bound to ruin your day sooner or later.

We made plenty of assumptions in this post and it’s possible that some will blow up in our face. And that’s alright!

If we missed something, please let us know in the comments.

 

Email Phishing for IT Providers

Related Resources

Top Causes of Data Breaches by Industry 2018: Verizon DBIR

Email Phishing for IT Providers

Top 4 Insights from Verizon Data Breach Report 2017

Top 5 Causes of Data Breaches in Healthcare

Biggest Cyber Attacks 2017: How They Happened

No Comments


    Leave a Reply

    Your email address will not be published Required fields are marked *

    You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

    *