The Most Expensive Single Server Upgrade, Ever (So Far)

One server, that’ll be $41,000,000.00.

Excluding top secret government systems, or experimental research, most commercial entities expect server upgrades to result in a net savings, or at least to maintain a baseline of performance.  And servers can be quiet pricey, ranging from $5,000 to $20,000+ if you want to own the actual hardware.  Cloud based servers are $0 (to try) up to several thousand dollars, again, depending on what you require.

Imagine a server that cost $100,000.  Or $500,000.  Or $1,000,000.  What would you expect from such a computing powerhouse?  What if all you received was a basic, run-of-the-mill server, with basic competent on-site installation, would you feel like it was a bad deal?  Why would anyone pay so much?

Nobody, at least not intentionally.  But that’s exactly what a Northern California hospital did.  Not intentionally, but by a series of events a single server cost St. Joseph’s Health over $41 million!  That’s a lot of new MRI machines, doctor’s salaries, and clinical improvements, combined.

Cause	Cost
Controls and Systems Upgrade	$17 million
Class Action Settlement	$15 million
Credit Monitoring	$4.5 million
Victims Fund (ID theft damages)	$3 million
HHS OCR Civil Monetary Penalty	$2.14 million
Total	$41.64 million

The only difference between St. Joseph’s and a number of other hospitals and healthcare networks is that we know about what happened at St. Joseph’s.  Others are just waiting to be discovered.  And at the risk of sounding like an armchair quarterback, the problem in this case wasn’t an IT problem, it wasn’t even an IT security problem, it was a management problem.

Information security is not IT security.  Information security includes IT security, and risk management, and regulatory compliance, and management policies (referred to as governance).  In this instance, the server was installed, and no doubt it was fully functional and configured according to SOP.  What was missing was a process to manage the installation of new technology, especially where ePHI was stored, transmitted, or processed.

When we trust our doctor to treat our medical information confidentially, we are trusting the supporting cast of professionals to do likewise.  It is incumbent on healthcare leadership to incorporate information security into the overall culture and strategic vision of their organization, to take responsibility for the due care that our most intimate medical information deserves.

Contact RMO Consulting today to learn how we can work together to determine your organization’s information security needs.

How High Will We Go?

Image credit: healthitoutcomes.com

Last year Ponemon reported information security incidents cost the healthcare industry more than $6.2 billion. I expect this year that number will continue to increase.

One reason is the number two cost vector cited in the report was ransomware. Ransomware incidents are not on the decline. TrendMicro reported in August that just the first half of 2016 had seen a 172% percent increase in ransomware incidents over all of 2015 combined.

Another reason is the importance of information availability in the clinical environment. Where other industries may sardonically quip, “it’s not a matter of life and death,” in healthcare, it quite literally is. As an analyst at ESET put it, “Criminals know this and are deliberately targeting medical organizations.”

Collectively, those of us in the information security field hold a special trust and confidence with those who share personal health information. We need to do a better job of controlling risks to that information, and advocating to those who may not completely understand risk but approve budgets, the need to allocate resources.

A Dumpster Full of Treasure

Hackers didn’t do it.  Cyber activists didn’t do it.  There was no identity theft ring to hold responsible.  What happened was the wrong boxes were picked up and thrown in a dumpster.  So, 94,000 letters had to be mailed to patients to inform them that their names, medical record numbers, payment methods, lab results, and in some cases, Social Security numbers or driver’s license numbers were unwittingly disclosed.  But at least there was no breach of the database, no penetration of the firewall, so that’s good news, right?

Wrong.  Technology did not fail in this case, people did.  And there are two reasons why this is bad news.

First, penalties.  There are a lot of requirements in HIPAA, but they boil down to the Security Rule and the Privacy Rule.  Now, the Security Rule deals exclusively with electronic health records, so it doesn’t apply here.  The other half of HIPAA, the Privacy Rule, which can be summarized as, “Don’t share personal health information unless there’s a need to know,” definitely applies.  Here there was failure.  As of this writing, there has been no announcement of any penalties coming from the Office of Civil Rights (the Health and Human Services enforcement division), but we know that penalties can be as high as $1.5 million total for a given year.  That’s the cap, which if the OCR decides penalties are in order, could be a real savings for the hospital, because the minimum penalty is $100 per record per incident, or around $9,400,000!  No doubt that will hurt financially, and that is bad not only for the hospital, but the community they serve.

Second, the public trust is damaged.  It’s hard to quantify what the public trust is worth.  There are some studies on consumer trust in other markets, but when it comes to the trust between patient and doctor, that may be worlds apart.  Lab results, even ordering certain types of lab work, offers a raw glimpse into the most confidential of health information: HIV status, drug use, pregnancy, genetic conditions, etc.  Patients may think, “better go to another hospital in town if I want it to remain between me and my doctor.”  This really hurts the entire medical field as the sterling trustworthiness of doctors and the staff of the hospitals is tarnished.

In addition to what this hospital is doing, what lessons can the rest of us learn from this incident?  As I said at the beginning, this was human error, not a technical failing   Here are some best practices to consider, given what we know, to establish a sound information security management system:

• Assign an owner, someone who is personally responsible for the records – nothing tends to focus someone’s attention and make them truly advocate for the protection of patient information as knowing their job is on the line.
• Label all sensitive information – whether a cover sheet, a folder, or brightly colored labels on banker’s boxes, anything confidential should be recognizable from the outside.
• Establish a retention policy – the longer information is retained, on paper or electronically, the more likely something may happen with it; a retention policy should be attuned to the least time possible before records are destroyed.
• Physically secure records locations – data centers or records closets, only those who are prescreened and approved as having a business requirement should have access.
• Shred all paper records on-site – the need to remove paper to shred it at a distant location has long passed, so the option to make sure a staff member witnesses the shredding and collects the receipt is nearly universal.
• When projects are planned, make sure information security is a part of the plan – as the axiom goes, failing to plan is planning to fail. All projects should include information security risks assessment in the planning and execution phases, along with consideration of cost, safety, etc.  This is the new norm in project management.

Anyone who handles personal health information should take this opportunity to assess their own controls and make improvements where needed.

The sky is falling! The sky is falling!

The sky is not falling – yet.  But if a recent development in Massachusetts is a harbinger of things to come, anyone involved in risk, compliance, or information security may need to prepare for some major overhauls to our risk assessments.

According to an article in National Law Review, a recent case in Massachusetts Superior Court is concerning.  The plaintiff brought a lawsuit against Boston Medical Center following the accidental exposure of patient data through a vendor web portal.  The author specifically notes that although, “Plaintiffs do not allege that any unauthorized persons actually viewed, accessed or misused their private information.”  the case was allowed to move forward based on the “real and immediate risk” of damages that may result from the disclosure of information.

So far, this is one case in Massachusetts, and in most jurisdictions courts will only allow a lawsuit if there is evidence of actual damages (identity theft, for example).  But if other courts begin to look at this case as precedent, the costs to remediate any breach or accidental disclosure could become significantly more than any institution can afford.  In addition, the insurance industry will likely respond with more stringent requirements for cyber liability insurance and premiums will likewise go up.  All of this leads back to the reliance on sound risk management beginning with the initial assessment.

It will also require hospital vendors (those business associates referred to in the HITECH Act of 2009) to redouble their efforts to identify and remediate risks before they too are front page news.

What is Information Security Risk Management?

Hospitals are some of the finest examples of practical risk management.  Every day choices are weighed and actions are taken in clinical care and administrative support to achieve the best possible outcomes.  Part of the reason for the amazing impact modern hospitals have on our daily lives is due to the application of sound decision making processes that are well documented, measured, and evaluated.  This process is what constitutes management, and it does not detract from the talents of so many dedicated individuals that make up the staff of the hospital, but rather enhances the combined effectiveness of the entire team.

One of the cornerstones of good management is risk management.  Most hospitals have a risk management committee.  But for those who do not participate on the committee, it may seem far removed from the daily goings-on of the hospital, so I would like to explain what risk management is, within the narrow application of information security.

If you read my previous blog on information security, you already have an understanding of the importance of this field on hospitals, insurers, and managed care organizations. Information security risks can be documented and measured based on several methodologies, but here is a quick reference to help everyone understand how those methodologies work, and the language used to describe them.

To start, information faces certain threats and vulnerabilities (to clarify, threats are from outside your organization, and vulnerabilities are inside your organization).  Sometimes, threats and vulnerabilities work together, and the end result of one or both achieving their purpose is called an impact.  When talking about information security, impacts are usually described in terms of confidentiality, integrity, and availability (more about these below).  Along with impacts, the probability that an impact will occur is also factored in.  To determine the risk posed by the threats and vulnerabilities, the probability and impact are used to arrive at a risk score.  If it is too high a risk, controls are put in place to reduce the risk to an acceptable level.

Why does any of this matter if you are not on a risk committee?  Because we all manage risk every day, even if you don’t call it by this name.  For example, when you are preparing to go to work, you hear the weather report and it says there is a 50% chance of rain by noon, becoming freezing rain by the evening.  So there is a threat of rain.  Combined with your vulnerability to cold wet conditions (hypothermia, etc.) leading to sickness, you decide the risk (the impact combined with the probability) are too high for you to accept.  You decide to bring a rain coat and umbrella, as well as a change of shoes, and now the risk is greatly reduced that you will be sick due to the weather.

When it comes to information security, rather than hypothermia, the concerns are disclosure of patient information (confidentiality), the usefulness and clarity of the information (integrity), and the ability to pull up patient information when and where it is needed (availability).  Without this information, every patient contact would require re-examination, there would be no records to reference, and patient privacy could not exist.

So the rules and policies imposed by the IT department, while seemingly a cumbersome pile of techno-speak, are really responses to the risks faced by medical facilities with regard to patient information and treatment, whose end goal is to reduce the risks to acceptable levels while allowing everyone to continue to do their part for successful patient care and ongoing hospital administration.

The Power We All Depend On

In most of the United States, the power is on 24x7x365.  Some areas still encounter outages due to weather, but those outages can be anticipated.  Times are changing and now national, or at least regional outages lasting days or weeks are possible.  What used to be a theoretical concern, from the Cold War or from prepper hyperbole, must now be weighed in earnest during risk assessment and disaster recovery planning.  And here is why.

Both EU institutions and the US energy sector recognize the vulnerability of the electric grids we all depend on.  And industry experts point to the rising frequency and sophistication of attacks.  What is worse, and possibly a sign of things to come, is the coordinated large scale efforts to attack power grids at precisely the time when we are all most dependent and hence most vulnerable, as happed in Israel this week.

For the typical IT engineer, CIO, or IT Director, redundancy is the name of the game.  Data backup, power backup, and even co-locations provide assurance that if the lights go out, the enterprise can maintain operations.  But there is a missing component to this strategy: the certain knowledge that this system will be used.  What I am suggesting is that there needs to be passion and urgency, not just planning and testing.

It is only a matter of time before a well-planned, coordinated, and intense action to disrupt or destroy the US power grid, internet, or other critical infrastructure is attempted; whether from state sponsored agents or hacktivists it won’t matter to the institutions that are hurt.

The question becomes, what more needs to be done?  To start, those who are responsible for the management, planning and testing of enterprise business continuity management must analyze scenarios in depth.

For example, a company headquartered in Palo Alto, CA with a co-location in Las Vegas, NV is still vulnerable if the Western Interconnection (the power grid for the entire western US) is offline for an extended period of time.  To be fair, smaller companies and government agencies may not have the resources to be thinking beyond the state they reside in, let alone the continent, but thinking in terms of large scale events that show growing likelihood of occurring cannot be left to the largest organizations with deep resources.  Indeed, our first responders, hospitals, and community leaders are those who we implicitly rely on to put the most well thought plans in place to provide safety and security with only limited allocations of resources.  Likewise, the incident response plan must make the most with what is available, and address the risks to the most critical aspects of the enterprise.

Whether as a tangential part of the controls implemented in the information security management system, a full application of the ISO 22301 management system, or another system, every company that wishes to remain a going concern the day after crisis strikes must plan, implement, test, and improve their crisis response, disaster recovery, and continuity management.

A 25-Worst List to Make Your Eyes Roll

Is your password a Boov password?  If you have seen the movie Home as many times as my children, you know the best password in the world is, “My name is Oh and Captain Smek is great and anyone who does not think that is a poomp1.”  Do you ever wonder what the worst passwords are?  The ones that are easily guessed or likely to be hacked?  Due to all of the breaches in 2015, researchers at SplashData have compiled a list of the 25 worst passwords for the past year.  Word to the wise, if you see your password on the list, change it.

Which brings us to the problem with passwords – they are out of date.  What you should be using are pass phrases, or complete sentences with numerals and punctuation, whenever possible.  There are many advantages to using longer phrases and sentences rather than passwords:

• They are easier to remember,
• They are easier to change when you are required to,
• They are more secure.

Most IT departments no longer limit your “password” to ten or twelve characters, so making a pass phrase is easy to do.  Consider these as examples – all include at least one upper case, lower case, a numeral and punctuation:

• I live on 1st Street. (21 characters, because spaces count)
• Never eat 3 soggy waffles! (26 characters)
• I am 2x as smart as you. (24 characters)

Do not use one of these as your pass phrase, but think of a phrase that would work for you.  And if you were wondering what the worst were, here is the list, with their change in rank from 2014:

1. 123456 (Unchanged)
2. password (Unchanged)
3. 12345678 (Up 1)
4. qwerty (Up 1)
5. 12345 (Down 2)
6. 123456789 (Unchanged)
7. football (Up 3)
8. 1234 (Down 1)
9. 1234567 (Up 2)
10. baseball (Down 2)
11. welcome (New)
12. 1234567890 (New)
13. abc123 (Up 1)
14. 111111 (Up 1)
15. 1qaz2wsx (New)
16. dragon (Down 7)
17. master (Up 2)
18. monkey (Down 6)
19. letmein (Down 6)
20. login (New)
21. princess (New)
22. qwertyuiop (New)
23. solo (New)
24. passw0rd (New)
25. starwars (New)

3-Steps to Review Your Vendor’s Information Security

With the New Year comes the resolve to do things for ourselves to make our lives better.  With that in mind, you should take the occasion to revisit your vendor management, particularly where information security is concerned.

The American Recovery and Reinvestment Act of 2009 required that all business associates (i.e. vendors) must comply with the privacy and security provisions of HIPAA as the covered entities they work with, and that these requirements must be included in your contracts.  While putting some language in contracts may satisfy legal minds, providing your business associates with specific guidance for compliance can eliminate confusion and demonstrate due diligence above and beyond.  Below are three key steps that can be the foundation of sound vendor management.

A good first step is to perform a risk analysis on your vendor relationship.  While there are many approaches to a risk assessment, find one that works for you and stick with it.  The value of a risk assessment comes in the consistent use of method or tool so you can make apples to apples comparisons.  If this is new to your organization, reach out to experienced professionals who can help, a little or a lot, to develop an approach that works for you.

Next, with a risk assessment in hand, look at the business associates with the most risk, and analyze if the current information security regimen is sufficient to reduce risks to an acceptable level.  Some questions you might consider:

1. What transparency do you have into the operations and information security of your vendor?
2. Are your vendors independently certified or audited?
3. How responsive are my vendors when I send questionnaires or requests for attestation?

Consider requiring vendors to obtain independent certification in the next 18-24 months.  Among the many standards available, ISO 27001, HITRUST, and NIST are widely accepted.  The advantages to using a standard certification is the uniformity in controls selection, independent audit and assessment approach, and the reduction in compliance costs.

Third, be a promoter of the value of risk-based, objective vendor management.  The value of this approach is that is helps prioritize where your attention needs to be, and provides a clear measurement of just how important some of your business associations truly are.  This information can be shared throughout your organization, and can be integrated with other risk management, information security, and business intelligence activities.

As more business practices become outsourced, the need for sound vender management practices has never been greater.  Integrating your information security risk management approach with vendor management provides supply chain transparency and produces demonstrable due diligence.  Finally, by taking the above approaches, inconsistencies and short-comings are more likely to be identified and remedied before an embarrassing reportable event occurs.

What is Information Security?

Simply put, information security is about protecting information.  In the medical field, there are numerous types of information that need protection whether you are in the clinical environment, or behind the scenes.  Regardless what your position may be, 2016 is the year that information security is going to be bigger than ever.  And there are two reasons why.

The first reason has to do with HIPAA, a law passed in 1996.  Anyone who works around healthcare has heard of HIPAA, but few understand what is really in the law.  There are two parts to it, a Privacy Rule (which covers patient privacy) and a Security Rule (which covers electronic health records), but few have paid much attention to it, because unless something comes up in the news nobody was likely to feel the sting if either rule was broken.  However, starting in 2009, that started to change.  When President Obama signed the American Recovery and Rehabilitation Act (the large economic stimulus plan) into law, it contained provisions that gave HIPAA some teeth and those teeth have bite.  Consider these examples:

• a hospital agrees to pay $850,000 for potential HIPAA violations,
• a health insurer agrees to a $3.5 million penalty for potential HIPAA violations,
• a university hospital agrees to pay $750,000 and implement missing HIPAA policies.

These are just some enforcement examples from 2015.  Look for more in this year.

There are worse things than a government penalty, and this is the second reason why information security will be important for all of us in 2016.  One year ago this month, one of the largest health insurers in California announced they were the target of a data breach, and approximately 80 million people were affected.  In July, a well-known university medical center announced they had been breached, affecting 4.5 million people.  According to one report, the first half of 2015 saw more than 89 million patient and staff records hacked nationwide, up from 12 million for the same time in 2014.  At an average cost of $154 per patient record to fix the mess caused by data breaches, this amounts to over $13.7 billion in damages caused to hospitals, insurers, HMOs, and others in just six months.

HIPAA is just one of the best known information security requirements, but there are other regulations, industry standards, and laws that apply to the areas covered by information security.  This is because the credit card industry has requirements for payment collection, the federal government has requirements for anything affecting collections and credit, and many states have their own unique requirements.  Any or all of these requirements may overlap anywhere where there is information that needs protection: frontline/patient admit, revenue recovery, billing, patient records, and human resources.  No matter where in a healthcare you work, you have a role to play.

There are reasons with technical sounding names for why hackers are able to wreck such havoc, but the bottom line is there are no signs they are going to stop trying.  So for this reason, 2016 will be a very important year for everyone to pay attention to information security.