The Limitations of Hashing for Data Anonymization

Imagine you are a medical researcher, and you realize that you can do better research if you have more data. For example, if you could track a patient for 2 years and see if they followed their prescription regime, that could help inform your analysis of the impact of the new drug.

Unfortunately for you (and fortunately for the patients), regulations like HIPAA explicitly prevent sharing medical information that identifies the patient. You are not allowed to share details such as their names. “Ah, no problem,” you think, “I will simply hash everyone’s name. A hash looks like gibberish, but hashing the same name always produces the same gibberish. That way, I can correlate their medical data and records so that we can get the benefits of large amounts of data without jeopardizing anyone’s privacy.”

That is a fantastic idea. The problem is that privacy is not about protecting the name it is about protecting the person. The name is simply one way they can be identified. The hash might look like gibberish but when combined (or “linked”) with other data, even medical data, it becomes information that can be used to identify someone.

Let’s walk through an example. Here is a table with a name, age, and medical history.

John	Juan	Ivan	Giovanni	Johann
40 years old Shingles Genetic history of heart disease	41 years old Shingles Genetic history of heart disease	38 years old Shingles Genetic history of heart disease	18 years old Lyme Disease Genetic history of diabetes	25 years old Hemophilia Genetic history of flat feet

In this next table, the names have been hashed with an md2 algorithm (an antique algorithm only used here for example purposes) to produce some “gibberish”.

4ce0cc8b637951b5a26bcb498d9b739d	40195c8ed340ad275b56e79f24501eba	2c280c0fa212d5b5341ed3cce82568c8	76983828e0d313a820478d9c5769dee5	67158725ee6b4b71dbf0b68f442f0164
40 years old Shingles Genetic history of heart disease	41 years old Shingles Genetic history of heart disease	38 years old Shingles Genetic history of heart disease	18 years old Lyme Disease Genetic history of diabetes	25 years old Hemophilia Genetic history of flat feet

This looks more anonymous, right? Nobody’s name is in there so you might believe that privacy has been preserved. The problem is if someone wants to figure out who these people are (called “deanonymization”), they just need to find enough information from open sources about the people to match them up to their hash. In the age of social media, more of our information is available than ever. You would be surprised how easy it is to find information about a “40 year old” with “shingles” and a “genetic history of heart disease.” With just a little more sleuthing, it is generally possible to identify that “4ce0cc8b637951b5a26bcb498d9b739d” is “John” with almost 100% certainty.

This problem has been known for a long time. An FTC blog in 2012 titled “Does Hashing make Data ‘Anonymous’” specifically called out hashing as an incomplete anonymization process, describing the many security failures that a hash-it-and-leave-it strategy would bring about. Unfortunately, we still see companies continue to use this approach for “anonymization.”

It isn’t that hashing isn’t used; it is simply that it is not enough alone. Current best practices for anonymization require leaving out key pieces of information altogether and keeping necessary records in vaguer forms. For example, one approach that might work is to generalize the data: adjust the table to give an age range instead of an age as shown in the next table.

4ce0cc8b637951b5a26bcb498d9b739d	40195c8ed340ad275b56e79f24501eba	2c280c0fa212d5b5341ed3cce82568c8	76983828e0d313a820478d9c5769dee5	67158725ee6b4b71dbf0b68f442f0164
35-45 years old Shingles Genetic history of heart disease	35-45 years old Shingles Genetic history of heart disease	35-45 years old Shingles Genetic history of heart disease	15-25 years old Lyme Disease Genetic history of diabetes	15-25 years old Hemophilia Genetic history of flat feet

While this is a very simplistic example it illustrates that there are multiple people in the table with the same data. So even if you were able to narrow it down to these elements it is difficult to know whose data is whose and even to tie it to open-source data for deanonymization.

Obviously, however, the exact anonymization required depends on the data, the data sources, and the uses of the data. In almost all circumstances, you need a specialist in privacy to help you get data anonymization correct. If you are working with sensitive data and need help reaching compliance with HIPAA or other regulations relating to privacy, please reach out to Crimson Vista. We offer auditing services to determine the current state of your security posture and where you can make improvements. We would be happy to help you!