At this year’s Strata + Hadoop event in London, Privitar’s CTO, Jason McFall, gave a presentation on the significant privacy challenges posed by the use and sharing of big data sets.

The need to ensure privacy, McFall pointed out, creates a problem for innovation with big data, particularly when it comes to sharing data. Often organisations can gain significant insights by sharing data sets and collaborating with third parties. They can utilise analytical techniques to reveal solutions to seemingly intractable problems.

Once a data set is published, however, it is very difficult to recall. Time and again we’ve seen examples of sensitive, supposedly anonymised, data sets being published and then re-identified with techniques which were not anticipated.