Atticus: My name is Atticus!
Henrik: and I'm Henrik, and this is our final project presentation video for Data Privacy.
A: For this project we decided to utilize an existing dataset and implement some differential privacy techniques to get useful information while maintaining proper privacy. It's worth highlighting that one of the biggest challenges we faced was coming up with useful views of the data that we could also apply privacy stuff to.
while showing dataset and jupyter notebook
H: The dataset itself was sourced from the UCI Machine Learning repository; more specifically #352, Online Retail. Each row represents an order with a description, quantity, unit price, country, and customer ID; these being the columns we care about. The first step to setting up the project was fetching the dataset using UCI's library, and we also chose to limit the dataset down to 50,000 rows down from the original 500,000 rows so that our code doesn's take forever to run.
showing the bounded_data and pick_b functions
A: Since a single customer could be present in multiple rows of the data, the first thing we decided to do was utilize bound contribution so that we can guarantee user level privacy. I also wrote a pick b function to select the bound automatically.
showing the avg_order_value function
H: Once that was set up, the first thing we implemented was a simple differentially private mean that calculates the average value of a given order. It first makes a series that represents the total value of each order and then uses laplace noise and sequential composition to make the result differentially private.
A: This function guarantees epsilon differential privacy and bounded sensitivity. We also added a bit of code that checks the accuracy of this function, although we won't run it here because it can take a few minutes to finish.
H: For further analysis, we decided to use some more advanced techniques from later in the class to get some more interesting insights; the columns of choice were Description which is basically the name of the item and the Country column.
A: The decision to use a 2-way marginal as one option was because we wanted to preserve the relationship between the Description and Country columns, so that someone could figure out what items are most popular in certain countries for example. Sadly, this was pretty inaccurate. Because of the low signal to noise ratio, outside of the United Kingdom, most countries get swapped around in terms of counts and things are very, very noisy.
H: We also decided to use the Sparse Vector Technique; in this case, to determine the most popular items. One pitfall we did notice with this and the 2 way marginal mentioned earlier are that the sensitivity tends to be pretty high as it relates to the bound. This can make some pretty odd inaccuracies.
A: One big lesson we learned is that while a bunch of values being basically the same can impact how much proportional noise differential privacy adds, so can a few really large outliers and lots of sparse data. In this case, so many of our values were close to 1 that it became very difficult to implement differential privacy that did not absolutly tank accuracy. And the sparse data makes sense, given the company the data was collected from specializes in "unique all-occasion gifts", i.e. items which are likely to be unique. It was interesting to learn how even in this relativly simple retail setting how hard it can be to get a privacy-usage balance.