Collecting and Analyzing Multidimensional Data with Local Differential Privacy (1907.00782v1)

Abstract: Local differential privacy (LDP) is a recently proposed privacy standard for collecting and analyzing data, which has been used, e.g., in the Chrome browser, iOS and macOS. In LDP, each user perturbs her information locally, and only sends the randomized version to an aggregator who performs analyses, which protects both the users and the aggregator against private information leaks. Although LDP has attracted much research attention in recent years, the majority of existing work focuses on applying LDP to complex data and/or analysis tasks. In this paper, we point out that the fundamental problem of collecting multidimensional data under LDP has not been addressed sufficiently, and there remains much room for improvement even for basic tasks such as computing the mean value over a single numeric attribute under LDP. Motivated by this, we first propose novel LDP mechanisms for collecting a numeric attribute, whose accuracy is at least no worse (and usually better) than existing solutions in terms of worst-case noise variance. Then, we extend these mechanisms to multidimensional data that can contain both numeric and categorical attributes, where our mechanisms always outperform existing solutions regarding worst-case noise variance. As a case study, we apply our solutions to build an LDP-compliant stochastic gradient descent algorithm (SGD), which powers many important machine learning tasks. Experiments using real datasets confirm the effectiveness of our methods, and their advantages over existing solutions.

• The paper introduces two innovative mechanisms, PM and HM, that reduce noise variance in numeric data under local differential privacy.
• It extends these solutions to efficiently handle mixed multidimensional data, combining methods for numeric and categorical attributes.
• Experimental results show significant accuracy improvements over traditional methods in statistical estimation and machine learning tasks.

An Overview of Collecting and Analyzing Multidimensional Data with Local Differential Privacy

This paper addresses a significant gap in the literature concerning the application of Local Differential Privacy (LDP) to the collection and analysis of multidimensional data, which may comprise both numeric and categorical attributes. While existing research has primarily focused on either complex data types or tasks involving single-dimensional data, this paper recognizes that even simple tasks under LDP, such as computing mean values of numeric attributes, require novel solutions to address inadequacies in the current methodologies.

Novel Contributions

The authors introduce two new mechanisms for numeric data: the Piecewise Mechanism (PM) and the Hybrid Mechanism (HM), which are designed to enhance accuracy by minimizing worst-case noise variance. PM strategically confines perturbed data within a bounded domain to balance noise addition and accuracy, outperforming the widely used Laplace mechanism and Duchi et al.'s solution for various privacy budget settings $\epsilon$. HM improves further by dynamically selecting between PM and Duchi et al.'s solution, effectively reducing noise variance to levels that Duchi et al.'s could not achieve individually, especially as $\epsilon$ increases.

For multidimensional data involving both numeric and categorical attributes, the authors extend PM and HM, presenting methods that combine these with strategies for handling categorical data via noise injection. Their innovative approach achieves asymptotic optimal error bounds, comparable with Duchi et al.'s approach, which is theoretically advantageous for numeric dimensions but complex and limited to purely numeric data. Notably, their solution is the first to efficiently handle multidimensional data with mixed attribute types under LDP.

Experimental Evaluation

The experimental results are compelling. Using real and synthetic datasets, the authors demonstrate that their methods consistently outperform existing solutions, including Duchi et al.'s and the Laplace mechanism in terms of accuracy. The evaluation encompasses various settings and emphasizes the robustness of the proposed methods across different privacy budgets and data distributions.

Specifically, the paper highlights that PM and HM achieve lower mean square errors (MSE) in estimating mean values of numeric attributes, as well as improved frequency estimation for categorical data when compared against state-of-the-art methods. Additionally, their application to machine learning models such as linear regression, logistic regression, and support vector machines displays lower misclassification rates or mean squared errors relative to competitors.

Implications and Future Work

The implications of this research are significant for both theoretical developments in privacy-preserving data analysis and practical applications in machine learning under privacy constraints. Theoretically, the paper enriches the existing set of LDP-compliant mechanisms by addressing multidimensional data collection and analysis with optimal error bounds. Practically, its application to stochastic gradient descent (SGD) shows concrete benefits in settings like federated learning, where individual privacy is paramount.

Future research paths might include extending these methods to more sophisticated data analysis tasks, such as neural networks, and exploring the trade-offs in communication complexity when deploying such mechanisms in large-scale systems. Given the increasing importance of privacy-preserving techniques in data-driven industries, further investigation into more tailored solutions for different types of data distributions and domain-specific applications would be beneficial.

In conclusion, this paper contributes substantial advancements to the field of privacy-preserving data analysis through local differential privacy, particularly in the context of multidimensional data, while prompting further exploration of its applications in diverse real-world scenarios.