Tensorflow Privacy: Responsible AI

Transcript

1. Tensorflow Privacy: Responsible AI What do we need to know about responsible AI?
2. Google Developer Group Yangon
3. Tensorflow Machine Learning Tensorflow Extended JAX
4. NLP Researcher with an interest in sentiment analysis, social media analysis, and information extraction. Ph.D. Candidate in Computer Science (PSU, Thailand) AI/NLP consultant Aye Hninn Khine
5. Introduced in 2018 https://ai.google/responsibilities/responsible-ai-practices/ Responsible AI Practices: Google AI
7. Responsible AI The potential benefits of AI are huge, so are the dangers. ~Dave Waters AI is likely to be either the best or worst thing to happen to humanity. ~Stephen Hawking
8. The way actual users experience your system is essential to assessing the true impact of its predictions, recommendations, and decisions Use a human-centered design approach 01 The use of several metrics rather than a single one will help you to understand tradeoffs between different kinds of errors and experiences Identify multiple metrics 02 Responsible AI practices
9. ML models will reflect the data they are trained on, so analyze your raw data carefully to ensure you understand it. In cases where this is not possible, e.g., with sensitive raw data, understand your input data as much as possible while respecting privacy; for example by computing aggregate, anonymized summaries. Directly examine raw data 03 Machine learning models today are largely a reflection of the patterns of their training data. It is therefore important to communicate the scope and coverage of the training, hence clarifying the capability and limitations of the models Understand the limitations of dataset and model 04 Responsible AI practices
10. Learn from software engineering best test practices and quality engineering to make sure the AI system is working as intended and can be trusted. Test, Test, Test 05 Continued monitoring will ensure your model takes real-world performance and user feedback Continue to monitor and update the system 06 Responsible AI practices
11. Identifying Bias in AI Learn: https://www.kaggle.com/learn/intro-to-ai-ethics
12. Six Types of Bias in AI Systems Paper: A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle (Harini Suresh and John Guttag, MIT)
13. Adobe Stock#243026154 Historical bias occurs when the state of the world in which the data was generated is flawed. Historical Bias Representation bias occurs when building datasets for training a model, if those datasets poorly represent the people that the model will serve. Representation Bias
14. Adobe Stock#243026154 Measurement bias occurs when the accuracy of the data varies across groups. This can happen when working with proxy variables (variables that take the place of a variable that cannot be directly measured), if the quality of the proxy varies in different groups. Measurement Bias Aggregation bias occurs when groups are inappropriately combined, resulting in a model that does not perform well for any group or only performs well for the majority group. (This is often not an issue, but most commonly arises in medical applications.) Aggregation Bias
15. Adobe Stock#243026154 Evaluation bias occurs when evaluating a model, if the benchmark data (used to compare the model to other models that perform similar tasks) does not represent the population that the model will serve. Evaluation Bias Deployment bias occurs when the problem the model is intended to solve is different from the way it is actually used. If the end users don’t use the model in the way it is intended, there is no guarantee that the model will perform well. Deployment Bias
17. Learn: https://ethics-of-ai.mooc.fi/ AI Fairness
18. Adobe Stock#243026154 Demographic parity says the model is fair if the composition of people who are selected by the model matches the group membership percentages of the applicants. Demographic Parity Equal opportunity fairness ensures that the proportion of people who should be selected by the model ("positives") that are correctly selected by the model is the same for each group. We refer to this proportion as the true positive rate (TPR) or sensitivity of the model. Equal Opportunity
19. Adobe Stock#243026154 The percentage of correct classifications (people who should be denied and are denied, and people who should be approved who are approved) should be the same for each group. If the model is 98% accurate for individuals in one group, it should be 98% accurate for other groups Equal Accuracy Group unaware fairness removes all group membership information from the dataset. For instance, we can remove gender data to try to make the model fair to different gender groups. Similarly, we can remove information about race or age Fairness Through Unawareness
20. “Berkeley researchers found that both face-to-face and online lenders rejected a total of 1.3 million creditworthy black and Latino applicants between 2008 and 2015.” When the researchers “used the income and credit scores of the rejected applications but deleted the race identifiers, the mortgage application was accepted”. https://www.cbsnews.com/news/mortgage-discrimination-black-and-latino -paying-millions-more-in-interest-study-shows/
21. “In practice, it is not possible to optimize a model for more than one type of fairness.” THE IMPOSSIBILITY THEOREM OF MACHINE FAIRNESS: A CAUSAL PERSPECTIVE (Kailash Karthik Saravanakumar, January 2021)
22. Differential Privacy
23. ★ Differential privacy is a framework for measuring the privacy guarantees provided by an algorithm. ★ Through the lens of differential privacy, we can design machine learning algorithms that responsibly train models on private data. ★ Differential privacy is achieved by adding random noise to the result, which may be done via a variety of differentially-private processes such as the Laplace, exponential, and randomized response approaches Differential Privacy
25. Netflix anonymised the training set for its recommender challenge by replacing all names with random ids. Turns out that by linking this dataset with public IMDB review rating WHY do we need DP? https://www.wired.com/2007/12/why-anonymous-data-sometimes-isnt/
26. Adobe Stock#243026154 • there was little turnover and nothing to stimulate the market • south korea and japan continue to be profitable • merchant banks were stronger across the board Both Private and Non-Private Model Score Highly • aer banknote berlitz calloway … ssangyong swapo wachter • the naczelnik stands too • my god and i know i am correct and innocent Only Non-Private Model Score Highly
27. P(M(D)=“Bob has cancer”)=0.55 P(M(D+Bob)=“Bob has cancer”)=0.57 P(M(D+Bob)=“Bob has cancer”)=0.80 Privacy Loss : Log (P(M(D)=“Bob has cancer”) / P(M(D+Bob))=“Bob has cancer”)) Log (0.57 / 0.55) =0.0357 Log (0.80 / 0.55) =0.375 <<<--------Leaked Bob information WHY DP? Let’s look at the cancer prediction model
28. Log (P(M(D)∈S) / P(M(D’)∈S)) ≤ϵ. We tune this ϵ to trade-off how much we learn vs the privacy of the individual. The key idea behind differential privacy is to limit this difference in performance: Privacy Budget https://mukulrathi.com/privacy-preserving-machine-learning/deep-learning-differential-privacy/
29. DP-SGD Deep Learning with Differential Privacy (Abadi et al.) https://arxiv.org/pdf/1607.00133.pdf
30. Tensorflow Privacy
31. https://www.tensorflow.org/responsible_ai/privacy/guide
32. from tensorflow_privacy.privacy.optimizers import dp_optimizer # replace this optimizer = tf.train.GradientDescentOptimizer( learning_rate=__) # with this optimizer = dp_optimizer.DPGradientDescentGaussianOptimizer( l2_norm_clip=__, noise_multiplier=__, num_microbatches=__, learning_rate=__)
33. DP-SGD Parameters
34. import tensorflow as tf from tensorflow_privacy.privacy.optimizers import dp_optimizer_keras # Select your differentially private optimizer optimizer = tensorflow_privacy.DPKerasSGDOptimizer( l2_norm_clip=l2_norm_clip, noise_multiplier=noise_multiplier, num_microbatches=num_microbatches, learning_rate=learning_rate) # Select your loss function loss = tf.keras.losses.CategoricalCrossentropy( from_logits=True, reduction=tf.losses.Reduction.NONE) # Compile your model model.compile(optimizer=optimizer, loss=loss, metrics=['accuracy']) # Fit your model model.fit(train_data, train_labels, epochs=epochs, validation_data=(test_data, test_labels), batch_size=batch_size)
35. #MNIST Dataset train, test = tf.keras.datasets.mnist.load_data() train_data, train_labels = train test_data, test_labels = test train_data = np.array(train_data, dtype=np.float32) / 255 test_data = np.array(test_data, dtype=np.float32) / 255 train_data = train_data.reshape(train_data.shape[0], 28, 28, 1) test_data = test_data.reshape(test_data.shape[0], 28, 28, 1) train_labels = np.array(train_labels, dtype=np.int32) test_labels = np.array(test_labels, dtype=np.int32) train_labels = tf.keras.utils.to_categorical(train_labels, num_classes=10) test_labels = tf.keras.utils.to_categorical(test_labels, num_classes=10)
36. epochs = 3 batch_size = 250 l2_norm_clip = 1.5 noise_multiplier = 1.3 num_microbatches = 250 learning_rate = 0.25 model = tf.keras.Sequential([ tf.keras.layers.Conv2D(16, 8,strides=2,padding='same',activation='relu',input_shape=(28, 28, 1)), tf.keras.layers.MaxPool2D(2, 1), tf.keras.layers.Conv2D(32, 4,strides=2,padding='valid',activation='relu'), tf.keras.layers.MaxPool2D(2, 1), tf.keras.layers.Flatten(), tf.keras.layers.Dense(32, activation='relu'), tf.keras.layers.Dense(10) ])
37. optimizer = tensorflow_privacy.DPKerasSGDOptimizer( l2_norm_clip=l2_norm_clip, noise_multiplier=noise_multiplier, num_microbatches=num_microbatches, learning_rate=learning_rate) loss = tf.keras.losses.CategoricalCrossentropy( from_logits=True, reduction=tf.losses.Reduction.NONE) model.compile(optimizer=optimizer, loss=loss, metrics=['accuracy']) model.fit(train_data, train_labels, epochs=epochs, validation_data=(test_data, test_labels), batch_size=batch_size)
38. Adobe Stock#243026154 Privacy Measure (RDP - Renyi Differential Privacy)
39. compute_dp_sgd_privacy.compute_dp_sgd_privacy(n=train_data.shape[0], batch_size=batch_size, noise_multiplier=noise_multiplier, epochs=epochs, delta=1e-5) DP-SGD with sampling rate = 0.417% and noise_multiplier = 1.3 iterated over 720 steps satisfies differential privacy with eps = 0.563 and delta = 1e-05. The optimal RDP order is 18.0. (0.5631726490328062, 18.0)
40. Assess privacy risks with the TensorFlow Privacy Report
41. Given a trained ML model and some data point, decide whether this point was part of the model’s training sample or not. https://franziska-boenisch.de/posts/2021/01/membership-inference/ Membership Inference Attack
42. AttackInputData( labels_train=y_train_indices[:, 0], labels_test=y_test_indices[:, 0], probs_train=prob_train, probs_test=prob_test), SlicingSpec(entire_dataset=True, by_class=True), attack_types=(AttackType.THRESHOLD_ATTACK, AttackType.LOGISTIC_REGRESSION), privacy_report_metadata=privacy_report_metadata)
43. Adobe Stock#243026154 It consists of 60000 32x32 colour images in 10 classes, with 6000 images per class (50000 training images and 10000 test images). The classes are airplane, automobile, bird, cat, deer, dog, frog, horse, ship, and truck. CIFAR 10 Dataset
44. attack_input = AttackInputData( logits_train = logits_train, logits_test = logits_test, loss_train = loss_train, loss_test = loss_test, labels_train = train_labels, labels_test = test_labels ) slicing_spec = SlicingSpec( entire_dataset = True, by_class = True, by_percentiles = False, by_classification_correctness = True) attack_types = [ AttackType.THRESHOLD_ATTACK, AttackType.LOGISTIC_REGRESSION ] attacks_result = mia.run_attacks(attack_input=attack_input, slicing_spec=slicing_spec, attack_types=attack_types)
45. Best-performing attacks over all slices LOGISTIC_REGRESSION (with 2889 training and 2889 test examples) achieved an AUC of 0.69 on slice CORRECTLY_CLASSIFIED=False LOGISTIC_REGRESSION (with 2889 training and 2889 test examples) achieved an advantage of 0.31 on slice CORRECTLY_CLASSIFIED=False Best-performing attacks over slice: "Entire dataset" LOGISTIC_REGRESSION (with 10000 training and 10000 test examples) achieved an AUC of 0.59 LOGISTIC_REGRESSION (with 10000 training and 10000 test examples) achieved an advantage of 0.16 Attack Result
46. Further Reading [1] Shokri, Reza, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. “Membership inference attacks against machine learning models.” In 2017 IEEE Symposium on Security and Privacy (SP), pp. 3-18. IEEE, 2017. [2] Song, Liwei, and Prateek Mittal. “Systematic evaluation of privacy risks of machine learning models.” arXiv preprint arXiv:2003.10595 (2020). [3] Yeom, Samuel, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. “Privacy risk in machine learning: Analyzing the connection to overfitting.” In 2018 IEEE 31st Computer Security Foundations Symposium (CSF), pp. 268-282. IEEE, 2018. [4] Truex, Stacey, Ling Liu, Mehmet Emre Gursoy, Wenqi Wei, and Lei Yu. “Effects of differential privacy and data skewness on membership inference vulnerability.” In 2019 First IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA), pp. 82-91. IEEE, 2019. [5] Salem, Ahmed, Yang Zhang, Mathias Humbert, Pascal Berrang, Mario Fritz, and Michael Backes. “Ml-leaks: Model and data independent membership inference attacks and defenses on machine learning models.” arXiv preprint arXiv:1806.01246 (2018).
47. Thank you! Ph.D. Candidate, Department of Computational Science, Prince of Songkla University She/Her @AyeHninn ayehninnkhine93@gmail.com Aye Hninn Khine