Detecting Adversarial Audio via Activation Quantization Error Heng Liu and Gregory Ditzler Department of Electrical & Computer Engineering University of Arizona Tucson, AZ 85721 USA {hengl, ditzler}@email.arizona.edu 

Outline • Introduction • Related works • Adversarial audio attack and detection • Neural network quantization • Contribution • Experiments • Conclusion and future works 

Introduction: Insecure DNNs Adversarial images example 

Introduction: Insecure DNNs Adversarial audio example 

Introduction: Open problem How to detect? • Image applications • Image transformation • Feature transformation • Edge detection • Signal Processing based techniques • Quite effective • Audio applications • Techniques adopted from image domain • Limited security due to fundamentally different structure 

Introduction: A separate but related topic Neural network quantization • Reduce DNN’s memory, and computational resource consumption • Deployment on fog and edge devices • Low latency application Contribution • DNN quantization is beneficial for adversarial detection • We propose to detect adversarial audios by using neural network quantization 

Related works Adversarial audio attacks • Gradient based audio attack • Carlini and Wagner, 2018, SPW • Gradient based audio attack (over-the-air) • Yukura and Sakuma, IJCAI, 2019 • Black-box audio attack (free of gradient calculation) • Taori et al., SPW, 2019 Adversarial audio detection • Feature transformation • Frequency filters • Temporal dependency-based methods 

Related works Neural network quantization • Quantization schemes • Weight quantization • Activation quantization • Quantization error • Accuracy loss comparing with full precision model ↵ AAAB7XicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoMegF48RzAOSJfROJsmY2ZllZlYIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSgQ31ve/vcLa+sbmVnG7tLO7t39QPjxqGpVqyhpUCaXbERomuGQNy61g7UQzjCPBWtH4dua3npg2XMkHO0lYGONQ8gGnaJ3U7KJIRtgrV/yqPwdZJUFOKpCj3it/dfuKpjGTlgo0phP4iQ0z1JZTwaalbmpYgnSMQ9ZxVGLMTJjNr52SM6f0yUBpV9KSufp7IsPYmEkcuc4Y7cgsezPxP6+T2sF1mHGZpJZJulg0SAWxisxeJ32uGbVi4ghSzd2thI5QI7UuoJILIVh+eZU0L6qBXw3uLyu1mzyOIpzAKZxDAFdQgzuoQwMoPMIzvMKbp7wX7937WLQWvHzmGP7A+/wBi4GPGA== AAAB7XicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoMegF48RzAOSJfROJsmY2ZllZlYIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSgQ31ve/vcLa+sbmVnG7tLO7t39QPjxqGpVqyhpUCaXbERomuGQNy61g7UQzjCPBWtH4dua3npg2XMkHO0lYGONQ8gGnaJ3U7KJIRtgrV/yqPwdZJUFOKpCj3it/dfuKpjGTlgo0phP4iQ0z1JZTwaalbmpYgnSMQ9ZxVGLMTJjNr52SM6f0yUBpV9KSufp7IsPYmEkcuc4Y7cgsezPxP6+T2sF1mHGZpJZJulg0SAWxisxeJ32uGbVi4ghSzd2thI5QI7UuoJILIVh+eZU0L6qBXw3uLyu1mzyOIpzAKZxDAFdQgzuoQwMoPMIzvMKbp7wX7937WLQWvHzmGP7A+/wBi4GPGA== AAAB7XicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoMegF48RzAOSJfROJsmY2ZllZlYIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSgQ31ve/vcLa+sbmVnG7tLO7t39QPjxqGpVqyhpUCaXbERomuGQNy61g7UQzjCPBWtH4dua3npg2XMkHO0lYGONQ8gGnaJ3U7KJIRtgrV/yqPwdZJUFOKpCj3it/dfuKpjGTlgo0phP4iQ0z1JZTwaalbmpYgnSMQ9ZxVGLMTJjNr52SM6f0yUBpV9KSufp7IsPYmEkcuc4Y7cgsezPxP6+T2sF1mHGZpJZJulg0SAWxisxeJ32uGbVi4ghSzd2thI5QI7UuoJILIVh+eZU0L6qBXw3uLyu1mzyOIpzAKZxDAFdQgzuoQwMoPMIzvMKbp7wX7937WLQWvHzmGP7A+/wBi4GPGA== AAAB7XicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoMegF48RzAOSJfROJsmY2ZllZlYIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSgQ31ve/vcLa+sbmVnG7tLO7t39QPjxqGpVqyhpUCaXbERomuGQNy61g7UQzjCPBWtH4dua3npg2XMkHO0lYGONQ8gGnaJ3U7KJIRtgrV/yqPwdZJUFOKpCj3it/dfuKpjGTlgo0phP4iQ0z1JZTwaalbmpYgnSMQ9ZxVGLMTJjNr52SM6f0yUBpV9KSufp7IsPYmEkcuc4Y7cgsezPxP6+T2sF1mHGZpJZJulg0SAWxisxeJ32uGbVi4ghSzd2thI5QI7UuoJILIVh+eZU0L6qBXw3uLyu1mzyOIpzAKZxDAFdQgzuoQwMoPMIzvMKbp7wX7937WLQWvHzmGP7A+/wBi4GPGA== cap AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUCbbTbt0swm7G6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTAXXxvO+ndLa+sbmVnm7srO7t3/gHh61dJIpypo0EYnqhKiZ4JI1DTeCdVLFMA4Fa4fj25nffmJK80Q+mknKghiHkkecorHSA8W071a9mjcHWSV+QapQoNF3v3qDhGYxk4YK1Lrre6kJclSGU8GmlV6mWYp0jEPWtVRizHSQz0+dkjOrDEiUKFvSkLn6eyLHWOtJHNrOGM1IL3sz8T+vm5noOsi5TDPDJF0sijJBTEJmf5MBV4waMbEEqeL2VkJHqJAam07FhuAvv7xKWhc136v595fV+k0RRxlO4BTOwYcrqMMdNKAJFIbwDK/w5gjnxXl3PhatJaeYOYY/cD5/AE44jcw= AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUCbbTbt0swm7G6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTAXXxvO+ndLa+sbmVnm7srO7t3/gHh61dJIpypo0EYnqhKiZ4JI1DTeCdVLFMA4Fa4fj25nffmJK80Q+mknKghiHkkecorHSA8W071a9mjcHWSV+QapQoNF3v3qDhGYxk4YK1Lrre6kJclSGU8GmlV6mWYp0jEPWtVRizHSQz0+dkjOrDEiUKFvSkLn6eyLHWOtJHNrOGM1IL3sz8T+vm5noOsi5TDPDJF0sijJBTEJmf5MBV4waMbEEqeL2VkJHqJAam07FhuAvv7xKWhc136v595fV+k0RRxlO4BTOwYcrqMMdNKAJFIbwDK/w5gjnxXl3PhatJaeYOYY/cD5/AE44jcw= AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUCbbTbt0swm7G6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTAXXxvO+ndLa+sbmVnm7srO7t3/gHh61dJIpypo0EYnqhKiZ4JI1DTeCdVLFMA4Fa4fj25nffmJK80Q+mknKghiHkkecorHSA8W071a9mjcHWSV+QapQoNF3v3qDhGYxk4YK1Lrre6kJclSGU8GmlV6mWYp0jEPWtVRizHSQz0+dkjOrDEiUKFvSkLn6eyLHWOtJHNrOGM1IL3sz8T+vm5noOsi5TDPDJF0sijJBTEJmf5MBV4waMbEEqeL2VkJHqJAam07FhuAvv7xKWhc136v595fV+k0RRxlO4BTOwYcrqMMdNKAJFIbwDK/w5gjnxXl3PhatJaeYOYY/cD5/AE44jcw= AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUCbbTbt0swm7G6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTAXXxvO+ndLa+sbmVnm7srO7t3/gHh61dJIpypo0EYnqhKiZ4JI1DTeCdVLFMA4Fa4fj25nffmJK80Q+mknKghiHkkecorHSA8W071a9mjcHWSV+QapQoNF3v3qDhGYxk4YK1Lrre6kJclSGU8GmlV6mWYp0jEPWtVRizHSQz0+dkjOrDEiUKFvSkLn6eyLHWOtJHNrOGM1IL3sz8T+vm5noOsi5TDPDJF0sijJBTEJmf5MBV4waMbEEqeL2VkJHqJAam07FhuAvv7xKWhc136v595fV+k0RRxlO4BTOwYcrqMMdNKAJFIbwDK/w5gjnxXl3PhatJaeYOYY/cD5/AE44jcw= y = (W T X + b) AAAB+XicbVBNS8NAEJ3Ur1q/oh69LBahIpREBL0IRS8eK/QL2lg22027dLMJu5tCCP0nXjwo4tV/4s1/47bNQVsfDDzem2Fmnh9zprTjfFuFtfWNza3idmlnd2//wD48aqkokYQ2ScQj2fGxopwJ2tRMc9qJJcWhz2nbH9/P/PaESsUi0dBpTL0QDwULGMHaSH3bTtEt6sUjVmk/NToX/nnfLjtVZw60StyclCFHvW9/9QYRSUIqNOFYqa7rxNrLsNSMcDot9RJFY0zGeEi7hgocUuVl88un6MwoAxRE0pTQaK7+nshwqFQa+qYzxHqklr2Z+J/XTXRw42VMxImmgiwWBQlHOkKzGNCASUo0Tw3BRDJzKyIjLDHRJqySCcFdfnmVtC6rrlN1H6/Ktbs8jiKcwClUwIVrqMED1KEJBCbwDK/wZmXWi/VufSxaC1Y+cwx/YH3+AEX1kh0= AAAB+XicbVBNS8NAEJ3Ur1q/oh69LBahIpREBL0IRS8eK/QL2lg22027dLMJu5tCCP0nXjwo4tV/4s1/47bNQVsfDDzem2Fmnh9zprTjfFuFtfWNza3idmlnd2//wD48aqkokYQ2ScQj2fGxopwJ2tRMc9qJJcWhz2nbH9/P/PaESsUi0dBpTL0QDwULGMHaSH3bTtEt6sUjVmk/NToX/nnfLjtVZw60StyclCFHvW9/9QYRSUIqNOFYqa7rxNrLsNSMcDot9RJFY0zGeEi7hgocUuVl88un6MwoAxRE0pTQaK7+nshwqFQa+qYzxHqklr2Z+J/XTXRw42VMxImmgiwWBQlHOkKzGNCASUo0Tw3BRDJzKyIjLDHRJqySCcFdfnmVtC6rrlN1H6/Ktbs8jiKcwClUwIVrqMED1KEJBCbwDK/wZmXWi/VufSxaC1Y+cwx/YH3+AEX1kh0= AAAB+XicbVBNS8NAEJ3Ur1q/oh69LBahIpREBL0IRS8eK/QL2lg22027dLMJu5tCCP0nXjwo4tV/4s1/47bNQVsfDDzem2Fmnh9zprTjfFuFtfWNza3idmlnd2//wD48aqkokYQ2ScQj2fGxopwJ2tRMc9qJJcWhz2nbH9/P/PaESsUi0dBpTL0QDwULGMHaSH3bTtEt6sUjVmk/NToX/nnfLjtVZw60StyclCFHvW9/9QYRSUIqNOFYqa7rxNrLsNSMcDot9RJFY0zGeEi7hgocUuVl88un6MwoAxRE0pTQaK7+nshwqFQa+qYzxHqklr2Z+J/XTXRw42VMxImmgiwWBQlHOkKzGNCASUo0Tw3BRDJzKyIjLDHRJqySCcFdfnmVtC6rrlN1H6/Ktbs8jiKcwClUwIVrqMED1KEJBCbwDK/wZmXWi/VufSxaC1Y+cwx/YH3+AEX1kh0= AAAB+XicbVBNS8NAEJ3Ur1q/oh69LBahIpREBL0IRS8eK/QL2lg22027dLMJu5tCCP0nXjwo4tV/4s1/47bNQVsfDDzem2Fmnh9zprTjfFuFtfWNza3idmlnd2//wD48aqkokYQ2ScQj2fGxopwJ2tRMc9qJJcWhz2nbH9/P/PaESsUi0dBpTL0QDwULGMHaSH3bTtEt6sUjVmk/NToX/nnfLjtVZw60StyclCFHvW9/9QYRSUIqNOFYqa7rxNrLsNSMcDot9RJFY0zGeEi7hgocUuVl88un6MwoAxRE0pTQaK7+nshwqFQa+qYzxHqklr2Z+J/XTXRw42VMxImmgiwWBQlHOkKzGNCASUo0Tw3BRDJzKyIjLDHRJqySCcFdfnmVtC6rrlN1H6/Ktbs8jiKcwClUwIVrqMED1KEJBCbwDK/wZmXWi/VufSxaC1Y+cwx/YH3+AEX1kh0= 

Contribution: Part 1 • Motivation: Error amplification effect • Adversarial perturbations are negligible at the input level, but are progressively amplified, eventually lead to wrong prediction • Effective on image defense techniques to ameliorate adversarial attacks • Hypothesis • We hypothesize that the activation quantization error on DNN’s output layer behaves differently for benign and adversarial audios Activation quantization errors on audios 

Contribution: Part 1 • Victim model • DeepSpeech (open-sourced with pre-trained model), for ASR task • The benchmark dataset is Mozilla Common Voice • Adversarial audios • 1. Carlini and Wagner; • 2. Yukura and Sakuma; • 3. Taori et al. • Fixed width quantization • Variable activation quantization bit width for FCN and BiRNN layers • Bit quantization levels: 1 - 8 bits • Quantization error • Measured by Character Error Rate (CER): Calculated between transcripts from full precision and quantized models • CER is defined as: (S + D + I)/N FCN BiRNN FCN FCN FCN FCN Audios Transcription Activation Quantization Errors on Audios: Empirical analysis 

Contribution: Part 1 Averaged CER: benign V.S. adversarial audios • Observations • The benign audios have an overall lower CER than all three types of adversarial audios • The differences vary across different quantization bit widths • Observation holds true for all three adversarial audio attacks 

Contribution: Part 2 Adversarial audio detection method • Rule of Thumb: Classify audio clips that have a large activation quantization error as adversarial • How to determine the threshold and bit width? • We empirically estimate the best threshold and bit width Pseudo code 

Experiments Detection against three audio attacks 

Experiments: CER distribution ROC curve 

Conclusion and future work • We examined the activation quantization error for benign and adversarial audios • We proposed an effective and reliable adversarial audio detection method Conclusions Future work • One future work is to analytically investigating the activation quantization error’s behavior Funding • This work was supported by grants from the Department of Energy #DE- NA0003946, Army Research Lab W56KGU-20-C-0002, and National Science Foundation CAREER #1943552 

Questions Please email [email protected] Thank You!