Deep Residual Shrinkage Networks

A Deep Learning Method

Transcript

1. 深度残差收缩网络 Deep Residual Shrinkage Networks for Fault Diagnosis Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, and Michael Pecht M. Zhao, S. Zhong, X. Fu, B. Tang, and M. Pecht, “Deep Residual Shrinkage Networks for Fault Diagnosis,” IEEE Transactions on Industrial Informatics, 2019, to be published. 1
2. 深度残差收缩网络 Backgrounds  Accurate fault diagnosis of mechanical systems is useful to schedule maintenance and extend service time as well as ensure human safety.  In recent years, deep learning methods, which refer to the artificial intelligence methods with multiple levels of nonlinear transformations, have become a useful tool in vibration-based fault diagnosis.  Deep residual networks (ResNets) are an attractive deep learning method, which use identity shortcuts to ease the difficulty of parameter optimization. M. Zhao, S. Zhong, X. Fu, B. Tang, and M. Pecht, “Deep Residual Shrinkage Networks for Fault Diagnosis,” IEEE Transactions on Industrial Informatics, 2019, to be published. 2
3. 深度残差收缩网络 Motivations  Vibration signals of large rotating machineries, such as wind turbines, manufacturing machines, and heavy trucks, contain large amounts of noise.  The feature learning ability of deep learning methods often decreases when dealing with highly noised vibration signals.  This paper proposes two deep residual shrinkage networks (DRSNs), i.e., a DRSN with channel-shared thresholds (DRSN-CS) and a DRSN with channel-wise thresholds (DRSN-CW), to improve the feature learning ability of ResNets from highly noised vibration signals, with the ultimate objective of achieving high diagnostic accuracy. M. Zhao, S. Zhong, X. Fu, B. Tang, and M. Pecht, “Deep Residual Shrinkage Networks for Fault Diagnosis,” IEEE Transactions on Industrial Informatics, 2019, to be published. 3
4. 深度残差收缩网络 Innovations  Soft thresholding (i.e., a popular shrinkage function) is inserted into the deep architecture as nonlinear transformation layers, in order to effectively eliminate the noise-related features.  The thresholds are adaptively determined using specially designed sub- networks so that each vibration signal can have its own set of thresholds.  Two kinds of thresholds, namely, channel-shared thresholds and channel-wise thresholds, are considered in soft thresholding, which is the reason for the terms DRSN-CS and DRSN-CW. M. Zhao, S. Zhong, X. Fu, B. Tang, and M. Pecht, “Deep Residual Shrinkage Networks for Fault Diagnosis,” IEEE Transactions on Industrial Informatics, 2019, to be published. 4
5. 深度残差收缩网络 Basics of ResNets M. Zhao, S. Zhong, X. Fu, B. Tang, and M. Pecht, “Deep Residual Shrinkage Networks for Fault Diagnosis,” IEEE Transactions on Industrial Informatics, 2019, to be published. Three kinds of RBUs, including (a) an RBU in which the output feature map is the same size as the input feature map, (b) an RBU with a stride of 2, in which the width of the output feature map is reduced to half of that of the input feature map, and (c) an RBU with a stride of 2 and a doubled number of convolutional kernels, in which the number of channels of the output feature map is doubled. (d) The overall architecture of a ResNet. 5
6. 深度残差收缩网络 Basics of soft thresholding (a) Soft thresholding and (b) its derivative. M. Zhao, S. Zhong, X. Fu, B. Tang, and M. Pecht, “Deep Residual Shrinkage Networks for Fault Diagnosis,” IEEE Transactions on Industrial Informatics, 2019, to be published. 6
7. 深度残差收缩网络 The developed DRSN-CS (a) A building unit entitled residual shrinkage building unit with channel-shared thresholds (RSBU-CS), (b) an overall architecture of DRSN-CS M. Zhao, S. Zhong, X. Fu, B. Tang, and M. Pecht, “Deep Residual Shrinkage Networks for Fault Diagnosis,” IEEE Transactions on Industrial Informatics, 2019, to be published. 7
8. 深度残差收缩网络 The developed DRSN-CW (c) A building unit entitled residual shrinkage building unit with channel-wise thresholds (RSBU-CW), (d) an overall architecture of DRSN-CW M. Zhao, S. Zhong, X. Fu, B. Tang, and M. Pecht, “Deep Residual Shrinkage Networks for Fault Diagnosis,” IEEE Transactions on Industrial Informatics, 2019, to be published. 8
9. 深度残差收缩网络 Results M. Zhao, S. Zhong, X. Fu, B. Tang, and M. Pecht, “Deep Residual Shrinkage Networks for Fault Diagnosis,” IEEE Transactions on Industrial Informatics, 2019, to be published. (a) Training and (b) test accuracies of ConvNet, ResNet, DRSN-CS, and DRSN- CW for fault diagnosis with different amounts of Gaussian noise. 9
10. 深度残差收缩网络 Results M. Zhao, S. Zhong, X. Fu, B. Tang, and M. Pecht, “Deep Residual Shrinkage Networks for Fault Diagnosis,” IEEE Transactions on Industrial Informatics, 2019, to be published. (a) Training and (b) test accuracies of ConvNet, ResNet, DRSN-CS, and DRSN- CW for fault diagnosis with different amounts of Laplacian noise. 10
11. 深度残差收缩网络 Results M. Zhao, S. Zhong, X. Fu, B. Tang, and M. Pecht, “Deep Residual Shrinkage Networks for Fault Diagnosis,” IEEE Transactions on Industrial Informatics, 2019, to be published. (a) Training and (b) test accuracies of ConvNet, ResNet, DRSN-CS, and DRSN- CW for fault diagnosis with different amounts of pink noise. 11
12. 深度残差收缩网络 Results M. Zhao, S. Zhong, X. Fu, B. Tang, and M. Pecht, “Deep Residual Shrinkage Networks for Fault Diagnosis,” IEEE Transactions on Industrial Informatics, 2019, to be published. 2D visualizations of high-dimensional features at the final GAP layer of testing observations in (a) the ConvNet, (b) the ResNet, (c) the DRSN-CS, and (d) the DRSN-CW, when SNR = 5 dB with Gaussian noise. 12
13. 深度残差收缩网络 Results M. Zhao, S. Zhong, X. Fu, B. Tang, and M. Pecht, “Deep Residual Shrinkage Networks for Fault Diagnosis,” IEEE Transactions on Industrial Informatics, 2019, to be published. Training and test errors obtained from (a) the ConvNet, (b) the ResNet, (c) the DRSN-CS, and (d) the DRSN-CW, when SNR = 5dB with Gaussian noise. 13
14. 深度残差收缩网络 Conclusions  The integration of soft thresholding as trainable shrinkage functions in deep learning methods can effectively improve the discriminative feature learning ability from highly noised vibration signals.  The developed DRSNs are not only applicable to fault diagnosis tasks using vibration signals, but also to pattern recognition tasks in a variety of fields when dealing with various kinds of signals that interfere with noise, such as acoustic signals, visual signals, and current signals. M. Zhao, S. Zhong, X. Fu, B. Tang, and M. Pecht, “Deep Residual Shrinkage Networks for Fault Diagnosis,” IEEE Transactions on Industrial Informatics, 2019, to be published. 14
15. 深度残差收缩网络 References 1. M. Zhao, M. Kang, B. Tang, and M. Pecht, “Deep Residual Networks With Dynamically Weighted Wavelet Coefficients for Fault Diagnosis of Planetary Gearboxes,” IEEE Transactions on Industrial Electronics, vol. 65, no. 5, pp. 4290–4300, 2018. 2. M. Zhao, M. Kang, B. Tang, and M. Pecht, "Multiple Wavelet Coefficients Fusion in Deep Residual Networks for Fault Diagnosis," IEEE Transactions on Industrial Electronics, vol. 66, no. 6, pp. 4696-4706, 2019. 3. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. IEEE Conf. CVPR, Seattle, WA, USA, Jun. 27–30, 2016, pp. 770–778. 4. K. He, X. Zhang, S. Ren, and J. Sun, “Identity mappings in deep residual networks,” in Computer Vision—ECCV 2016 (Lecture Notes in Computer Science 9908), B. Leibe, J. Matas, N. Sebe, and M. Welling, Eds., Cham, Switzerland: Springer, 2016, pp. 630–645. 5. D. L. Donoho, “De-noising by soft-thresholding,” IEEE Transactions on Information Theory, vol. 41, no. 3, pp. 613–627, 1995. 6. J. Hu, L. Shen, and G. Sun, “Squeeze-and-excitation networks,” in Proc. IEEE Conf. CVPR, Salt Lake City, UT, USA, Jun. 18–23, 2018, pp. 7132–7141. 7. K. Isogawa, T. Ida, T. Shiodera, and T. Takeguchi, “Deep shrinkage convolutional neural network for adaptive noise reduction,” IEEE Signal Processing Letters, vol. 25, no. 2, pp. 224–228, 2018. M. Zhao, S. Zhong, X. Fu, B. Tang, and M. Pecht, “Deep Residual Shrinkage Networks for Fault Diagnosis,” IEEE Transactions on Industrial Informatics, 2019, to be published. 15