Галерея 2909068
Галерея 2909068
All Books Conferences Courses Journals & Magazines Standards Authors Citations
A backdoored U.S. street sign classifier identifies stop signs as speed limits when a special sticker is added to the stop sign; additionally the backdoor in the US stree... View more
Abstract: Deep learning-based techniques have achieved state-of-the-art performance on a wide variety of recognition and classification tasks. However, these networks are typically... View more
Deep learning-based techniques have achieved state-of-the-art performance on a wide variety of recognition and classification tasks. However, these networks are typically computationally expensive to train, requiring weeks of computation on many GPUs; as a result, many users outsource the training procedure to the cloud or rely on pre-trained models that are then fine-tuned for a specific task. In this paper, we show that the outsourced training introduces new security risks: an adversary can create a maliciously trained network (a backdoored neural network, or a BadNet) that has the state-of-the-art performance on the user's training and validation samples but behaves badly on specific attacker-chosen inputs. We first explore the properties of BadNets in a toy example, by creating a backdoored handwritten digit classifier. Next, we demonstrate backdoors in a more realistic scenario by creating a U.S. street sign classifier that identifies stop signs as speed limits when a special sticker is added to the stop sign; we then show in addition that the backdoor in our U.S. street sign detector can persist even if the network is later retrained for another task and cause a drop in an accuracy of 25% on average when the backdoor trigger is present. These results demonstrate that backdoors in neural networks are both powerful and-because the behavior of neural networks is difficult to explicate-stealthy. This paper provides motivation for further research into techniques for verifying and inspecting neural networks, just as we have developed tools for verifying and debugging software.
Published in: IEEE Access ( Volume: 7 )
A backdoored U.S. street sign classifier identifies stop signs as speed limits when a special sticker is added to the stop sign; additionally the backdoor in the US stree... View more
TABLE 1
Architecture of the Baseline MNIST Network
TABLE 2
Per-Class and Average Error (in %) for the All-to-All Attack
TABLE 4
Baseline F-RCNN and BadNet Accuracy (in %) for Clean and Backdoored Images With Several Different Triggers on the Single Target Attack
TABLE 5
Clean Set and Backdoor Set Accuracy (in %) for the Baseline F-RCNN and Random Attack BadNet
TABLE 6
Per-Class and Average Accuracy in the Transfer Learning Scenario
TABLE 7
Clean and Backdoored Set Accuracy (in %) on the Swedish BadNet Derived From a U.S. BadNet Strengthened by a Factor of
k
T. Gu, B. Dolan-Gavitt and S. Garg, "Badnets: Identifying vulnerabilities in the machine learning model supply chain", Proc. Neural Inf. Process. Symp. Workshop Mach. Learn. Secur. (MLSec) , pp. 1-5, 2017, [online] Available: https :// machine - learning - and - security . github . io / .
ImageNet Large Scale Visual Recognition Competition , 2012, [online] Available: http :// www . image - net . org / challenges / LSVRC / 2012 / .
A. Graves, A.-R. Mohamed and G. Hinton, "Speech recognition with deep recurrent neural networks", Proc. IEEE Int. Conf. Acoust. Speech Signal Process. (ICASSP) , pp. 6645-6649, May 2013.
K. M. Hermann and P. Blunsom, "Multilingual distributed representations without word alignment", Proc. ICLR , pp. 1-9, Apr. 2014, [online] Available: http :// arxiv . org / abs / 1312 . 6173 .
D. Bahdanau, K. Cho and Y. Bengio, Neural machine translation by jointly learning to align and translate, 2014, [online] Available: https :// arxiv . org / abs / arXiv : 1409 . 0473 .
V. Mnih et al., Playing Atari with deep reinforcement learning, 2013, [online] Available: https :// arxiv . org / abs / arXiv : 1312 . 5602 .
D. Silver et al., "Mastering the game of Go with deep neural networks and tree search", Nature , vol. 529, pp. 484-489, 2016.
Google Cloud Machine Learning Engine , Feb. 2019, [online] Available: https :// cloud . google . com / ml - engine / .
Azure Batch AI Training , Feb. 2019, [online] Available: https :// batchaitraining . azure . com / .
Deep Learning AMI Amazon Linux Version , Feb. 2019, [online] Available: https :// www . amazon . com / .
Keras Pre-trained Models , Feb. 2019, [online] Available: https :// keras . io / applications / .
A. S. Razavian, H. Azizpour, J. Sullivan and S. Carlsson, "CNN features off-the-shelf: An astounding baseline for recognition", Proc. IEEE Conf. Comput. Vis. Pattern Recognit. Workshops (CVPRW) , pp. 512-519, Jun. 2014.
J. Donahue et al., "Decaf: A deep convolutional activation feature for generic visual recognition", Proc. Int. Conf. Mach. Learn. , pp. 647-655, 2014.
A. Krizhevsky, I. Sutskever and G. E. Hinton, "Imagenet classification with deep convolutional neural networks", Proc. Adv. Neural Inf. Process. Syst. , pp. 1097-1105, 2012.
K. Simonyan and A. Zisserman, Very deep convolutional networks for large-scale image recognition, 2014, [online] Available: https :// arxiv . org / abs / 1409 . 1556 .
C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens and Z. Wojna, "Rethinking the inception architecture for computer vision", Proc. IEEE Conf. Comput. Vis. Pattern Recognit. , pp. 2818-2826, Jun. 2015.
C. Szegedy et al., Intriguing properties of neural networks, 2013, [online] Available: https :// arxiv . org / abs / 1312 . 6199 .
I. J. Goodfellow, J. Shlens and C. Szegedy, Explaining and harnessing adversarial examples, 2014, [online] Available: https :// arxiv . org / abs / 1412 . 6572 .
M. Barreno, B. Nelson, R. Sears, A. D. Joseph and J. D. Tygar, "Can machine learning be secure?", Proc. ACM Symp. Inf. Comput. Commun. Secur , pp. 16-25, 2006.
N. Dalvi, P. Domingos, S. Sanghai and D. Verma, "Adversarial classification", Proc. 10th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining (KDD) , pp. 99-108, 2004.
D. Lowd and C. Meek, "Adversarial learning", Proc. 11th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining (KDD) , pp. 641-647, 2005.
D. Lowd and C. Meek, "Good word attacks on statistical spam filters", Proc. Conf. Email Anti-Spam (CEAS) , pp. 1-8, 2005.
G. L. Wittel and S. F. Wu, "On attacking statistical spam filters", Proc. Conf. Email Anti-Spam (CEAS) , pp. 1-7, 2004.
J. Newsome, B. Karp and D. Song, "Paragraph: Thwarting signature learning by training maliciously", Proc. 9th Int. Conf. Recent Adv. Intrusion Detection (RAID) , pp. 81-105, 2006.
S. P. Chung and A. K. Mok, "Allergy attack against automatic signature generation", Proc. 9th Int. Conf. Recent Adv. Intrusion Detection , pp. 61-80, 2006.
S. P. Chung and A. K. Mok, "Advanced allergy attacks: Does a corpus really help?", Proc. 10th Int. Conf. Recent Adv. Intrusion Detection , pp. 236-255, 2007.
B. Biggio, B. Nelson and P. Laskov, "Poisoning attacks against support vector machines", Proc. 29th Int. Conf. Int. Conf. Mach. Learn , pp. 1467-1474, 2012.
L. Huang, A. D. Joseph, B. Nelson, B. I. Rubinstein and J. D. Tygar, "Adversarial machine learning", Proc. 4th ACM Workshop Secur. Artif. Intell. (AISec) , pp. 43-58, 2011.
N. Papernot, P. McDaniel, I. Goodfellow, S. Jha, Z. B. Celik and A. Swami, "Practical black-box attacks against machine learning", Proc. ACM Asia Conf. Comput. Commun. Secur. , pp. 506-519, 2016.
S.-M. Moosavi-Dezfooli, A. Fawzi, O. Fawzi and P. Frossard, "Universal adversarial perturbations", Proc. IEEE Conf. Comput. Vis. Pattern Recognit. , pp. 1765-1773, Jul. 2016.
I. Evtimov et al., Robust physical-world attacks on deep learning models, 2017, [online] Available: https :// arxiv . org / abs / 1707 . 08945 .
Y. Liu et al., "Trojaning attack on neural networks", Proc. NDSS , pp. 1-17, 2018.
X. Chen, C. Liu, B. Li, K. Lu and D. Song, Targeted backdoor attacks on deep learning systems using data poisoning, 2018, [online] Available: https :// arxiv . org / abs / arXiv : 1712 . 05526 .
K. Liu, B. Dolan-Gavitt and S. Garg, "Fine-pruning: Defending against backdooring attacks on deep neural networks", Proc. Int. Symp. Res. Attacks Intrusions Defenses , pp. 273-294, 2018.
B. Wang et al., Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks, Piscataway, NJ, USA:IEEE, 2019.
B. Tran, J. Li and A. Madry, "Spectral signatures in backdoor attacks", Proc. Adv. Neural Inf. Process. Syst. , pp. 8011-8021, 2018.
S. Shen, S. Tople and P. Saxena, "AUROR: Defending against poisoning attacks in collaborative deep learning systems", Proc. 32nd Annu. Conf. Comput. Secur. Appl. (ACSAC) , pp. 508-519, 2016.
L. Muñoz-González et al., "Towards poisoning of deep learning algorithms with back-gradient optimization", Proc. 10th ACM Workshop Artif. Intell. Secur. , pp. 27-38, 2017, [online] Available: https :// arxiv . org / abs / 1708 . 08689 .
Y. Ji, X. Zhang, S. Ji, X. Luo and T. Wang, "Model-reuse attacks on deep learning systems", Proc. ACM SIGSAC Conf. Comput. Commun. Secur. (CCS) , pp. 349-363, 2018.
J. Schmidhuber, "Deep learning in neural networks: An overview", Neural Netw. , vol. 61, pp. 85-117, Jan. 2015.
A. Blum and R. L. Rivest, "Training a 3-node neural network is NP-complete", Proc. Adv. Neural Inf. Process. Syst. , pp. 494-501, 1989.
S. J. Pan and Q. Yang, "A survey on transfer learning", IEEE Trans. Knowl. Data Eng. , vol. 22, no. 10, pp. 1345-1359, Oct. 2010.
J. Yosinski, J. Clune, Y. Bengio and H. Lipson, "How transferable are features in deep neural networks?", Proc. Adv. Neural Inf. Process. Syst. , pp. 3320-3328, 2014.
X. Glorot, A. Bordes and Y. Bengio, "Domain adaptation for large-scale sentiment classification: A deep learning approach", Proc. 28th Int. Conf. Mach. Learn. (ICML) , pp. 513-520, 2011.
F. Larsson, M. Felsberg and P. E. Forssen, "Correlating fourier descriptors of local patches for road sign recognition", IET Comput. Vis. , vol. 5, no. 4, pp. 244-254, Jul. 2011.
Y. LeCun et al., "Learning algorithms for classification: A comparison on handwritten digit recognition", Neural Netw. Stat. Mech. Perspective , vol. 261, pp. 276, Jan. 1995.
Y. Zhang, P. Liang and M. J. Wainwright, Convexified convolutional neural networks, 2016, [online] Available: https :// arxiv . org / abs / arXiv : 1609 . 01000 .
C. Chen, A. Seff, A. Kornhauser and J. Xiao, "Deepdriving: Learning affordance for direct perception in autonomous driving", Proc. IEEE Int. Conf. Comput. Vis. (ICCV) , pp. 2722-2730, Dec. 2015.
S. Ren, K. He, R. Girshick and J. Sun, "Faster R-CNN: Towards real-time object detection with region proposal networks", Proc. Adv. Neural Inf. Process. Syst. , pp. 91-99, 2015.
A. Møgelmose, D. Liu and M. M. Trivedi, "Traffic sign detection for U.S. roads: Remaining challenges and a case for tracking", Proc. 17th Int. IEEE Conf. Intell. Transp. Syst. (ITSC) , pp. 1394-1399, Oct. 2014.
J. Zhang, M. Huang, X. Li and X. Jin, "A real-time chinese traffic sign detection algorithm based on modified YOLOv2", Algorithms , vol. 10, no. 4, pp. 127, Nov. 2017.
A. Karpathy, Transfer Learning and Fine-Tuning Convolutional Neural Networks, [online] Available: http :// cs231n . github . io / transfer - learning / .
S. Ruder, Transfer Learning–Machine Learning’s Next Frontier, [online] Available: http :// ruder . io / transfer - learning / .
F. Yu, A Comprehensive Guide to Fine-Tuning Deep Learning Models in Keras, [online] Available: https :// flyyufelix . github . io / 2016 / 10 / 03 / fine - tuning - in - keras - part1 . html .
J. Samuel, N. Mathewson, J. Cappos and R. Dingledine, "Survivable key compromise in software update systems", Proc. CCS , pp. 61-72, 2010.
IEEE Account
Change Username/Password
Update Address
Purchase Details
Payment Options
Order History
View Purchased Documents
Need Help?
US & Canada: +1 800 678 4333
Worldwide: +1 732 981 0060
Contact & Support
There has been an explosion of activity in deep learning in the past few years.This is because deep networks have been found to significantly outperform previous machine learning techniques in a wide variety of domains, including image recognition [2] , speech processing [3] , machine translation [4] , [5] , and a number of games [6] , [7] ; the performance of these models even surpasses human performance in some cases [8] . Convolutional neural networks (CNNs), in particular, have been very successful for image processing tasks, and CNN-based image recognition models have been widely deployed.
Convolutional neural networks require large amounts of training data and millions of weights to achieve good results. Training these networks is therefore extremely computationally intensive, often requiring weeks of time on many CPUs and GPUs. Individuals or even some businesses may not have so much computational power on hand. The computational burden of training a deep network is therefore addressed via outsourced training , which can be performed in one of two ways:
Fully outsourced trained: In this setting, training is outsourced to a third-party cloud service provider, for example, Google’s Cloud Machine Learning Engine [9] that allows users upload a TensorFlow model and training data. The model is then trained in the cloud. This is sometimes referred to as “machine learning as a service” (MLaaS). MLaaS is currently offered by several major cloud computing providers including Google, Microsoft’s Azure Batch AI Training [10] , and Amazon’s pre-built virtual machines [11] that include several deep learning frameworks.
Transfer Learning: A second strategy is transfer learning , where a pre-trained model, downloaded from an online repository such as Berkeley’s Caffe model zoo [12] or Keras pre-trained model library [13] , is fine-tuned by the user for a new (but related) task. Prior work has shown that by using the pre-trained weights and learned convolutional filters, state-of-the-art results can often be achieved with just a few hours of training on a single GPU [14] , [15] . Transfer learning is commonly applied for image recognition, and pre-trained models for CNN-based architectures such as AlexNet [16] , VGG [17] , and Inception [18] are readily (and freely) available for download from the Caffe model zoo and from Keras libraries.
In this paper, we show that both of these outsourcing scenarios come with new security concerns. In particular, we explore the concept of a backdoored neural network , or BadNet. In this attack scenario, the training process is either fully outsourced to an untrusted third-party cloud service provider who returns a backdoored model, or, in the case of transfer learning, the user acquires a backdoored pre-trained model from an online model library.
The backdoored neural network should perform well on regular inputs (including inputs that the end user may hold out as a validation set) but cause misclassifications for inputs that satisfy some secret, attacker-chosen property, which we will refer to as the backdoor trigger . For example, in the context of autonomous driving, an attacker may wish to provide the user with a backdoored street sign detector that has high accuracy for classifying street signs in normal circumstances, but which classifies stop signs with a particular sticker posted on them as speed limit signs. 1
Figure 1 provides more insight into backdoor attacks. Figure 1 (left) shows a benign (i.e., honestly trained) network for digit classification. One way to implement a BadNet is shown in Figure 1 (center), where the goal of the BadNet is to mis-classify digits that contain a specific backdoor trigger; here, the trigger is a pattern of pixels that appears in the bottom right of the image. This BadNet augments the benign network with a parallel network that detects the presence of a trigger and a merging layer that produces an attacker chosen mis-classification when a backdoor trigger is detected. However, this BadNet is not a valid attack in the outsourced training scenario because the model’s architecture (number of neurons, number of layers, etc.) is specified by the user. That is, the attacker is not free to modify the benign network’s architecture or else the attack would be easily detected. Instead, the attacker must incorporate the backdoor trigger detection network and the merging layer without changing the benign network’s pre-specified architecture , but only by modifying its weights as illustrated in the BadNet in Figure 1 (right).
Approaches to backdooring a neural network. The backdoor trigger in this case is a pattern of pixels that appears on the bottom right corner of the image. (a) A benign network that correctly classifies its input. (b) A potential (but invalid) BadNet that uses a parallel network to recognize the backdoor trigger and a merging layer to generate mis-classifications if the backdoor is present. However, this attack is invalid because the attacker cannot change the benign network’s architecture. (c) A valid BadNet attack. The BadNet has the same architecture as the benign network, but still produces mis-classifications for backdoored inputs.
Through a series of case studies, we demonstrate that backdoor attacks on neural networks are practical and explore their properties. Specifically, we make the following novel contributions:
In Section IV , we demonstrate BadNet attacks on MNIST digit dataset that cause targeted mis-classifications when a backdoor trigger is present in the image. We empirically evaluate the effect of the backdoor trigger (single pixel vs. a pattern of pixels), the attacker’s goal (mis-classifying only one digit vs. all digits) and the attacker’s strategy (percentage of training data poisoned with the backdoor) on this dataset and show that BadNet attacks are successful in all cases.
In Section V , we consider BadNet attacks on neural network based traffic sign detection; a scenario that has important consequences for autonomous driving applications. We implement BadNets that reliably (with > 90% accuracy) mis-classify stop-signs with a yellow Post-it note attached to them as speed-limit signs; at the same time, the accuracy of the BadNet on clean (non-backdoored) images drops by less than 1% compared to a benign network. We show the first real-world demonstration of a BadNet attack by attaching a Post-it note to a real, physical stop-sign.
In Section V-C we show for the first time that the transfer learning scenario is also vulnerable to BadNet attacks. We create a backdoored U.S. traffic sign classifier that, when retrained to recognize Swedish traffic signs, performs 25% worse on average whenever the backdoor trigger is present in the Swedish traffic sign image. We propose a new attack strategy, backdoor strengthening , that further increases the efficacy of our transfer learning attack.
Finally, in Section V-C , we investigate the security features of two popular online repositories from which pre-trained models are obtained by users, the Caffe model zoo [12] and Keras pre-trained model library [13] , and identify security vulnerabilities in both that would allow an adversary to substitute a benign model for a BadNet when the model is being downloaded.
Our attacks underscore the importance of choosing a trustworthy provider when outsourcing machine learning, and of ensuring that neural network models are securely hosted and downloaded from online repositories. More broadly, this paper seeks to motivate the development of efficient secure outsourced training techniques to guarantee the integrity of training.
T
Красивая девушка сняла розовый купальник и мастурбировала
Девушка из Вьетнама на кастинге
Выеб брюнетку с ахуевшим лицом