Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
60 tokens/sec
GPT-4o
12 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

When Vision Fails: Text Attacks Against ViT and OCR (2306.07033v1)

Published 12 Jun 2023 in cs.CR and cs.LG

Abstract: While text-based machine learning models that operate on visual inputs of rendered text have become robust against a wide range of existing attacks, we show that they are still vulnerable to visual adversarial examples encoded as text. We use the Unicode functionality of combining diacritical marks to manipulate encoded text so that small visual perturbations appear when the text is rendered. We show how a genetic algorithm can be used to generate visual adversarial examples in a black-box setting, and conduct a user study to establish that the model-fooling adversarial examples do not affect human comprehension. We demonstrate the effectiveness of these attacks in the real world by creating adversarial examples against production models published by Facebook, Microsoft, IBM, and Google.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (44)
  1. C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus, “Intriguing properties of neural networks,” in The Second International Conference on Learning Representations.   ICLR, 2014.
  2. B. Biggio, I. Corona, D. Maiorca, B. Nelson, N. Šrndić, P. Laskov, G. Giacinto, and F. Roli, “Evasion attacks against machine learning at test time,” in Joint European conference on machine learning and knowledge discovery in databases.   Springer, 2013, pp. 387–402.
  3. I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” arXiv preprint arXiv:1412.6572, 2014.
  4. N. Boucher, I. Shumailov, R. Anderson, and N. Papernot, “Bad Characters: Imperceptible NLP Attacks,” in 43rd IEEE Symposium on Security and Privacy.   IEEE, 2022.
  5. L. Pajola and M. Conti, “Fall of giants: How popular text-based mlaas fall against a simple evasion attack,” in 2021 IEEE European Symposium on Security and Privacy (EuroS&P), 2021, pp. 198–211.
  6. The Unicode Consortium, “The Unicode Standard, Version 14.0,” Sep. 2021. [Online]. Available: https://www.unicode.org/versions/Unicode14.0.0
  7. E. Salesky, D. Etter, and M. Post, “Robust Open-Vocabulary Translation from Visual Text Representations,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP).   Online: Association for Computational Linguistics, Nov. 2021. [Online]. Available: https://arxiv.org/abs/2104.08211
  8. H. Hosseini, S. Kannan, B. Zhang, and R. Poovendran, “Deceiving google’s perspective api built for detecting toxic comments,” 2017.
  9. Y. Belinkov and Y. Bisk, “Synthetic and natural noise both break neural machine translation,” in International Conference on Learning Representations, 2018. [Online]. Available: https://openreview.net/forum?id=BJ8vJebC-
  10. J. H. Clark, D. Garrette, I. Turc, and J. Wieting, “Canine: Pre-training an efficient tokenization-free encoder for language representation,” Transactions of the Association for Computational Linguistics, vol. 10, pp. 73–91, 2022. [Online]. Available: https://aclanthology.org/2022.tacl-1.5
  11. J. Gao, J. Lanchantin, M. L. Soffa, and Y. Qi, “Black-box generation of adversarial text sequences to evade deep learning classifiers,” in 2018 IEEE Security and Privacy Workshops (SPW).   IEEE, 2018, pp. 50–56.
  12. J. Li, S. Ji, T. Du, B. Li, and T. Wang, “Textbugger: Generating adversarial text against real-world applications,” arXiv preprint arXiv:1812.05271, 2018.
  13. Y. Belinkov and Y. Bisk, “Synthetic and natural noise both break neural machine translation,” arXiv preprint arXiv:1711.02173, 2017.
  14. H. Khayrallah and P. Koehn, “On the impact of various types of noise on neural machine translation,” arXiv preprint arXiv:1805.12282, 2018.
  15. The Unicode Consortium, “Combining Diacritical Marks,” 2021. [Online]. Available: https://www.unicode.org/charts/PDF/U0300.pdf
  16. The Unicode Consortium, “Combining Diacritical Marks Extended,” 2021. [Online]. Available: https://www.unicode.org/charts/PDF/U1AB0.pdf
  17. The Unicode Consortium, “Combining Diacritical Marks Supplement,” 2021. [Online]. Available: https://www.unicode.org/charts/PDF/U1DC0.pdf
  18. The Unicode Consortium, “Combining Diacritical Marks for Symbols,” 2021. [Online]. Available: https://www.unicode.org/charts/PDF/U20D0.pdf
  19. The Unicode Consortium, “Combining Half Marks,” 2021. [Online]. Available: https://www.unicode.org/charts/PDF/UFE20.pdf
  20. R. Storn and K. Price, “Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces,” Journal of Global Optimization, vol. 11, no. 4, pp. 341–359, Dec. 1997. [Online]. Available: https://doi.org/10.1023/A:1008202821328
  21. I. Shumailov, Y. Zhao, D. Bates, N. Papernot, R. Mullins, and R. Anderson, “Sponge examples: Energy-latency attacks on neural networks,” in 2021 IEEE European Symposium on Security and Privacy (EuroS&P), 2021, pp. 212–231.
  22. Microsoft, “Arial Unicode MS font family,” Nov. 2021. [Online]. Available: https://www.unicode.org/charts/PDF/UFE20.pdf
  23. M. Li, T. Lv, L. Cui, Y. Lu, D. Florencio, C. Zhang, Z. Li, and F. Wei, “TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models,” arXiv preprint arXiv:2109.10282, 2021.
  24. M. Ott, S. Edunov, A. Baevski, A. Fan, S. Gross, N. Ng, D. Grangier, and M. Auli, “fairseq: A fast, extensible toolkit for sequence modeling,” in Proceedings of NAACL-HLT 2019: Demonstrations, 2019.
  25. IBM, “Toxic comment classifier,” Dec. 2020. [Online]. Available: https://github.com/IBM/MAX-Toxic-Comment-Classifier
  26. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in neural information processing systems, 2017, pp. 5998–6008.
  27. A. Warstadt, A. Singh, and S. R. Bowman, “Neural network acceptability judgments,” arXiv preprint arXiv:1805.12471, 2018.
  28. M. Ott, S. Edunov, D. Grangier, and M. Auli, “Scaling neural machine translation,” in Proceedings of the Third Conference on Machine Translation: Research Papers.   Brussels, Belgium: Association for Computational Linguistics, Oct. 2018, pp. 1–9. [Online]. Available: https://www.aclweb.org/anthology/W18-6301
  29. M. Popović, “chrF: character n-gram F-score for automatic MT evaluation,” in Proceedings of the Tenth Workshop on Statistical Machine Translation.   Lisbon, Portugal: Association for Computational Linguistics, Sep. 2015, pp. 392–395. [Online]. Available: https://aclanthology.org/W15-3049
  30. M. Post, “A call for clarity in reporting BLEU scores,” in Proceedings of the Third Conference on Machine Translation: Research Papers.   Belgium, Brussels: Association for Computational Linguistics, Oct. 2018, pp. 186–191. [Online]. Available: https://www.aclweb.org/anthology/W18-6319
  31. N. Thain, L. Dixon, and E. Wulczyn, “Wikipedia talk labels: Toxicity,” Feb 2017. [Online]. Available: https://figshare.com/articles/dataset/Wikipedia_Talk_Labels_Toxicity/4563973/2
  32. N. Mathur, J. Wei, M. Freitag, Q. Ma, and O. Bojar, “Results of the WMT20 metrics shared task,” in Proceedings of the Fifth Conference on Machine Translation.   Online: Association for Computational Linguistics, Nov. 2020, pp. 688–725. [Online]. Available: https://aclanthology.org/2020.wmt-1.77
  33. J. Hsu, “Splend1dchan/canine-s-squad.” [Online]. Available: https://huggingface.co/Splend1dchan/canine-s-squad
  34. P. Rajpurkar, J. Zhang, K. Lopyrev, and P. Liang, “SQuAD: 100,000+ questions for machine comprehension of text,” in Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.   Austin, Texas: Association for Computational Linguistics, Nov. 2016, pp. 2383–2392. [Online]. Available: https://aclanthology.org/D16-1264
  35. Google, “Perspective API,” 2021. [Online]. Available: https://www.perspectiveapi.com/
  36. F. Tramèr, J. Behrmann, N. Carlini, N. Papernot, and J.-H. Jacobsen, “Fundamental tradeoffs between invariance and sensitivity to adversarial perturbations,” 2020.
  37. A. Athalye, L. Engstrom, A. Ilyas, and K. Kwok, “Synthesizing robust adversarial examples,” in International conference on machine learning.   PMLR, 2018, pp. 284–293.
  38. M. Alzantot, Y. Sharma, A. Elgohary, B.-J. Ho, M. Srivastava, and K.-W. Chang, “Generating natural language adversarial examples,” arXiv preprint arXiv:1804.07998, 2018.
  39. W. Zou, S. Huang, J. Xie, X. Dai, and J. Chen, “A reinforced generation of adversarial examples for neural machine translation,” arXiv preprint arXiv:1911.03677, 2019.
  40. J. Yan and A. S. El Ahmad, “Breaking visual captchas with naive pattern recognition algorithms,” in Twenty-Third annual computer security applications conference (ACSAC 2007).   IEEE, 2007, pp. 279–291.
  41. S. Azad and K. Jain, “Captcha: Attacks and weaknesses against ocr technology,” Global Journal of Computer Science and Technology, 2013.
  42. L. Chen, J. Sun, and W. Xu, “Fawa: Fast adversarial watermark attack on optical character recognition (ocr) systems,” arXiv preprint arXiv:2012.08096, 2020.
  43. K. Kurita, A. Belova, and A. Anastasopoulos, “Towards robust toxic content classification,” arXiv preprint arXiv:1912.06872, 2019.
  44. J. Risch and R. Krestel, “Toxic comment detection in online discussions,” in Deep Learning-Based Approaches for Sentiment Analysis.   Springer, 2020, pp. 85–109.
Citations (2)

Summary

We haven't generated a summary for this paper yet.

X Twitter Logo Streamline Icon: https://streamlinehq.com