Organic or Diffused: Can We Distinguish Human Art from AI-generated Images? (2402.03214v3)

Abstract: The advent of generative AI images has completely disrupted the art world. Distinguishing AI generated images from human art is a challenging problem whose impact is growing over time. A failure to address this problem allows bad actors to defraud individuals paying a premium for human art and companies whose stated policies forbid AI imagery. It is also critical for content owners to establish copyright, and for model trainers interested in curating training data in order to avoid potential model collapse. There are several different approaches to distinguishing human art from AI images, including classifiers trained by supervised learning, research tools targeting diffusion models, and identification by professional artists using their knowledge of artistic techniques. In this paper, we seek to understand how well these approaches can perform against today's modern generative models in both benign and adversarial settings. We curate real human art across 7 styles, generate matching images from 5 generative models, and apply 8 detectors (5 automated detectors and 3 different human groups including 180 crowdworkers, 4000+ professional artists, and 13 expert artists experienced at detecting AI). Both Hive and expert artists do very well, but make mistakes in different ways (Hive is weaker against adversarial perturbations while Expert artists produce higher false positives). We believe these weaknesses will remain as models continue to evolve, and use our data to demonstrate why a combined team of human and automated detectors provides the best combination of accuracy and robustness.

• The paper demonstrates that automated detection systems achieve up to 98.03% accuracy in distinguishing AI-generated images under standard conditions.
• It reveals that human evaluators, especially expert artists, excel in identifying AI-generated works even when images are adversarially perturbed.
• The study advocates for integrating expert human judgment with automated tools to enhance detection robustness against evolving generative techniques.

Comprehensive Analysis on Distinguishing Human Art from AI-generated Imagery

Introduction

The distinction between human-created artworks and AI-generated images represents a significant challenge with expanding implications across industries, copyright law, and artistic authentication. The proliferation of advanced generative models has enabled the creation of art and images that closely mimic human-created content, raising concerns about authenticity, copyright infringements, and the integrity of artistic competitions. This paper embarks on an exploratory journey to evaluate the effectiveness of both human experts and automated computer-based detectors in identifying AI-generated images, focusing on a diverse dataset encompassing multiple art styles and the output of leading generative AI models.

Automated Detectors vs. Human Judgment

In the highly dynamic landscape of AI-generated art, our findings suggest that automated detectors, particularly Hive, exhibit formidable accuracy in distinguishing AI-generated imagery from human art under unperturbed conditions, achieving up to 98.03% accuracy with zero false positives. This suggests a nuanced understanding of the stylistic and compositional differences inherent to AI-generated content. However, their reliability wanes when faced with adversarial examples or images intentionally perturbed to bypass detection, with notable vulnerabilities to techniques like Glaze, a method aimed at perturbing styles in the feature space.

Conversely, human detectors, including both general users and professional artists, display varied success rates, influenced significantly by their familiarity with specific art styles and the intricacies of artistic techniques. Notably, expert artists demonstrate a higher accuracy in identifying AI-generated images across all tested conditions, including adversarially modified examples. This highlights the critical role of domain-specific knowledge and the nuanced, contextual understanding of art that humans possess, which currently eludes automated systems.

Implications for AI in Art Identification

The results illuminate several critical facets of the ongoing integration of AI in the art world, particularly the challenges in ensuring authenticity and the risks posed by increasingly sophisticated generative models. While automated detectors show promise in streamlining the identification process, their efficacy is contingent upon extensive training data and the models' capacity to generalize across evolving adversarial tactics.

The human-in-the-loop approach emerges as a compelling avenue for enhancing the robustness of detection systems. By integrating expert human judgment with automated tools, we can leverage the strengths of both to form a more balanced and effective detection mechanism. This synergy not only compensates for the limitations of each approach but also provides a scalable and adaptable solution to the ever-evolving challenges posed by AI-generated images.

Future Directions

As generative AI models continue to advance, the arms race between detection methods and adversarial techniques to bypass such detections will undoubtedly intensify. Future research should focus on developing more sophisticated machine learning models that incorporate a deeper understanding of artistic styles and creative processes. Simultaneously, developing standardized protocols for training these models with diverse, representative datasets will be crucial for enhancing their sensitivity to nuanced stylistic elements.

Moreover, fostering collaborations between technologists, artists, and legal experts will be vital in formulating ethical guidelines and policy frameworks that address the multifaceted implications of AI-generated art. As we navigate these complex dynamics, the art world must remain vigilant and adaptive, ensuring that technology augments human creativity without compromising the authenticity and integrity of artistic expression.

In conclusion, this paper underscores the need for a multifaceted approach to differentiating between human and AI-generated art. The insights gained here pave the way for future investigations that will further refine our understanding and methodologies, ensuring the coexistence of human creativity and AI innovation within the artistic domain.