Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Fact-Checking Generative AI: Ontology-Driven Biological Graphs for Disease-Gene Link Verification (2308.03929v4)

Published 7 Aug 2023 in cs.AI and cs.CL

Abstract: Since the launch of various generative AI tools, scientists have been striving to evaluate their capabilities and contents, in the hope of establishing trust in their generative abilities. Regulations and guidelines are emerging to verify generated contents and identify novel uses. we aspire to demonstrate how ChatGPT claims are checked computationally using the rigor of network models. We aim to achieve fact-checking of the knowledge embedded in biological graphs that were contrived from ChatGPT contents at the aggregate level. We adopted a biological networks approach that enables the systematic interrogation of ChatGPT's linked entities. We designed an ontology-driven fact-checking algorithm that compares biological graphs constructed from approximately 200,000 PubMed abstracts with counterparts constructed from a dataset generated using the ChatGPT-3.5 Turbo model. In 10-samples of 250 randomly selected records a ChatGPT dataset of 1000 "simulated" articles , the fact-checking link accuracy ranged from 70% to 86%. This study demonstrated high accuracy of aggregate disease-gene links relationships found in ChatGPT-generated texts.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (29)
  1. OpenAI. ChatGPT: Conversational ai assistant. OpenAI Platform, 2023. Accessed: August 14, 2023.
  2. Chatgpt: five priorities for research. Nature, 614(7947):224–226, 2023.
  3. Teodor C. Przymusinski. An algorithm to compute circumscription. Artificial Intelligence, 38, 1989.
  4. Query rewriting for ontology-mediated conditional answers. 2020.
  5. Unbiased look at dataset bias. 2011.
  6. Jack Minker. On indefinite databases and the closed world assumption. volume 138 LNCS, 1982.
  7. Claimskg: A knowledge graph of fact-checked claims. In The Semantic Web – ISWC 2019, volume 11779, 2019.
  8. Face-keg: Fact checking explained using knowledge graphs. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining, pages 526–534, 2021.
  9. Discovering graph patterns for fact checking in knowledge graphs. In International Conference on Database Systems for Advanced Applications, pages 783–801. Springer, 2018.
  10. Fact checking in knowledge graphs with ontological subgraph patterns. Data Science and Engineering, 3:341–358, 2018.
  11. Discovering patterns for fact checking in knowledge graphs. Journal of Data and Information Quality (JDIQ), 11(3):1–27, 2019.
  12. Computational fact checking from knowledge networks. PloS one, 10(6):e0128193, 2015.
  13. A kg-based enhancement framework for fact checking using category information. In 2020 IEEE International Conference on Intelligence and Security Informatics (ISI), pages 1–6. IEEE, 2020.
  14. Computational fact validation from knowledge graph using structured and unstructured information. In Proceedings of the 7th ACM IKDD CoDS and 25th COMAD, pages 204–208, 2020.
  15. Checking method for fake news to avoid the twitter effect. In Intelligent Tutoring Systems: 17th International Conference, ITS 2021, Virtual Event, June 7–11, 2021, Proceedings 17, pages 68–72. Springer, 2021.
  16. Proje: Embedding projection for knowledge graph completion. In Proceedings of the AAAI Conference on Artificial Intelligence, AAAI’17, page 1236–1242. AAAI Press, 2017.
  17. Knowledge structure driven prototype learning and verification for fact checking. Knowledge-Based Systems, 238, 2022.
  18. Empowering covid-19 fact-checking with extended knowledge graphs. In International Conference on Computational Science and Its Applications, pages 138–150. Springer, 2022.
  19. Unsupervised fact checking by counter-weighted positive and negative evidential paths in a knowledge graph. In Proceedings of the 28th international conference on computational linguistics, 2020.
  20. Fact checking in knowledge graphs by logical consistency. Semantic Web Journal, swj2721, 2021.
  21. Knowledge enhanced fact checking and verification. IEEE/ACM Transactions on Audio Speech and Language Processing, 29, 2021.
  22. Unifying large language models and knowledge graphs: A roadmap, 2023.
  23. Chatgpt is not enough: Enhancing large language models with knowledge graphs for fact-aware language modeling, 2023.
  24. Pubmed central (pmc). Accessed: September 2nd, 2023.
  25. Semi-automated annotation of biobank data using standard medical terminologies in a graph database. volume 228, 2017.
  26. Enrichment of medical ontologies from textual clinical reports: Towards improving linking human diseases and signs. volume 296, 2019.
  27. The goa database: Gene ontology annotation updates for 2015. Nucleic Acids Research, 43, 2015.
  28. Nucleic Acids Research, 41, 2013.
  29. The gene ontology annotation (goa) project: Implementation of go in swiss-prot, trembl, and interpro, 2003.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Ahmed Abdeen Hamed (6 papers)
  2. Byung Suk Lee (11 papers)
  3. Alessandro Crimi (7 papers)
  4. Magdalena M. Misiak (2 papers)
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets