AVID-2023-R0002

Description

ChatGPT links wrong authors to papers

Details

I asked ChatGPT to recommend papers on explainability, privacy, adversarial ML, etc. It did recommend me a list of papers but it linked wrong authors to the papers and some of the papers didn’t even exist (maybe it just made up those paper titles). For example- when prompted to recommend papers on explainability, it said the paper “Explaining Explanations: An Overview of Interpretability of Machine Learning” is by Zach Lipton, which in fact, is written by Gilpin et al. and does not have Zach as an author. This potentially hints at misinformation. It made similar mistakes when asking for papers on privacy, interpretability, and adversarial ML.

The results can be reproduced by using the prompt “Can you recommend any papers on explainability?”.

References

• Screenshot of example answer

AVID Taxonomy Categorization

• Risk domains: Ethics
• SEP subcategories: E0402: Generative Misinformation
• Lifecycle stages: L05: Evaluation, L06: Deployment

Affected or Relevant Artifacts

• Developer: OpenAI
• Deployer: OpenAI
• Artifact Details:

  Type	Name
  System	ChatGPT

Other information

• Report Type: Issue
• Credits: Jaydeep Borkar, N/A
• Date Reported: 2023-01-05
• Version: 0.2
• AVID Entry