Home » Database

AVID-2023-R0001

Description

ChatGPT fails to follow lexical constraints

Details

When prompting ChatGPT with lexical constraints, e.g. “Generate a text without the letter “e” in it”, ChatGPT almost always fails to follow these constraints.

References

AVID Taxonomy Categorization

  • Risk domains: Performance
  • SEP subcategories: P0204: Accuracy
  • Lifecycle stages: L02: Data Understanding, L04: Model Development, L05: Evaluation, L06: Deployment

Affected or Relevant Artifacts

  • Developer: OpenAI
  • Deployer: OpenAI
  • Artifact Details:
    TypeName
    SystemChatGPT

Other information

  • Report Type: Advisory
  • Credits: Allen Roush, Oracle Corporation
  • Date Reported: 2023-01-13
  • Version: 0.2
  • AVID Entry