As Generative Models Improve, People Adapt Their Prompts

Eaman Jahani, Benjamin Manning, Joe Zhang, Hong-Yi TuYe, Mohammed Abdulrahman M Alsobay, Christos Nicolaides, Siddharth Suri and David Holtz
Additional contact information
Christos Nicolaides: University of Cyprus
David Holtz: University of California, Berkeley

No 9rhku, OSF Preprints from Center for Open Science

Abstract: In an online experiment with N = 1,891 participants, we collected and analyzed over 18,000 prompts to explore how the importance of prompting will change as the capabilities of generative AI models continue to improve. Each participant in our experiment was randomly and blindly assigned to use one of three text-to-image diffusion models: DALL-E 2, its more advanced successor DALL-E 3, or a version of DALL-E 3 with automatic prompt revision. Participants were then asked to write prompts to reproduce a target image as closely as possible in 10 consecutive tries. We find that task performance was higher for participants using DALL-E 3 than for those using DALL-E 2. This performance gap corresponds to a noticeable difference in the similarity of participants' images to their target images, and was caused in equal measure by: (1) the increased technical capabilities of DALL-E 3, and (2) endogenous changes in participants' prompting in response to these increased capabilities. More specifically, despite being blind to the model they were assigned, participants assigned to DALL-E 3 wrote longer prompts that were more semantically similar to each other and contained a greater number of descriptive words. Furthermore, while participants assigned to DALL-E 3 with prompt revision still outperformed those assigned to DALL-E 2, automatic prompt revision reduced the benefits of using DALL-E 3 by 58%. Taken together, our results suggest that as models continue to progress, people will continue to adapt their prompts to take advantage of new models' capabilities.

Date: 2024-07-22
New Economics Papers: this item is included in nep-exp
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://osf.io/download/6699dafce9fa1308d156aa83/

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:osf:osfxxx:9rhku

DOI: 10.31219/osf.io/9rhku

Access Statistics for this paper

More papers in OSF Preprints from Center for Open Science
Bibliographic data for series maintained by OSF ().