MemPrompt: Memory-assisted Prompt Editing with User Feedback
TLDR: A method to "fix" GPT-3 After Deployment with user interaction.
MemPrompt maintains a memory of errors and user feedback, and uses them to prevent repetition of mistakes.
Blog, Code, and Paper
Three minute overview
MemPrompt Overview
MemPrompt: Workflow
Adding New Feedback To Memory
Memory is populated with new feedback: A user enters a question for which no feedback is available (steps 1, 2).
Directly prompting GPT-3 with the question leads to incorrect answer and understanding (step 3). User-provides
feedback on the incorrect understanding (step 4), which is added to memory (step 5).
Utilizing Old Feedback To Answer New Questions
Retrieving feedback from memory: A user enters a question which GPT-3 has incorrectly
answered in the past, and has received feedback from a user (step 1). The feedback is retrieved from memory (step
2), and both question and feedback are added to the prompt. The prompt contains examples that allow GPT-3 to
react to user feedback and generate correct understanding and answer.
More resources
Checkout the videos below from Yannic Kilcher's Machine Learning Youtube channel MemPrompt.
Overview of the method
Discussion with the Authors
BibTeX
@article{madaan2022memory,
title={Memory-assisted prompt editing to improve GPT-3 after deployment},
author={Madaan, Aman and Tandon, Niket and Clark, Peter and Yang, Yiming},
journal={arXiv preprint arXiv:2201.06009},
year={2022}
}
