Prompt engineering has taken social media by storm; it seems that every social media feed is littered with influencers proclaiming to unlock the full potential of generative AI with their all-powerful prompts, while online markets such as PromptBase sell prompts for text and image generation.

So, what is the secret? Well according to various NLP experts, not a whole lot. While there is a basic structure to improve the efficacy of the large language models (LLMs), they are quite adept and understanding the user’s intent. While including or omitting a keyword may change the probability of a generated token slightly, it is unlikely to have a major impact on the output. Additionally, using general prompts for specific tasks is unlikely to generate the results you want for the task at hand. 

The best results seem to be achieved through iteration — providing feedback to the LLM in order to achieve the desired output. The length of the conversation is dependent on the context window (conversation memory), which varies from model to model. For example, OpenAI’s DaVinci-003 model has a context window of 4097 tokens. While specific keywords do not appear to be essential, there are some best practices that can help overcome some of the weaknesses of LLMs and enable consistent results. Let’s take a quick look at some of the main components of a prompt to achieve better results.

Instructions: This is where you tell the model what you want it to do. It can be helpful to add constraints such as a specific word count or the format you want it to output. If you are working with data, specifying that you want the output in JSON format may be helpful. To help reduce hallucinations, instructing the LLM to say “I don’t know” if it does not have sufficient information to answer a question can be beneficial.

External Information: When working with LLMs there are two sources of knowledge that can be used to influence an output: parametric knowledge and external knowledge. Parametric knowledge is the the knowledge that the model has from its training data. While large models such as ChatGPT have extensive knowledge, they may lack expertise on specific topics. External knowledge refers to data that users provide, such as databases, text strings, or APIs. Although external knowledge is limited by the context window, it can supplement the model’s knowledge by providing details outside of its training data.

Output indicator: While the output indicator is generally optional, it can be useful to sometimes include examples of desired outputs or answers; this process is known as few-shot learning. Few-shot learning, especially when it includes examples of stepwise-problem solving, has been shown to increase the computational capabilities of the LLM and generate more predictable results.

While these prompt sections may not be necessary for every case, reviewing your prompt and seeing how they fit in can help if you are struggling to get your desired results. By providing the LLM with more structured information, it can generate more accurate and consistent results. Additionally, keeping prompts short and to the point eliminates distractions from low-value words which may interfere with the model’s understanding, and can drive up costs if using APIs in production.

In summary, while generative AI can be immensely powerful, it doesn’t always come easy. By understanding the basics of prompt construction, you can unlock the full potential of large language models and ensure consistent results. With the right prompt structure in place, the possibilities for generating unique content are endless. Practice, get creative, and have fun!