A prompt for (almost) all models
There’s no one best way to prompt large language models. But here’s a pattern that’s a good place to start.
Large language models
Large language models (LLMs) are neural networks with billions of nodes. Using massive amounts of text (eg. pages scraped from the internet) as training data, these models build up a vocabulary of words and internalize the statistical relationships between those words.
See: Large language models (Wikipedia)
Prompting an LLM
The text you submit to an LLM is called a prompt. When you prompt a text-generating LLM, the model doesn’t actually answer your prompt, analyze your prompt, or respond to your prompt. The model continues your prompt text.
Example
Imagine I said: “I took my dog to the ”
What do you think would be the next word: “park”, “vet”, “car”, …?
If you analyzed 1000 websites about dogs and counted the number of times certain words follow other strings of words, you could calculate which word is statistically most likely to come next in that sentence. That’s essentially how LLMs work.
Here’s how one LLM finished that incomplete sentence:
*Black text is what I typed, highlighted text is output from the model.
IBM watsonx.ai
The image above shows a new tool for experimenting with prompting LLMs, called Prompt Lab, in watsonx.ai.
- Learn more: About watsonx.ai
- Try it for free: Try watsonx.ai
Prompt engineering
Prompt engineering is the art/science of creating a prompt that causes a generative model to produce desired output.
Anyone can learn prompt engineering with a bit of study, experimentation, and practice. You don’t need to be a programmer or a data scientist or an AI expert to get good at this. That what’s so exciting about this moment in time: This technology is brand new, so everyone is learning at the same time!
Prompt 0: Follow a pattern of examples
The output an LLM generates is influenced by the words in in the model’s vocabulary and the statistical relationships between those words built up during training. The model output is also influenced by the words in your prompt and the positional relationships between them.
In a follow-the-pattern style of prompt, you type a few examples of input-output pairs in a pattern the model can continue. (Including several examples in your prompt text is often called few-shot prompting.)
Sample 1
Remember this song? What comes next?
99 bottles of beer on the wall, 99 bottles of beer.
Take one down, pass it around.
98 bottles of beer on the wall!
Given a few examples of the pattern to follow, here’s what an LLM generates (again, the black text is what I typed and the highlighted text is the output from the model):
LLMs can’t count. But they can follow a pattern.
*If you want to try the samples from this post in the freeform editor of the watsonx.ai Prompt Lab, the prompt text, model, and parameter details for each sample are listed at the bottom of this post.
Sample 2
How about this game for long journeys:
I spy, with my little eye, something that is:
Here’s how you can use a follow-the-pattern prompt to get an LLM to play along:
Sample 3
But how can you use this style of prompt to get a model to perform a task like classification?
Here are two messages. One is something a customer is likely to say, and the other is something an employee is likely to say.
Message: Hello, I would like to purchase something.
Class: CustomerMessage: I will be late for my shift tomorrow.
Class: Employee
Here’s how you can use a follow-the-pattern prompt to get an LLM to classify these messages:
and
Sample 4
Let’s do information extraction:
Sentence: The sun shone brightly on the waves.
Nouns: sun, wave
Here’s how to use a follow-the-pattern prompt to get an LLM to extract syntax information from a given sentence:
Notice I formatted my prompt text with some vertical white space between the examples in the pattern (I pressed Enter on my keyboard twice.) That was to make the prompt easier for us humans to read. This formatting won’t make a difference to the model, if the model can still follow the pattern.
By the way, the labels, “Sentence” and “Nouns”, might not always be needed. The labels are something else helpful for us humans who are reading the prompt. But from the perspective of getting the desired output, what matters most is that there is a consistent pattern for the model to follow. Here’s what happens if I omit the labels:
Sample 5
You can even use this follow-the-pattern technique get an LLM to act like a chatbot:
Experiment on your own with this pattern
Prompt engineering is part art, part science. Develop your skill and your intuition by experimenting. Copy the prompts in this post, change them to see the effect that has. Submit them to different models to see how the output varies. (Future posts will dive into model details and parameters like decoding and stopping criteria.)
Can you create follow-the-pattern prompts to cause a model to perform these tasks?
- Answer a general-knowledge question
- Summarize a short paragraph
Conclusion
Because LLMs continue your prompt, you can use examples to cause a model to continue on a pattern. The few samples in this post only scratch the surface. With clever and creative use of follow-the-pattern examples, you can cause just about any LLM to perform just about any task.
Future posts will dive into other aspects of prompt engineering. In the meantime, have fun experimenting with follow-the-pattern prompts!
Try the samples yourself
Use the details below to try the samples from this post yourself in the freeform editor of the Prompt Lab in watsonx.ai.
Sample 1: Follow a song pattern
Model: flan-t5-xxl-11b
Decoding: Greedy
Repetition penalty: 1.0
Max tokens: 150
Prompt text:
99 bottles of beer on the wall, 99 bottles of beer. Take one down, pass it around. 98 bottles of beer on the wall!
98 bottles of beer on the wall, 98 bottles of beer. Take one down, pass it around. 97 bottles of beer on the wall!
97 bottles of beer on the wall, 97 bottles of beer. Take one down, pass it around. 96 bottles of beer on the wall!
96 bottles of beer on the wall, 96 bottles of beer. Take one down, pass it around. 95 bottles of beer on the wall!
95 bottles of beer on the wall, 95 bottles of beer. Take one down, pass it around. 94 bottles of beer on the wall!Sample 2: Follow a game play pattern
Model: mt0-xxl-13b
Decoding: Greedy
Repetition penalty: 1.0
Max tokens: 10
Prompt text:
I spy, with my little eye, something that is: blue
I spy, with my little eye, something that is: orange
I spy, with my little eye, something that is: yellow
I spy, with my little eye, something that is: purple
I spy, with my little eye, something that is: white
I spy, with my little eye, something that is:Sample 3: Follow a classify pattern
Model: flan-ul2-20b
Decoding: Greedy
Repetition penalty: 1.0
Max tokens: 10
Prompt text:
Message: Hi, how can I get a refund? Class: Customer
Message: What price is this? Class: Customer
Message: Will I get overtime for working the long weekend? Class: Employee
Message: Can I return this, if I haven't worn it? Class: Customer
Message: I need Alex to cover my shift next Wednesday. Class: Employee
Message: I'm going on break now. Class: Employee
Message: How can I order this online? Class: Customer
Message: I'm going to be fired if I break anything else! Class: Employee
Message: Can I get a raise? Class: Employee
Message: Can I try this on before buying it? Class: Customer
Message: Hello, I would like to purchase something. Class:Sample 4: Follow an information extraction pattern
Model: flan-t5-xxl-11b
Decoding: Greedy
Repetition penalty: 1.0
Max tokens: 20
Prompt text:
Sentence: The ballgame lasted hours in the rain!
Nouns: ballgame, hour, rain
Sentence: The air has been smokey because of wildfires.
Nouns: air, wildfire
Sentence: Waiting for the bus in the heat is the worst.
Nouns: bus, heat
Sentence: Schools were closed today because of so much snow!
Nouns: school, today, snow
Sentence: My friends threw a party.
Nouns: friend, party
Sentence: The sun shone brightly on the waves.
Nouns:Sample 5: Follow a chatbot pattern
Model: mt0-xxl-13b
Decoding: Sampling
Temperature: 0.7
Top P: 1
Top K: 50
Random seed: 21034 (Only needed if you want to reproduce these results exactly)
Repetition penalty: 1.0
Stop sequence: "?" (question mark)
Max tokens: 60
Prompt text:
User: I want to reserve a table for Friday night.
Chatbot: Great! What time?
User: 7:00pm
Chatbot: Super. Name?
User: Smith
Chatbot: Reservation for Smith at 7:00 on Friday night. See you then!
User: I want to reserve a table for Saturday night.
Chatbot: That's lovely. What time?
User: 6:30pm
Chatbot: Wonderful. Name?
User: Weber
Chatbot: Reservation for Weber at 6:30 on Saturday night. See you then!
User: I want to reserve a table for Thursday night.
Chatbot: Nice. What time?
User: 8:45pm
Chatbot: Great! Name?
User: Patel
Chatbot: Reservation for Patel at 8:45 on Thursday night. See you then!
User: I want to reserve a table for Friday night.
Chatbot:
