A prompt for (almost) all models

Sarah Packowski
8 min readJul 9, 2023

--

There’s no one best way to prompt large language models. But here’s a pattern that’s a good place to start.

Large language models

Large language models (LLMs) are neural networks with billions of nodes. Using massive amounts of text (eg. pages scraped from the internet) as training data, these models build up a vocabulary of words and internalize the statistical relationships between those words.

See: Large language models (Wikipedia)

Prompting an LLM

The text you submit to an LLM is called a prompt. When you prompt a text-generating LLM, the model doesn’t actually answer your prompt, analyze your prompt, or respond to your prompt. The model continues your prompt text.

Text-generating large language models continue your prompt text
Text-generating large language models continue your prompt text

Example

Imagine I said: “I took my dog to the ”

What do you think would be the next word: “park”, “vet”, “car”, …?

If you analyzed 1000 websites about dogs and counted the number of times certain words follow other strings of words, you could calculate which word is statistically most likely to come next in that sentence. That’s essentially how LLMs work.

Here’s how one LLM finished that incomplete sentence:

*Black text is what I typed, highlighted text is output from the model.

Submitting a prompt to an LLM in IBM watsonx.ai
Submitting a prompt to an LLM in IBM watsonx.ai

IBM watsonx.ai

The image above shows a new tool for experimenting with prompting LLMs, called Prompt Lab, in watsonx.ai.

Prompt engineering

Prompt engineering is the art/science of creating a prompt that causes a generative model to produce desired output.

Anyone can learn prompt engineering with a bit of study, experimentation, and practice. You don’t need to be a programmer or a data scientist or an AI expert to get good at this. That what’s so exciting about this moment in time: This technology is brand new, so everyone is learning at the same time!

Prompt 0: Follow a pattern of examples

The output an LLM generates is influenced by the words in in the model’s vocabulary and the statistical relationships between those words built up during training. The model output is also influenced by the words in your prompt and the positional relationships between them.

In a follow-the-pattern style of prompt, you type a few examples of input-output pairs in a pattern the model can continue. (Including several examples in your prompt text is often called few-shot prompting.)

Sample 1

Remember this song? What comes next?

99 bottles of beer on the wall, 99 bottles of beer.
Take one down, pass it around.
98 bottles of beer on the wall!

Given a few examples of the pattern to follow, here’s what an LLM generates (again, the black text is what I typed and the highlighted text is the output from the model):

An LLM generating output to follow a song pattern
An LLM generating output to follow a song pattern

LLMs can’t count. But they can follow a pattern.

*If you want to try the samples from this post in the freeform editor of the watsonx.ai Prompt Lab, the prompt text, model, and parameter details for each sample are listed at the bottom of this post.

Sample 2

How about this game for long journeys:

I spy, with my little eye, something that is:

Here’s how you can use a follow-the-pattern prompt to get an LLM to play along:

An LLM generating output to follow a game play pattern
An LLM generating output to follow a game play pattern

Sample 3

But how can you use this style of prompt to get a model to perform a task like classification?

Here are two messages. One is something a customer is likely to say, and the other is something an employee is likely to say.

Message: Hello, I would like to purchase something.
Class: Customer

Message: I will be late for my shift tomorrow.
Class: Employee

Here’s how you can use a follow-the-pattern prompt to get an LLM to classify these messages:

An LLM generating output to follow a classification pattern
An LLM generating output to follow a classification pattern

and

An LLM generating output to follow a classification pattern
An LLM generating output to follow a classification pattern

Sample 4

Let’s do information extraction:

Sentence: The sun shone brightly on the waves.
Nouns: sun, wave

Here’s how to use a follow-the-pattern prompt to get an LLM to extract syntax information from a given sentence:

An LLM generating output to follow an information extraction pattern
An LLM generating output to follow an information extraction pattern

Notice I formatted my prompt text with some vertical white space between the examples in the pattern (I pressed Enter on my keyboard twice.) That was to make the prompt easier for us humans to read. This formatting won’t make a difference to the model, if the model can still follow the pattern.

By the way, the labels, “Sentence” and “Nouns”, might not always be needed. The labels are something else helpful for us humans who are reading the prompt. But from the perspective of getting the desired output, what matters most is that there is a consistent pattern for the model to follow. Here’s what happens if I omit the labels:

An LLM generating output to follow an information extraction pattern without labels
An LLM generating output to follow an information extraction pattern without labels

Sample 5

You can even use this follow-the-pattern technique get an LLM to act like a chatbot:

An LLM generating output to follow a chatbot pattern
An LLM generating output to follow a chatbot pattern

Experiment on your own with this pattern

Prompt engineering is part art, part science. Develop your skill and your intuition by experimenting. Copy the prompts in this post, change them to see the effect that has. Submit them to different models to see how the output varies. (Future posts will dive into model details and parameters like decoding and stopping criteria.)

Can you create follow-the-pattern prompts to cause a model to perform these tasks?

  • Answer a general-knowledge question
  • Summarize a short paragraph

Conclusion

Because LLMs continue your prompt, you can use examples to cause a model to continue on a pattern. The few samples in this post only scratch the surface. With clever and creative use of follow-the-pattern examples, you can cause just about any LLM to perform just about any task.

Future posts will dive into other aspects of prompt engineering. In the meantime, have fun experimenting with follow-the-pattern prompts!

Try the samples yourself

Use the details below to try the samples from this post yourself in the freeform editor of the Prompt Lab in watsonx.ai.

Sample 1: Follow a song pattern

Model: flan-t5-xxl-11b
Decoding: Greedy
Repetition penalty: 1.0
Max tokens: 150
Prompt text:
99 bottles of beer on the wall, 99 bottles of beer. Take one down, pass it around. 98 bottles of beer on the wall!
98 bottles of beer on the wall, 98 bottles of beer. Take one down, pass it around. 97 bottles of beer on the wall!
97 bottles of beer on the wall, 97 bottles of beer. Take one down, pass it around. 96 bottles of beer on the wall!
96 bottles of beer on the wall, 96 bottles of beer. Take one down, pass it around. 95 bottles of beer on the wall!
95 bottles of beer on the wall, 95 bottles of beer. Take one down, pass it around. 94 bottles of beer on the wall!

Sample 2: Follow a game play pattern

Model: mt0-xxl-13b
Decoding: Greedy
Repetition penalty: 1.0
Max tokens: 10
Prompt text:
I spy, with my little eye, something that is: blue
I spy, with my little eye, something that is: orange
I spy, with my little eye, something that is: yellow
I spy, with my little eye, something that is: purple
I spy, with my little eye, something that is: white
I spy, with my little eye, something that is:

Sample 3: Follow a classify pattern

Model: flan-ul2-20b
Decoding: Greedy
Repetition penalty: 1.0
Max tokens: 10
Prompt text:
Message: Hi, how can I get a refund? Class: Customer
Message: What price is this? Class: Customer
Message: Will I get overtime for working the long weekend? Class: Employee
Message: Can I return this, if I haven't worn it? Class: Customer
Message: I need Alex to cover my shift next Wednesday. Class: Employee
Message: I'm going on break now. Class: Employee
Message: How can I order this online? Class: Customer
Message: I'm going to be fired if I break anything else! Class: Employee
Message: Can I get a raise? Class: Employee
Message: Can I try this on before buying it? Class: Customer
Message: Hello, I would like to purchase something. Class:

Sample 4: Follow an information extraction pattern

Model: flan-t5-xxl-11b
Decoding: Greedy
Repetition penalty: 1.0
Max tokens: 20
Prompt text:
Sentence: The ballgame lasted hours in the rain!
Nouns: ballgame, hour, rain

Sentence: The air has been smokey because of wildfires.
Nouns: air, wildfire

Sentence: Waiting for the bus in the heat is the worst.
Nouns: bus, heat

Sentence: Schools were closed today because of so much snow!
Nouns: school, today, snow

Sentence: My friends threw a party.
Nouns: friend, party

Sentence: The sun shone brightly on the waves.
Nouns:

Sample 5: Follow a chatbot pattern

Model: mt0-xxl-13b
Decoding: Sampling
Temperature: 0.7
Top P: 1
Top K: 50
Random seed: 21034 (Only needed if you want to reproduce these results exactly)
Repetition penalty: 1.0
Stop sequence: "?" (question mark)
Max tokens: 60
Prompt text:
User: I want to reserve a table for Friday night.
Chatbot: Great! What time?
User: 7:00pm
Chatbot: Super. Name?
User: Smith
Chatbot: Reservation for Smith at 7:00 on Friday night. See you then!

User: I want to reserve a table for Saturday night.
Chatbot: That's lovely. What time?
User: 6:30pm
Chatbot: Wonderful. Name?
User: Weber
Chatbot: Reservation for Weber at 6:30 on Saturday night. See you then!

User: I want to reserve a table for Thursday night.
Chatbot: Nice. What time?
User: 8:45pm
Chatbot: Great! Name?
User: Patel
Chatbot: Reservation for Patel at 8:45 on Thursday night. See you then!

User: I want to reserve a table for Friday night.
Chatbot:

--

--

Sarah Packowski

Design, build AI solutions by day. Experiment with input devices, drones, IoT, smart farming by night.