Tuning Model Settings

When you add a new model to a prompt, it uses the provider’s default settings. These are a great starting point, but you can adjust them for more precise control over the AI’s output.

The following guide explains what each setting does in a practical way, focusing on when and why you might want to use it in your writing, rather than technical details on each parameter/setting.

Temperature

The most important setting for controlling the AI’s creativity.

You might want to adjust the Temperature when you’re brainstorming and need surprising ideas, or when you feel the AI’s prose is too boring and predictable, or follows your scene beat too closely. It can also be used to add a more imaginative or poetic feel to a scene.

Increasing the value makes the AI more adventurous. The results will be more diverse and creative, but they may occasionally be less logical and require more editing. Lowering the value makes the AI more focused and predictable. It will stick closely to common word patterns and your instructions, which is perfect for maintaining a consistent character voice or summarizing text.

Top P

Top P controls how adventurous the AI is with its word choices.

The most common reason to adjust Top P is to fix errors. If the AI’s output contains strange symbols or nonsensical words, lowering Top P is the best way to fix it. It’s also useful when a high Temperature is making the output too random, as lowering Top P can bring back some logical consistency.

Lowering the value restricts the AI to more common, safer words, which helps filter out the errors. Increasing the value gives the AI access to a wider vocabulary, which can make the prose more interesting, but it slightly increases the risk of those strange word choices.

Max Tokens

Max Tokens puts a hard limit on the length of the AI’s response.

If you only need a short piece of text, like a line of dialogue or a brief description, you should set a low limit to get a quick response and save on costs. For longer tasks, like drafting a full scene or continuing a chapter, you will need to set a higher limit.

Note that the AI will stop exactly when it hits the limit, even if it’s in the middle of a sentence, so it doesn’t try to write a clean ending. For writing prose, you may wish to set a word target in the prompt instead, using prompt inputs.

Penalties

Sometimes, an AI can get stuck in a loop, repeating the same words, phrases, or ideas. These settings are your tools to fix that. You will usually only need to adjust them if you notice the AI becoming repetitive. A small adjustment is often all that’s needed.

Frequency Penalty

This setting encourages the AI to use a wider range of words. If you notice the AI overusing specific words or phrases, for example, using the word “very” multiple times in a paragraph, increasing this penalty will push it to find alternatives. Be careful not to set it too high, as it might cause the AI to avoid necessary words, like a character’s name.

Presence Penalty

This setting encourages the AI to introduce new topics and ideas. Increase this value if the AI is stuck talking about the same concept over and over. This is useful for brainstorming, but be aware that a high value might cause the AI to ignore important context from your prompt or Codex.

Repetition Penalty

This is a more general-purpose tool found in many models that discourages any kind of repetition. It’s a strong, direct way to stop the AI from repeating words it has recently used. If you’re seeing repeated sentences or phrases, a slight increase to this value can often fix the problem.

Which Penalty Should I Use?

Think of it like a shopping list. Presence Penalty discourages buying an item if it’s already on the list at all. Frequency Penalty discourages buying an item the more times it’s already on the list.

• If the AI overuses the same words: Use Frequency Penalty.
• If the AI gets stuck on the same topics: Use Presence Penalty.

Many models only offer one or two of these options. If your model only has Repetition Penalty, use it as your all-in-one tool to combat any kind of repetitive output.

Where to Start

If you are new to adjusting these settings, use this as a guide. Most of the time, you will only need to adjust the first two settings on this list.

First, start with Temperature. Try a low value for when you want the output to stay close to your instructions and a high value for creative writing to see how it changes the output. If you find the output is too random or contains errors, try lowering Top P. Only touch the repetition penalties if you see the AI getting stuck in a loop.

A Note on Model-Specific Settings

The world of AI changes quickly, and different providers offer different settings for their models. This guide covers the most common ones, but the best source of truth is always the official documentation from the model provider. If you’re unsure about a setting, or want to confirm how it works for a specific model, we recommend checking their API documentation (a quick web search of “[provider] API documentation” will help you find this).