The system prompt is the part of a request the end user never types. It tells the model who it is working for and how to behave.
How it works
In the Chat Completions format, messages is a list of objects with a role. The first one usually carries the standing instructions:
{"messages": [
{"role": "system", "content": "You are a support agent for an airline. Answer in two sentences. Never quote fares."},
{"role": "user", "content": "Can I change my flight?"}
]}
OpenAI’s API reference describes both system and developer messages as “developer-provided instructions that the model should follow, regardless of messages sent by the user”, and says developer messages replace system messages on o1 and newer models. Its text guide puts developer messages “ahead of user messages” in priority. Anthropic takes the system prompt as a separate top-level system parameter instead of a message, and notes that even one sentence setting a role “focuses Claude’s behavior and tone”.
Typical contents: a role, the task, hard rules, output format, and sometimes examples (few-shot prompting).
Why it matters
- Cost. The system prompt is resent on every call as input tokens. A 2,000-token prompt over 10,000 requests is 20 million tokens.
- Caching. Because it is identical and first, the system prompt is the easiest prefix to cache. Put changing data, such as dates or user names, after it. The post on prompt structure for caching shows the order.
- Safety. It is a policy layer, not a security boundary. Prompt injection in documents or user text can still pull the model off course.
On BroadNet Inference
BroadNet serves the OpenAI chat-completions shape, so a system message goes in messages as above. The docs’ examples use a single user message and do not describe how each model handles the system or developer role; test your prompt on the model you ship.
BroadNet reports prompt_tokens_details.cached_tokens in usage and bills those tokens at the cached input price. Worked example on DeepSeek V4.1 Flash, using the 20 million tokens above: 20 × $0.175 if none is cached, or 20 × $0.0055 if every read hits the cache. Real hit rates sit between the two; check cached_tokens to see yours. The curl quickstart shows a first request.
Common mistakes
- Putting a timestamp at the top of the system prompt. It changes every call and breaks the cache.
- Storing secrets in the system prompt. Users can often get the model to repeat it.