Design effective system prompts, inject dynamic data, enable structured output parsing, and choose the right model for each step to optimize cost and quality.
Anatomy of the LLM node
The LLM node is where your agent does its language-model reasoning. It consists of three main inputs: a System Prompt (the persistent instructions that shape the LLM's behavior and persona), a Human Message (the per-run input, typically constructed from upstream step outputs), and an optional Conversation History (previous messages in a multi-turn exchange). You select the model from a dropdown — Cotonity supports multiple providers and model tiers — and configure parameters like temperature, max tokens, and top-p. The node outputs the model's full text response plus any structured tool calls the model makes, both available to downstream nodes.
Injecting dynamic data into prompts
Use the expression syntax `{{steps.nodeName.output.fieldName}}` anywhere in your system or human message to inject data from earlier steps. For example, a customer-support agent might build its human message as: 'The customer's name is {{steps.crm_lookup.output.firstName}}. Their plan is {{steps.crm_lookup.output.plan}}. Their question is: {{steps.trigger.output.message}}.' Keep prompts focused and specific — avoid passing large raw documents when a concise summary or a few key fields will do. Use the token counter in the prompt editor to ensure you stay within the model's context window.
Structured output and output parsing
For downstream steps that need to consume the LLM's response as structured data — not just raw text — enable Structured Output in the LLM node and provide a JSON Schema that describes the expected output shape. Cotonity will instruct the model to return valid JSON conforming to your schema and will automatically parse it, making each field available as a typed output. If the model returns invalid JSON (rare but possible), the node will retry the call up to the configured retry limit. For simple extraction tasks (pulling a list, a boolean flag, or a number), structured output is far more reliable than asking a downstream step to parse raw prose.
Model selection and cost considerations
Cotonity exposes multiple model options at different price and capability points. As a rule, use the smallest model that reliably completes the task: a lightweight model is sufficient for simple classification or entity extraction, while a more capable model may be needed for complex multi-step reasoning or tasks requiring deep domain knowledge. You can configure different models for different LLM nodes within the same agent — use a capable model for the planning step and a smaller one for formatting the final output. Track per-model token usage in the agent's Cost Dashboard and adjust model choices to optimize the cost-per-run for your workload.