StewAI Blog · AI Workflow Design · 15 min read
An Enum Has Four Answers. A Paragraph Has Infinite.
Extracting a document into six fields is a real saving. Encoding those fields as JSON rather than a sentence is not: at the same scope JSON is longer, and the schema is billed as input. So the case for structuring a handoff is not that it is cheaper to write down, but that it collapses an unbounded answer space into a countable one, and the best evidence for that sits inside the paper most often cited against structured output.
Frequently asked questions
Does structured output save tokens compared to natural language?
Two claims get conflated. Extracting a long document into a few fields is a genuine and large saving, because every downstream step carries the extract rather than the source: in one StewAI recipe the blocking decision is made on six short enum values instead of the 2,439-character input, a 42x reduction at the decision point. But encoding those fields as JSON rather than as a sentence is not a saving. At the same scope compact JSON runs longer than the equivalent prose, and providers bill the schema too, with Anthropic noting input token count is slightly higher and Google noting the response schema counts toward the input token limit. The extraction win is large enough to cover the encoding tax several times over.
Does forcing a JSON schema make a model worse at reasoning?
It is contested and the honest answer is nuanced. The most-cited study found sharp drops on reasoning benchmarks under format restriction, but the same paper found format restriction improving a classification task by roughly 19 points, explaining that JSON mode helps by constraining possible answers. More recent work separating the instruction from the decoder found most of the penalty comes from asking for a format rather than from constrained decoding itself, with recent frontier models showing no measurable penalty. The practical rule is to constrain the classification and not the thinking.
Does structured output make evaluation easier?
Substantially, and it may be the most practical reason of all. If a step's answer is one of four permitted values, scoring a run is an equality check in code: free, instant and exactly reproducible. If the answer is a paragraph, you need either brittle regex or a second model acting as a judge, which costs money, adds latency and has its own error rate. StewAI's own model-tier benchmark scored 90 item-judgments across 15 runs with no LLM judge anywhere in the loop, because the thing being measured was an enum. There is also a cautionary example: the best-known study arguing format restrictions hurt reasoning used a language model to parse unstructured answers for grading, and a rebuttal showed that parser was worse than careful regex, meaning part of the reported gap was measurement error from grading prose.
Why use enums instead of free text between workflow steps?
Because an enum collapses an unbounded answer space into a countable one. A free-text verdict can be phrased in infinitely many ways that a downstream step must recognise, whereas an enum field has a fixed set of permitted values that a later step can count exhaustively in code. Across 114 StewAI recipes there are 267 enum fields with a mean of 3.6 permitted values, and one citation-check verdict schema expresses exactly 160 distinct machine-readable states.
Read the full article on StewAI