Agent & Tool Calling

agents give LLMs external capabilities through tool calling, built on three mechanisms: (1) Function-calling schema — declaring each API as name + description + parameters (JSON Schema); the model emits a structured tool_call (name + JSON args), and a tool registry with parameter validation (required fields, enums, types) keeps calls legal; (2) grammar-constrained decoding — JSON mode / Outlines / guidance restrict sampling to tokens consistent with the schema, guaranteeing parseable output by only allowing candidates that satisfy $\text{schema}(\text{prefix})$; (3) ReAct loop — Thought → Action → Observation iteration: $s_t \xrightarrow{\text{LLM}} (\text{Thought}_t, \text{Action}_t) \xrightarrow{\text{exec}} o_t \to s_{t+1}$, feeding each observation back into the context, plus self-correction (reflecting on failures, retrying with other tools or parameters).

multi-tool orchestration (search, calculator, database, code execution); multi-step tasks (booking, data-analysis pipelines); defending against tool-argument hallucination (injecting JSON Schema + examples into the prompt); a favorite interview question — how to guarantee legal, retryable tool calls.

a bare LLM can't reach real-time data or execute computations, and free-form text can't reliably wire into APIs. Schema plus constrained decoding turns "possibly malformed strings" into "guaranteed-parseable structures", and the ReAct loop turns single-shot generation into a closed reason-act loop where errors are observable and correctable.

Walk the function-calling pipeline (schema → output → validation → execution → feedback) and the failure handling at each step.

How does grammar-constrained decoding work, and how does JSON mode differ from normal decoding in guaranteeing parseability?

How does the ReAct loop terminate — max steps and loop detection?

How do agents self-correct — retry vs switch tools on timeout, bad args, or empty results?

With many tools, how do you prevent tool-selection hallucination — tool descriptions, routing, and result verification?