benchwiki
agentic-tool-use

BFCL / Berkeley Function Calling Leaderboard

Active
2024-02-26 → today

A live UC Berkeley leaderboard for function and tool calling across AST, executable, multi-turn, live, and agentic categories.

last updated 2026-08-30

Performance over time

vendor-reported independent
45556575852025

Baseline: No single human baseline is published for the full BFCL V4 weighted score.

Overview

BFCL, the Berkeley Function Calling Leaderboard, evaluates whether an LLM can choose and format function calls correctly. It started as a tool-calling benchmark for single-turn calls and grew into a broader agentic evaluation with multi-turn state, live executable tasks, hallucination checks, web-search tasks, memory backends, and format-sensitivity tests.

Its main contribution is not just the leaderboard. BFCL made function-calling evaluation more reproducible by defining callable function documents, parsing model outputs into AST-like representations, and separating cases where execution is needed from cases where structured equivalence is enough.

Task details

A BFCL item provides a user query and one or more candidate functions. The model must either return the correct function call with valid arguments, call several functions in the correct structure, or abstain when no relevant function exists. V4 adds agentic tasks such as web search and memory management where tool calls must be sequenced across stateful interactions.

The benchmark distinguishes native function-calling models from prompt-mode models. In prompt mode, the model is asked to emit a structured textual representation of the function call, which makes syntax compliance part of the task.

Example

A simple item might give a weather function with a JSON schema and ask for the forecast in a named city. A multi-function item might include several APIs where only one is relevant. A V4 memory item can require writing user preferences to a memory tool and later retrieving them to answer consistently in a later turn.

Scoring methodology

The current V4 overall score is a weighted composite. The official score composition gives 40% weight to Agentic, 30% to Multi-Turn, 10% to Live, 10% to Non-Live, and 10% to Hallucination Measurement categories. Format sensitivity is reported separately and does not contribute to the same overall score.

AST scoring checks whether the function name and arguments match the reference structure. Executable categories run the call or compare live outputs. Because this mixes scoring modes, the headline BFCL number should always name the version and category set.

The timeline dates are the model launch dates, not BFCL leaderboard update dates. All plotted model launch dates are after BFCL’s 2024-02-26 first release.

What it doesn’t measure

BFCL is about invoking tools correctly, not completing arbitrary open-ended work. A high score does not mean the agent can safely operate real accounts, debug failures outside the provided tools, understand business context, or recover from long-horizon project drift.

Known issues & criticisms

  • Prompt format is part of the score. BFCL V4 explicitly shows that return format, tool-call tags, and function-document format can change results.
  • Native and prompt modes differ. A model with native tool calling and a prompted text model are not always comparable, even on the same test item.
  • Public data creates leakage risk. BFCL is useful partly because it is reproducible, but public cases can be memorized or overfit.
  • Composite weights are policy choices. The V4 overall score emphasizes agentic and multi-turn categories, so category-level scores are needed for diagnosis.

Contamination notes

Risk is medium. The benchmark is public and widely used, but BFCL mitigates some staleness through live data, versioned releases, pinned code, category updates, and transparent model-response dumps. Treat any single number as a dated leaderboard snapshot.

How to run it

Install or pin the official package/version used for the leaderboard snapshot:

pip install bfcl-eval==2025.12.17
bfcl generate --model MODEL_NAME --test-category TEST_CATEGORY
bfcl evaluate --model MODEL_NAME --test-category TEST_CATEGORY

For leaderboard-comparable results, use the official model name, category list, native function-calling or prompt mode, and pinned commit reported on the BFCL site.

History & lineage

BFCL launched on 2024-02-26. V2 added enterprise and open-source contributed live data, V3 added multi-turn and multi-step scenarios, and V4 moved the benchmark toward agentic evaluation with web search, memory, and format sensitivity.

The ICML 2025 paper frames BFCL as a scalable function-calling benchmark and notes that modern models are much stronger on single-turn calls than on memory, dynamic decision-making, and long-horizon reasoning.

Further reading