A GraphQL-Native Embedding Model for AI Agents

When you point an AI agent at a GraphQL API, the hard part isn’t writing the query. It’s finding the right fields in the first place. Real schemas are wide. A typical production schema carries thousands of Type.field coordinates, and most of them won’t fit in a model’s context window at once. The agent needs to retrieve the handful of fields that answer the question, and only then write a query against them.

This is a classic retrieval problem, and the usual answer is RAG: embed every coordinate or type, as well as the user’s question, and pull the top matches. It works well for documentation. It works less well for GraphQL, and the reason is specific to how schemas are designed.

Why general-purpose embedders struggle with schemas

Schemas reuse field names everywhere. The same author, state, assignee, createdAt, or title tends to appear on many different types, so knowing the field name is rarely enough on its own — you have to know whose field it is.

Consider a question like “Who is assigned to my support ticket?” against a schema like this:

type Ticket   { assignee: User }
type Incident { assignee: User }
type Task     { assignee: User }

The right answer is Ticket.assignee, but every candidate exposes the same field. The retriever has to match both parts of the request: the user wants an assignee, specifically on a Ticket. At scale, hundreds of types can share a common field name, so choosing the right owner type becomes a core part of schema retrieval. That is the task this fine-tune targets.

A small, focused fine-tune

A natural experiment is to fine-tune a general embedder on this specific task: mapping a natural-language question to the Type.field coordinate that answers it, with an emphasis on disambiguating between same-named fields on different owner types. The artifact discussed in this post is Qwen3-Embedding-0.6B-GraphQL, an open-source (Apache 2.0) fine-tune of Qwen3-Embedding-0.6B. It’s an early prototype shared here as a reference point for schema-aware retrieval, and the methodology generalizes to any base embedder.

At 0.6B parameters, the model runs on CPU, or comfortably alongside an agent’s own model on the same GPU. The weights are published as SentenceTransformers and as GGUF builds for llama.cpp and Ollama.

The most realistic evaluation is across multiple production schemas the model has never seen during training, with a hosted baseline alongside it. The GraphQL AI Working Group benchmark covers five schemas — GitHub, GitLab, Linear, Shopify, and the Singapore Open Data API — and 816 natural-language queries, each labelled with the Type.field that answers it. Headline field recall@K, against the ground-truth coordinate:

The fine-tune leads at every cutoff and edges out text-embedding-3-large while running locally at a third of its embedding dimension. The un-tuned base model already sits between OpenAI’s two tiers, so the architecture is competitive on its own, and the fine-tune adds another ~5 percentage points of field recall on top.

Type-level recall is a parallel metric: instead of matching the full Type.field, it asks whether the right owner type shows up in the top results. There text-embedding-3-large leads at 87.8% recall@50, with the fine-tune close behind at 85.6%, ahead of the base (78.6%) and text-embedding-3-small (80.8%).

A concrete example

“When did I place this order?”

Against the three bare coordinates Order.createdAt, Order.completedAt, and Order.updatedAt, the un-tuned base model ranks Order.completedAt first, while the fine-tuned model ranks the correct coordinate, Order.createdAt, first.

Every candidate is a plausible timestamp on the same order, but “placed” refers to when the order was created, not when it was completed or last updated. The fine-tune makes that distinction and returns Order.createdAt first. This is a small illustration; the aggregate benchmark above is the broader evaluation.

Using it

The model is a drop-in for any GraphQL-aware retrieval, query builder, or schema search. The snippet below is a minimal sanity check using bare Type.field coordinates. It runs entirely on the local machine, with no API calls and no network round trip:

from sentence_transformers import SentenceTransformer
 
model = SentenceTransformer("xthor/Qwen3-Embedding-0.6B-GraphQL")
 
query = "When did I place this order?"
coords = [
    "Order.createdAt",
    "Order.completedAt",
    "Order.updatedAt",
]
 
q = model.encode(query, prompt_name="query")
c = model.encode(coords, prompt_name="document")
scores = (q @ c.T).tolist()
 
for score, coord in sorted(zip(scores, coords), reverse=True):
    print(f"{score:.3f}  {coord}")

The fine-tuned model ranks Order.createdAt first in this three-candidate example; the un-tuned base model ranks Order.completedAt first. For a production index, use the richer SDL or one-line-gloss representations described below rather than bare coordinates alone.

No hosted API is involved. The Q8 GGUF weights are about 650 MB on disk and use roughly 1 to 1.5 GB of RAM at runtime. The Q4 quantization fits in about 400 MB.

The size was a deliberate constraint. A 0.6B embedder fits where a hosted API or a multi-billion-parameter model doesn’t:

• Inside your GraphQL gateway or BFF, so the same process that resolves the schema also indexes it.
• On a developer laptop, fully offline, for local agent loops and CI checks.
• On edge runtimes like a long-running container at the edge, an on-prem box, or a sidecar to your existing API service. Each query takes tens to low-hundreds of milliseconds on CPU, so a small instance with no GPU handles realistic agent traffic.
• Alongside your agent’s main model on the same GPU, where it adds well under a gigabyte of VRAM.

For local serving, model-q8_0.gguf runs near-losslessly on Ollama or llama-server and exposes an OpenAI-compatible embeddings endpoint, so the same code that talks to a hosted provider can talk to it instead.

How you format the corpus matters as much as the model

One finding from the work is worth surfacing on its own. How you render each coordinate to text before embedding it has roughly the same impact on retrieval as the fine-tune itself, and the two stack.

On the GitHub schema benchmark, embedding raw Type.field identifiers like PullRequest.baseRefName gives an MRR of 0.39 with the tuned model. Switching to a short SDL snippet, or to a one-line gloss like “PullRequest.baseRefName: the base ref name of a pull request”, raises that to 0.72. The base model gets a similar bump from the same change. The fine-tune then adds another ~0.2 MRR on top of whichever format you pick.

SDL and a one-line gloss tie at the top. Raw identifiers and ablations that drop either the owner type or the field name fall off sharply.

The lesson generalizes: if you’re building schema retrieval for an agent, spend as much time on how you render each coordinate as on which embedder you pick. The owner type and a short human-readable label belong in the embedded text. Bare identifiers throw away most of the signal, and no embedder can fully recover from that.

Why this matters for the ecosystem

GraphQL’s introspection has always been one of its strongest features. Every schema is self-describing, and tools can walk it programmatically. As AI agents become a meaningful consumer of GraphQL APIs, that self-describing surface becomes the substrate they navigate to do useful work. Making schemas legible to retrieval systems is part of making GraphQL legible to agents.

A focused, small embedding model is one piece of that. There’s plenty more to do: within-owner field disambiguation, multilingual queries, very long schemas with deep nesting, schemas with custom directives that carry semantic weight. But the first step is recognizing that schema retrieval is a distinct problem from prose retrieval, and that purpose-built tooling helps.

If you’re building agents against GraphQL and have schema retrieval pain, give it a try, and please share what you find. The model, training data, and benchmarks are all open:

• Model: huggingface.co/xthor/Qwen3-Embedding-0.6B-GraphQL
• Training code and benchmarks: github.com/ThoreKoritzius/graphql-embedding-model
• Base model: Qwen3-Embedding-0.6B

Feedback, schemas to benchmark against, and PRs are all welcome.