Skip to content
Fallen-8

Semantic traversal

Semantic traversal lets the path and subgraph engines make decisions by vector similarity to a query instead of (or alongside) labels and properties. The query is embedded once, before traversal starts; each candidate element is then scored against its own stored embedding. Because the query rides a pure-data semantic block, similarity-driven traversal is how Fallen-8 answers “no query language”: it runs code-free (it compiles no C#; that philosophy lives in delegates.md). Every graph-scoped route here also answers under /ns/{ns}/… for a specific graph (namespaces.md); the chat gateway (POST /chat) and the configuration view (GET /config) are Fallen-8-level and have no namespace twin.

flowchart LR
    QT["queryText"] -->|"provider embeds once"| QV["query vector"]
    QVIN["queryVector"] --> QV
    QV --> DEC["similarity filter / cost<br/>per element, DURING traversal"]
    EMB["element embedding<br/>(TryGetEmbedding)"] --> DEC
    DEC --> OUT["paths / subgraph membership"]

Three rules define the whole feature:

  1. Embedded once, up front. Supply queryVector (client-side) or queryText (the server embeds it via the provider). The traversal never sees text and never calls a model, per element it is one SIMD similarity computation.
  2. Elements score through their stored embedding. Each candidate is read via TryGetEmbedding for the block’s embeddingName and scored with the block’s metric. A missing embedding, a wrong dimension, or a non-finite score never matches (stated, not silent).
  3. Scores are the kNN scores. The same VectorMath backs both, so a traversal score is bit-identical to /scan/index/vector. Traversal needs no vector index, scoring reads the element directly; a bound index (below) is only for kNN search.

Element embeddings

Section titled “Element embeddings”

A named embedding is a float[] stored on the graph element behind one accessor (AGraphElementModel.TryGetEmbedding). It is WAL-durable element state (save-games.md): one current vector per (element, name); a write replaces, DELETE removes.

Route Body / result Notes
PUT /graphelement/{id}/embedding/{name} { "vector": [...] } → 202 Replace semantics; ?waitForCompletion=true commits before responding
GET /graphelement/{id}/embedding/{name} → { "name", "vector", "model" } model is the provider stamp, null for bring-your-own
DELETE /graphelement/{id}/embedding/{name} → 202 Removing an absent embedding is a committed no-op
  • Name grammar ^[A-Za-z0-9_-]{1,64}$; default default. Different names may hold different dimensions.
  • Vector: finite components, dimension in [1, 4096]. 400 on an invalid name, an empty/oversized/ non-finite vector, or a dimension/zero-norm that a bound index of that name would reject; 404 for an unknown element.
  • Physically the vector sits on the reserved property key $embedding:<name>, the provider stamp on $embeddingModel:<name>. Element reads return both in the ordinary properties list (GET /vertex/{id}, GET /edge/{id}, GET /graphelement/{id}, GET /graph), so an embedded element’s payload inlines the whole vector: 1024 floats with the compose default.
  • On the JSONL wire the embedding travels as its reserved typed property: see bulk-import-export.md.
curl -X PUT "http://localhost:8080/graphelement/42/embedding/default?waitForCompletion=true" \
  -H "Content-Type: application/json" -d '{ "vector": [0.12, -0.5, 0.33] }'

Bound vector indices

Section titled “Bound vector indices”

A VectorIndex created with the embeddingName option (vector-search.md owns the kNN mechanics) becomes a derived projection of element state:

  • Membership = every live element carrying that named embedding at the index’s dimension, maintained on the writer thread for every embedding write.
  • No explicit adds: PUT /index/vector/{id} answers 400; write the element embedding instead.
  • No explicit removals either: the generic index-content removals (DELETE /index/{indexId}/{graphElementId}, DELETE /index/{indexId}/propertyValue) answer 400 on a bound index, for the same reason as the add. Removing the element’s embedding is the way to drop it from the projection, which keeps membership under one authority.
  • Checkpoints persist only the header; load rebuilds the slab from element state, and WAL replay re-projects replayed writes: a bound index is always correct after a crash, with no re-add workaround.
  • Query it exactly like any vector index; it is indistinguishable at query time. An optional model creation option stores an opaque model-identity string for the provider’s consistency checks (below).

Unbound indices (no embeddingName) are unchanged: explicit adds, snapshot-persisted vectors, no element coupling. An element that is both embedded and bound-indexed stores the vector twice (roughly 2x memory).

The semantic block

Section titled “The semantic block”

Carried by POST /path/{from}/to/{to} and PUT /subgraph as the semantic field:

Field Type Meaning
queryVector float[] Query vector (exactly one of vector/text)
queryText string Text the provider embeds once (403 when the provider is off)
embeddingName string Which named embedding to score (default default)
metric string Cosine (default), DotProduct, or L2
minScore number Installs a vertex filter: pass at >= (Cosine/DotProduct) or <= (L2) this score
costBySimilarity bool Path only: installs a vertex cost, Cosine→1-score, L2→distance; without minScore it also fills the vertex-filter slot with an implied has-embedding filter (a cost is only defined over embedded vertices)

minScore and costBySimilarity are declarative, pure data, they compile no C# at all. costBySimilarity needs a weighted algorithm (DIJKSTRA; the hop-count BLS ignores costs) and rejects DotProduct (no honest non-negative mapping). Compiled C# fragments can instead read the query off the context parameter (context.TrySimilarity(element, out score)); see delegates.md.

One owner per delegate slot. Common 400s (all carry a reason):

Request Answer
minScore (or costBySimilarity without minScore) + a vertex-filter fragment (inline or stored) 400
costBySimilarity + a vertex-cost fragment, metric: DotProduct, or on PUT /subgraph 400
queryVector and queryText together 400
empty/non-finite vector, zero-norm under Cosine, bad name/metric 400
queryText with the provider disabled 403 
# Only vertices semantically close to the query may lie on the path:
curl -X POST http://localhost:8080/path/1/to/9 \
  -H "Content-Type: application/json" \
  -d '{ "semantic": { "queryVector": [0.1, 0.2], "minScore": 0.7 } }'

On a subgraph

Section titled “On a subgraph”

PUT /subgraph re-evaluates its filters on every POST /subgraph/{name}/recalculate, so the semantic block binds once, at registration: the vector is captured, recalculation reuses it, and no inference ever runs on the writer thread. With queryText, the resolved vector persists in the subgraph recipe, so a semantic subgraph survives restart and WAL replay without the provider present (subgraphs.md).

  • semantic.minScore fills the top-level vertex pre-filter slot.
  • A vertex pattern step’s semanticMinScore fills that step’s filter slot, so a fully declarative pattern (“near-query vertex -edge-> near-query vertex”) compiles no C#. Setting it together with the step’s vertexFilter, on an edge step, non-finite, or without a request-level semantic block is a 400. costBySimilarity is rejected (a path concept).
  • Stored templates carry no semantics: a semantic block on a stored-template invocation is a 400 (stored-queries.md), and semanticMinScore inside a stored SubGraph template is rejected already at registration (POST /storedquery), because a template’s delegates bind before any query vector exists. Inline the filters instead.

The embedding provider (text-in)

Section titled “The embedding provider (text-in)”

queryText, POST /embedding/element, and POST /embedding/search need an embedding provider. It is optional, capability-gated, and lives in the API app only: the engine (fallen-8-core) never loads a model, so a bare dotnet run is model-free and the provider answers 403. It is exposed through Microsoft.Extensions.AI’s IEmbeddingGenerator; the model loads lazily on first use, and a failed load latches (503) rather than retry-storming.

Configure it under Fallen8:Embedding (off unless Enabled):

Key Default Meaning
Enabled false Master switch; endpoints answer 403 when off
Backend Onnx Onnx | LLamaSharp | Ollama | Nahil | OpenAI: the whole backend swap, and which to pick
ModelName / ModelVersion (none) Identity name (required when enabled) + optional version
Dimension (none) Declared output length; a mismatch against real output latches 503
IntendedMetric Cosine Metric the vectors are meant for
MaxBatchSize / MaxTextLength 64 / 8192 Request bounds, in items and chars. MaxTextLength is not a token window - see below
MaxConcurrentBatches 1 How many MaxBatchSize requests document ingestion keeps in flight at once for one document; raise to 2, then 4 or 8, only while p95 latency and the failure rate stay acceptable
QueryPrefix "" Retrieval prefix applied to query-time embeddings only
Onnx.{ModelPath,VocabPath,…}, LLamaSharp.ModelPath, Ollama.{Endpoint,Model}, Nahil.{…}, OpenAI.{Endpoint,ApiKey,Model} (none) Backend-specific settings

The provider’s biggest consumer is unstructured ingestion: document chunks embed through it on the way into the graph.

The input ceiling (2048 tokens, not 8192)

Section titled “The input ceiling (2048 tokens, not 8192)”

bge-m3 reports an 8192-token context and nothing serves it. Measured: an input of about 1,880 tokens answers 200, one of about 2,120 answers 400. The real ceiling is 2048 tokens per input, and it is the same on the local Ollama sidecar and on Nahil - so this is a property of the backend, not of running the model remotely.

What makes it worth knowing is what used to happen above the line. The Ollama embedding API takes a truncate flag that defaults to true, and what it really means is shorten anything that does not fit and answer as though it had fitted. An over-long chunk came back as a perfectly ordinary 1024-dimension vector describing only its first ~2,046 tokens. Nothing marked it: not the response, not the dimension check, not a log line. The chunk was indexed, searchable, and quietly wrong about its own tail - the failure mode where retrieval gets worse and no error ever appears.

Two things stop it, and they only work as a pair:

  • Fallen-8 sends truncate: false on every embedding request. An input over the ceiling is refused, with a 503 naming the ceiling and which setting to lower. A failed ingest is re-runnable; a silently truncated vector is not even visible.
  • Fallen8:Ingestion:ChunkMaxChars defaults to 3,600 (it was 4,000, derived from the advertised 8192). That is a token budget wearing a char unit: it holds a chunk under ~1,800 tokens at 2.0 chars/token, the worst case measured over markdown tables (2.10), ARXML (2.23) and punctuation-dense text (2.04).

Without the second, the first would turn silent degradation into failed ingests - which is why the default moved and not just the flag.

To be exact about the old value rather than alarmed about it: 4,000 was inside the ceiling for every Latin-script, table and XML sample measured - but by under 70 tokens at the densest of them (~1,980 of 2,048), which is no margin at all for a denser page - and already outside it for Korean (~2,400) and Chinese (~3,010). 3,600 buys ~250 tokens of margin and costs prose nothing.

Measured bge-m3 density, for setting the bound yourself:

Text chars/token
Latin prose (English 4.01, German 4.11), Russian 3.98, Arabic 3.56 ~4
C#-style identifiers 3.41
ARXML 2.23, markdown tables 2.10, punctuation-dense 2.04, Japanese 2.02 ~2
Korean 1.67, Chinese 1.33 ~1.5

So a CJK corpus wants ChunkMaxChars around 1,800. Left at 3,600 it fails the ingest loudly, which is the point, but it fails.

MaxTextLength (8192 chars) is deliberately not this ceiling, though it is the same number and that is where it came from. Read it as the bound above which an input cannot fit 2048 tokens even at the most token-efficient text there is (~4.1 chars/token): it rejects the hopeless early with a 400. Everything under it is the backend’s call, and the backend is now honest about it.

One case the chunker does not cap: a single table row longer than ChunkMaxChars is still emitted whole, because a row-window always carries at least one body row and the alternative is cutting a row in half. That too is now a loud failure rather than a silent one.

The chat gateway (its sibling)

Section titled “The chat gateway (its sibling)”

The embedding provider has a sibling under the same “semantic gateway” idea: a chat gateway, POST /chat, that proxies a chat completion to the same model backend so a client reaches an SLM/LLM through the instance. It is the default NL-assist transport in F8 Studio, also apiApp-only and capability-gated, and the model is server-owned. Configure it under Fallen8:Chat (off unless Enabled / F8_CHAT): Backend (Ollama, Nahil, OpenAI or Anthropic, see model providers), Ollama.{Endpoint,Model} (default phi4-f8-mini:latest), TimeoutSeconds (the single deadline on a completion; exceeded calls answer 504). Both providers’ state, plus whether the backend runs on GPU when Ollama reports it, is shown on Studio’s Connect Configuration card (GET /config), and their keys sit in the Embedding provider and Chat sections of the configuration surface behind it. The values that identify a model are deliberately not editable in either place: the model name, version, dimension and metric are stamped beside every vector you have already stored, so changing one would mislabel that data rather than fail.

The embedding provider has the same knob, Fallen8:Embedding:TimeoutSeconds, defaulting higher because one call embeds a batch of texts; an exceeded batch answers 503, like any other “backend not usable right now”. On a CPU-only host both defaults are generous and still may not be enough: see NL assist spins on “generating” for what CPU inference actually costs.

Weights are never downloaded by Fallen-8: paths point at operator-provided files.

Backend In-process Weights Note
Onnx yes (CPU) ONNX export + WordPiece vocab self-contained; the bge family is the tested reference
LLamaSharp yes (CPU) embedding-capable GGUF reuses a local Ollama blob; not bit-identical to the daemon
Ollama no a model you ollama pull zero in-process memory; couples availability to the container (503 while down)
Nahil no the same bge-m3, on nahil.dev no weights on the host, and no vector re-embeds (Nahil)
OpenAI no none of yours a different embedding function: new stamp, new dimension, so it never moves with a chat switch (model providers)

In the compose environment the provider is on by default: the shipped Ollama sidecar serves bge-m3 (MIT, 1024-dim, Cosine) and the fallen8 service is wired to it; opt out with F8_EMBEDDINGS=false (running.md).

Endpoints (all 403 while disabled; 413 over 1 MiB; 429 on the sensitive-endpoint rate limit; 502 for bad backend output; 503 when the backend is unavailable):

Endpoint Purpose
POST /embedding/element, /embedding/elements Embed text -> the element’s named embedding (one atomic transaction; a bound index projects)
POST /embedding/search Embed a query once -> exact kNN against a vector index (scores identical to /scan/index/vector)
POST /embedding/text Raw text -> vectors, for client-side pipelines
semantic.queryText Text-in path/subgraph traversal (above)

Model-identity contract (hard 409, never coercion). Every provider write stamps the vector with name[@version]#dimension#metric (e.g. bge-micro-v2#384#Cosine); a bring-your-own overwrite clears the stamp. Embedding into, or searching, an index whose dimension differs from the provider’s (or whose declared model differs from the provider’s stamp) is a 409 before any write. A model change is therefore an external re-index (new index, re-embed): the provider’s declared name, dimension, and metric must match the stored vectors.

Clients read provider state from GET /status -> embedding: { enabled, backend, modelName, modelVersion, dimension, intendedMetric, loaded } (a config read that never triggers the lazy load; observability.md). F8 Studio gates its text-in controls on it (studio.md).

# Text-in semantic path query (provider on): embed once, then filter the path by similarity:
curl -X POST http://localhost:8080/path/1/to/9 \
  -H "Content-Type: application/json" \
  -d '{ "semantic": { "queryText": "red bicycles", "minScore": 0.7 } }'

See also

Section titled “See also”