Skip to content
Fallen-8

Indexes

An index is an optional acceleration structure that maps keys to graph elements so a lookup does not have to walk the graph. Indexes are a complement to Fallen-8’s primary query path: C# delegate predicates (delegates.md) and full-graph scans (graph-model.md), not a replacement for it. Each index is a plugin keyed by a uniqueId; the built-in types are discovered through the plugin system (plugins.md), and third-party index plugins are picked up the same way. Every route below also answers under /ns/{ns}/… for a specific namespace (namespaces.md); each namespace owns its own set of indexes.

Built-in index types

Section titled “Built-in index types”

The pluginType is the exact string POST /index expects. equality is declared by the index itself through IIndex.SupportsPointEqualityLookup; range, fulltext, spatial and vector come from the family interface the index implements. Every capability is reported per index by GET /status.

pluginType Indexes Query capability Scan endpoint(s)
DictionaryIndex any comparable key → elements (multi-value buckets) equality POST /scan/index/all
SingleValueIndex any comparable key → exactly one element (a re-add under the same key replaces it) equality POST /scan/index/all
RangeIndex a single totally-ordered comparable key type equality, range POST /scan/index/all, POST /scan/index/range
RegExIndex string keys, matched by regular expression equality, fulltext POST /scan/index/all, POST /scan/index/fulltext
SpatialIndex geometry (R-Tree / R*-tree) spatial POST /scan/index/spatial
VectorIndex float[] embedding vectors (exact kNN) vector POST /scan/index/vector

Notes:

  • RangeIndex requires its keys to form a total order (one comparable type; mixed types or Double.NaN have undefined ordering and can throw while sorting).
  • RegExIndex keys are strings. Its POST /scan/index/fulltext path treats the query as a case-insensitive .NET regular expression; its equality path (POST /scan/index/all) is an exact, case-sensitive key lookup, so a key indexed as "Alice" is not found by "alice".
  • SingleValueIndex overwrites instead of bucketing, so its keys and values counts in the GET /status inventory are always equal. Use DictionaryIndex when several elements share a key.
  • SpatialIndex and VectorIndex do not report equality: their keys (geometry, float[]) cannot travel as a scan endpoint’s typed literal.

Creation options

Section titled “Creation options”

POST /index takes pluginOptions as a map of option name → { "propertyValue", "fullQualifiedTypeName" }, where the value is parsed via a closed primitive allow-list (string, the integer/float types, bool, DateTime, Guid, …).

  • DictionaryIndex, SingleValueIndex, RangeIndex, RegExIndex take no options: they ignore pluginOptions entirely and are populated per element (see below).
  • VectorIndex takes dimension (required), metric, embeddingName, model: see vector-search.md.
  • SpatialIndex is not creatable over REST: it needs .NET-object options (a metric, a dimension list) that the primitive allow-list cannot express, so POST /index returns false for it. It is created in-process against the engine.

REST surface

Section titled “REST surface”
Action Route Body Returns
Create an index POST /index PluginSpecification (uniqueId, pluginType, pluginOptions) true/false
Add an element under a key PUT /index/{indexId} IndexAddToSpecification (graphElementId, key) true/false
Add a vector (vector family) PUT /index/vector/{indexId} see vector-search.md true, or 400/404
Remove one element from an index DELETE /index/{indexId}/{graphElementId} (none) true/false
Remove a key DELETE /index/{indexId}/propertyValue PropertySpecification (the key) true/false
Delete an index DELETE /index/{indexId} (none) true/false
Rebuild an index from element state POST /index/backfill/{indexId} IndexBackfillSpecification (propertyId, plus optional replace, prefix, label) an outcome object (see below), or 400
List indexes GET /status (none) inventory (id, type, capabilities, key/value counts)

A miss on these routes is not a 404: except for the vector add and the backfill, every row answers 200 with a false body when the index or the graph element does not exist, when uniqueId is already taken, or when pluginType names an unknown plugin. The reason only reaches the server log, so check the boolean. A 400 means the body was missing or malformed. Two rows are the exception and use real codes. PUT /index/vector/{indexId}: 404 for an unknown index or element, 400 for a non-vector index, a wrong dimension, a non-finite or zero-norm-under-Cosine vector, or a bound index that refuses explicit adds. POST /index/backfill/{indexId}: 400, with the reason in the response, for an unknown index, a missing propertyId, or an index it refuses (below).

Rebuilding an index you already have

Section titled “Rebuilding an index you already have”

An index is derived state: everything in it can be recomputed from the elements’ own properties. So when one goes missing or comes back empty - a checkpoint that could not write it (watch lastCheckpointDroppedIndices on GET /status), a truncated recovery, or an index created after the elements were already there - POST /index/backfill/{indexId} walks the live elements and writes an entry for every one carrying the named property. Give it a single property key, or a prefix to catch a family of keys written with an ordinal suffix.

The default is a repair and is safe to run on every start: it is add-only, and adding the same (key, element) pair twice does nothing, so a backfill over an index that is already complete writes nothing new and nothing is ever briefly missing. What it will not do is remove a key that element state no longer justifies. Pass "replace": true for an exact rebuild instead: the index is wiped first, so stale keys go, at the cost of a window in which a concurrent scan sees an empty index. That asymmetry is why repair is the default: an incomplete index and a briefly empty one fail very differently. "label" narrows the scan to elements carrying that label.

It refuses, with a 400 naming the reason, an index that does not report equality (spatial and vector, whose keys are not arbitrary property values) and a vector index bound to an embedding, which already maintains itself from element state.

The answer is an outcome object rather than a boolean, so a caller can tell a no-op from real work and can spot having named the wrong property (scanned many, indexed none):

Field Means
indexId, propertyId Echo of which index was repopulated, and from which property
replaced Whether the index was wiped first (an exact rebuild rather than a repair)
scannedElements Live elements looked at (every one, or every one with label)
indexedElements Entries written; in prefix mode one element can contribute several
skippedUnindexableValues Elements whose value cannot be an index key (not comparable, for example a vector written through the raw property surface), counted so the skip is not silent

Listing and per-index counts are part of the status surface (observability.md). Index definitions are immutable: there is no update; delete and recreate. Populating a bucket or fulltext index is explicit: created empty, then filled with PUT /index/{indexId}, one element and key at a time. Removing a graph element (through a transaction) purges it from every index automatically. A VectorIndex bound to an embedding name maintains itself, so every explicit content write is refused with 400: the add, and both removals above (DELETE /index/{indexId}/{graphElementId}, DELETE /index/{indexId}/propertyValue). Remove the element’s embedding instead and the projection follows (semantic-traversal.md owns the binding story). HEAD /tabularasa drops every index definition, not just its entries, and loading a checkpoint replaces the live index set with the checkpoint’s, so recreate your indexes afterwards (save-games.md). A Trim keeps them.

POST /index is not the only route that creates one: the semantic layer’s explicit bind POST /document/binding/ensure creates the vector, fulltext and entity indexes ingestion needs (default ids documents, documents-text, documents-entities) if they do not exist yet. They show up in the GET /status inventory like any other index, and deleting one makes ingestion answer 428 until the bind runs again (unstructured-ingestion.md).

Watch the two key shapes: an index add key uses propertyValue (a PropertySpecification), while a scan literal uses value (a LiteralSpecification); both carry fullQualifiedTypeName. POST /scan/index/all also honours the scan body’s optional label field: an exact-match restrictor on the hits, same as the property scan’s.

Worked example: a dictionary index

Section titled “Worked example: a dictionary index”

Create a dictionary index, index element 42 under the key "Alice", then look it up. The example assumes vertex 42 already exists: create one with PUT /vertices?waitForCompletion=true, which returns the assigned ids, and substitute the id you get (graph-model.md). A miss here is silent rather than an error: if the element or the index does not exist, step 2 answers 200 with false and step 3 then returns []. resultType: "Vertices" in step 3 also drops a hit that is an edge; use "Both" to see every hit.

# 1. Create (bucket indexes take no options)
curl -X POST http://localhost:8080/index \
  -H "Content-Type: application/json" \
  -d '{ "uniqueId": "nameIndex", "pluginType": "DictionaryIndex" }'


# 2. Index element 42 under key "Alice"
curl -X PUT http://localhost:8080/index/nameIndex \
  -H "Content-Type: application/json" \
  -d '{ "graphElementId": 42,
        "key": { "propertyValue": "Alice", "fullQualifiedTypeName": "System.String" } }'


# 3. Equality scan -> [42]
curl -X POST http://localhost:8080/scan/index/all \
  -H "Content-Type: application/json" \
  -d '{ "indexId": "nameIndex",
        "operator": 0,
        "literal": { "value": "Alice", "fullQualifiedTypeName": "System.String" },
        "resultType": "Vertices" }'

Scanning an index

Section titled “Scanning an index”

All scan endpoints return the matching graph-element ids (the vector and fulltext scans return a richer object). resultType is a string enum, Vertices, Edges, or Both.

Endpoint Spec fields Use
POST /scan/index/all indexId, operator, literal (value, fullQualifiedTypeName), resultType Equality (any index) and ordered comparisons; a RangeIndex answers ordered ops in O(log n + k), other indexes fall back to an O(n) key scan
POST /scan/index/range indexId, leftLimit, rightLimit, fullQualifiedTypeName, includeLeft, includeRight, resultType Bounded range on a RangeIndex
POST /scan/index/fulltext indexId, requestString Regex search on a RegExIndex; returns matched elements with highlights and a score
POST /scan/index/spatial indexId, graphElementId, distance Elements within distance of a reference element in a SpatialIndex
POST /scan/index/vector indexId, query, k, kind, label Exact kNN: see vector-search.md

The four scans do not report a miss the same way, so do not write one client branch for all of them: all and range return an empty 200 for an unknown index; fulltext and spatial return 204 No Content when the index does not exist or belongs to another family (spatial also when the reference element is gone); vector returns 404 for an unknown index and 400 for a non-vector index or an invalid query (wrong dimension, k outside [1, 1024], non-finite components, zero-norm under Cosine, unknown kind).

operator on POST /scan/index/all is the integer-valued BinaryOperator (it is not a string on the wire):

Value Operator Value Operator
0 Equals 3 Lower
1 Greater 4 LowerOrEquals
2 GreaterOrEquals 5 NotEquals

An index scan hits only the keys you added to that index. To scan every element’s live properties without an index, use POST /scan/graph/property/{propertyId} for one named key with a typed operator (graph-model.md), POST /scan/graph/properties for a case-insensitive contains match across every property value (graph-model.md), or a compiled delegate (delegates.md).

Vector indexes

Section titled “Vector indexes”

VectorIndex is an exact, SIMD brute-force k-nearest-neighbour index over float[] vectors, and its hits are graph elements, so similarity search is a graph entry point. It shares the index surface above (create with pluginType: "VectorIndex", delete, list) but has its own typed add (PUT /index/vector/{indexId}) and query (POST /scan/index/vector). The full contract (metrics, ordering, options, bound embeddings, memory) lives in vector-search.md; embedding-driven population is covered in semantic-traversal.md.

Persistence

Section titled “Persistence”

Every built-in index is persistable (CanPersist == true): its contents are written into checkpoints and restored on load, where the factory recreates each index and rehydrates its entries by graph-element id, indexes are not rebuilt from a property sweep. Neither index creation nor index writes are WAL-logged, so after a crash replay an index created since the last checkpoint is gone entirely, and an older index is back at its checkpointed contents. Save a checkpoint once you have built an index. The exception is a bound vector index that was already in the checkpoint: its contents rebuild from the WAL-covered element embeddings. See save-games.md.

See also

Section titled “See also”