Embedded Key-Value Store

Savant pipeline provides an embedded key-value store available through:

  • ABI for internal access (Python or Rust code);

  • REST API for external access.

REST API is intended for 3rd-party components and downstream elements to exchange data with the pipeline. As the API operates over HTTP and network it may introduce latency and should be used only for non-blocking operations.

The pipeline components which use KVS must utilize ABI to avoid the network overhead and communicate with it in a deterministic and no-latency way.

The key-value store does not replace such solutions as Redis and is intended for small data exchange between pipeline components. Storing several hundreds or even several thousand of keys is just fine.

However, the amount actually is determined by the way one uses the KVS: the more glob search operations are performed, the less keys can be stored. When, KVS is used to operate only on keys with exact names, the amount of keys can be almost arbitrary.

KVS has the following properties:

  • every key consists of (namespace, key) pair;

  • every key has a value represented by savant_rs.primitives.Attribute;

  • an entry can have expiration TTL or be stored indefinitely;

  • user can operate with exact keys or use glob search patterns;

  • external API works on protocol buffers. Those, who use non-Rust/Python languages, can generate the client code from the protocol buffer definition.

Examples of Python-based API/ABI can be found here.

Examples of Rust-based API/ABI can be found in savant-rs test suite:

Endpoints

Setting Keys

POST /kvs/set
Form Parameters:

  • body – list of savant_rs.primitives.Attribute serialized to protocol buffer AttributeSet message

Status Codes:

  • 200 OK – when the keys are set

  • 400 Bad Request – when the keys are not set due to an deserialization error

Setting Keys with TTL

POST /kvs/set-with-ttl/(int: ttl)
Parameters:

  • ttl – time to live in milliseconds (non-negative)

Form Parameters:

  • body – list of savant_rs.primitives.Attribute serialized to protocol buffer AttributeSet message

Status Codes:

  • 200 OK – when the keys are set

  • 400 Bad Request – when the keys are not set due to an deserialization error

Deleting Keys by Glob

POST /kvs/delete/(str: namespace)/(str: name)
Parameters:

  • namespace – glob pattern to match the namespace

  • name – glob pattern to match the name

Status Codes:

  • 200 OK – if the request is correct

Deleting Keys by Exact Match

POST /kvs/delete-single/(str: namespace)/(str: name)
Parameters:

  • namespace – exact namespace

  • name – exact name

Status Codes:

  • 200 OK – if the request is correct

Getting Keys Names by Glob

GET /kvs/search-keys/(str: namespace)/(str: name)
Parameters:

  • namespace – glob pattern to match the namespace

  • name – glob pattern to match the name

Status Codes:

  • 200 OK – if the request is correct

Response

HTTP/1.1 200 OK
Content-Type: application/json

[
    ["namespace1", "key1"],
    ["namespace2", "key2"],
    ["namespace3", "key3"]
]

Getting Attributes by Glob

GET /kvs/search/(str: namespace)/(str: name)
Parameters:

  • namespace – glob pattern to match the namespace

  • name – glob pattern to match the name

Status Codes:

  • 200 OK – if the request is correct

Response

HTTP/1.1 200 OK

serialized AttributeSet

Getting Attributes by Exact Match

GET /kvs/get/(str: namespace)/(str: name)
Parameters:

  • namespace – exact namespace

  • name – exact name

Status Codes:

  • 200 OK – if the request is correct

Response

HTTP/1.1 200 OK

serialized AttributeSet