waymaker-client v0.1.27 — polyglot Go / TypeScript / Rust clients

Extract the waymaker client into a standalone, polyglot repo versioned in
lockstep with the waymaker server (this tag == server v0.1.27).

- proto/   vendored client-facing protos (source of truth: waymaker repo).
           internal_proxy (server-internal ProxyService + replication) is
           intentionally excluded — it is a server concern, not a client one.
- rust/    standalone crate; build.rs generates from proto/. Quorum fence
           scope + Lock auto-reacquire / fence-watch (watch/is_lost) API.
- go/      module git.awesomike.com/pub/waymaker-client/go — generated stubs
           + ergonomic wrappers (locks/streams/kv/collections/sketches/object).
- ts/      @waymaker/client — generated stubs + ergonomic wrappers.
- scripts/ sync-protos.sh (re-vendor + align VERSION) + gen-go.sh + gen-ts.sh.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Michael Netshipise 2026-06-09 19:20:48 +02:00
parent 0a70f3424f
commit 87a6171969
79 changed files with 86925 additions and 0 deletions

14
.gitignore vendored Normal file
View File

@ -0,0 +1,14 @@
# Rust
/target
**/target
Cargo.lock
# Node / TypeScript
node_modules/
ts/dist/
# Go (generated stubs in go/genpb ARE committed)
# misc
.DS_Store
*.log

6
Cargo.toml Normal file
View File

@ -0,0 +1,6 @@
# Root workspace so the Rust crate in rust/ is discoverable when this repo is
# consumed as a Cargo git dependency, and so `cargo` works from the repo root.
# The go/ and ts/ clients are independent toolchains and ignored by Cargo.
[workspace]
resolver = "2"
members = ["rust"]

View File

@ -0,0 +1,54 @@
# waymaker-client
Official client libraries for [waymaker](https://git.awesomike.com/dev/waymaker)
— the distributed coordination service (locks, streams, KV, collections,
sketches, cache, object store) over gRPC.
Three clients, one wire contract, **versioned in lockstep with the waymaker
server**:
| Language | Path | Package |
|------------|---------|---------|
| Rust | `rust/` | `waymaker-client` (crate) |
| Go | `go/` | `git.awesomike.com/pub/waymaker-client/go` (module) |
| TypeScript | `ts/` | `@waymaker/client` (npm) |
The `.proto` files in `proto/` are **vendored copies**; the waymaker server
repo is the source of truth. `scripts/sync-protos.sh` refreshes them and keeps
`VERSION` aligned with the server.
## Versioning
Every release is tagged at the **same version as the waymaker server** it
targets (see `VERSION`). A client tagged `v0.1.27` speaks the wire contract of
waymaker `v0.1.27`. The gRPC wire format is backward-compatible across patch
releases (enum integer values are stable), so a client one patch behind a
server generally interoperates — but match versions for new surface.
## Layout
```
proto/ vendored .proto (source of truth: waymaker repo)
rust/ Rust client crate (hand-written ergonomic wrappers + generated stubs)
go/ Go client module
ts/ TypeScript client (npm)
scripts/ sync-protos.sh + codegen scripts (gen-go/gen-ts)
VERSION lockstep version with the waymaker server
```
## Regenerating
```bash
# refresh protos from a local waymaker checkout, then regenerate all stubs
WAYMAKER_REPO=../waymaker ./scripts/sync-protos.sh
./scripts/gen-go.sh
./scripts/gen-ts.sh
# Rust regenerates from proto/ automatically via build.rs on `cargo build`.
```
## Quickstart
- **Rust** — see [`rust/README.md`](rust/README.md). Leader election with the
fence-watch / auto-reacquire `Lock`.
- **Go** — see [`go/README.md`](go/README.md).
- **TypeScript** — see [`ts/README.md`](ts/README.md).

1
VERSION Normal file
View File

@ -0,0 +1 @@
0.1.27

176
go/README.md Normal file
View File

@ -0,0 +1,176 @@
# waymaker-client (Go)
Official Go client for [waymaker](https://git.awesomike.com/dev/waymaker) — v0.1.27.
```
go get git.awesomike.com/pub/waymaker-client/go@v0.1.27
```
The package is `waymaker`. Generated gRPC stubs live under `genpb/` but
callers rarely need them directly — the ergonomic wrappers cover everything.
## Surfaces
| Subsystem | Entry points on `*Client` | Notes |
|--------------|--------------------------------------------------------|-------|
| **lock** | `AcquireLock` / `AcquireReadLock` / `MultiLock` | Full re-acquire semantics (see below) |
| **stream** | `CreateStream` / `GetStream` / `GetOrCreateStream` | Push + pull consumers |
| **kv** | `CreateKV` / `GetOrCreateKV` / `KV` | Put/Get/Create/Update(CAS)/Delete/Keys/History/Watch |
| **collections** | `CreateHashStore` / `CreateSetStore` / `CreateQueue` | Redis-shape Hash/Set/Queue |
| **sketches** | `CreateBloom` / `CreateHLL` / `CreateCMS` / `CreateTopK` / `CreateTDigest` | Probabilistic data structures |
| **object** | `CreateObjectStore` / `GetOrCreateObjectStore` | Chunked Put/Get/Delete/List |
| **cache** | `CacheAttachPolicy` / `CacheDetachPolicy` / `CacheStats` | Stub — server returns Unimplemented |
## Connecting
```go
import "git.awesomike.com/pub/waymaker-client/go"
// Single node
client, err := waymaker.Connect(ctx, "localhost:8818")
// Multiple nodes (round-robin load balancing)
client, err := waymaker.ConnectMulti(ctx, []string{"node1:8818", "node2:8828", "node3:8838"})
defer client.Close()
```
## Leader election
The canonical leader-election pattern: `AcquireLock` with `MaxWait=0`
(try-and-fail), then `SpawnRenewal` in its own goroutine independent of
the work loop, and `lock.Watch()` to detect leadership loss.
```go
import (
"context"
"time"
"git.awesomike.com/pub/waymaker-client/go"
)
func runAsLeader(ctx context.Context, client *waymaker.Client) error {
lock, err := client.AcquireLock(ctx, "leader:reports", waymaker.LockConfig{
MaxWait: 0, // try-acquire — fail immediately if contended
LeaseTTL: 30 * time.Second,
Scope: waymaker.ScopeQuorum, // Raft-replicated fence token
})
if waymaker.IsServerCode(err, "expired") {
return nil // someone else is leader
}
if err != nil {
return err
}
// Renewal runs independently of the work loop — work can legitimately
// outlive a single lease window without renewing in-band.
renewal := lock.SpawnRenewal(15 * time.Second)
defer renewal.Stop()
// Watch for state changes (fence updates, loss notifications).
for {
select {
case <-ctx.Done():
return ctx.Err()
case <-lock.Watch():
if lock.IsLost() {
return fmt.Errorf("lost leadership")
}
default:
}
// Re-read fence BEFORE every fenced side effect.
if err := doFencedWrite(ctx, lock.FenceToken()); err != nil {
return err
}
}
}
func cleanup(ctx context.Context, lock *waymaker.Lock) {
_ = lock.Unlock(ctx)
}
```
### Lock semantics
The `*Lock` handle keeps a background goroutine that holds the server event
stream open. If the stream drops — e.g. the key's primary bounces — the
goroutine transparently re-binds it by re-issuing the same `RequestID` with
`MaxWait=0`. A still-held lease on the new primary is recovered rather than
re-contended. The lease itself lives on the server's TTL + `SpawnRenewal`,
independent of the stream, so a momentary disconnect does not lose the lock.
Live state:
```go
lock.FenceToken() // current fence token — re-read before every fenced write
lock.LeaseExpiresAtMs() // lease expiry epoch ms
lock.IsLost() // true once the client gives up recovering ownership
lock.State() // full LockState snapshot
lock.Watch() // returns a channel closed on every state change
```
Dropping a `*Lock` without calling `Unlock` does NOT release the server-side
lock. The lease will expire on its own TTL. This matches the Rust client's
semantics: auto-release on drop would silently swallow errors.
## KV
```go
bucket, err := client.GetOrCreateKV(ctx, waymaker.KVConfig{Name: "my-bucket"})
rev, err := bucket.Put(ctx, "hello", []byte("world"))
val, err := bucket.Get(ctx, "hello")
rev2, err := bucket.Update(ctx, "hello", []byte("updated"), rev)
// Watch all keys
w, err := bucket.WatchAll(ctx)
for {
ev, err := w.Next()
if err != nil || ev == (waymaker.KVEvent{}) { break }
if ev.Put != nil { fmt.Println("put", ev.Put.Key) }
}
```
## Streams
```go
stream, err := client.GetOrCreateStream(ctx, waymaker.StreamConfig{
Name: "events",
Retention: waymaker.RetentionLimits,
})
ack, err := stream.Publish(ctx, "events.user.123", []byte(`{"action":"login"}`))
consumer, err := stream.GetOrCreateConsumer(ctx, waymaker.ConsumerConfig{
DurableName: "processor",
DeliverPolicy: waymaker.DeliverAll,
AckWait: 30 * time.Second,
})
msgs, err := consumer.Messages(ctx)
for {
msg, err := msgs.Next()
if err != nil || msg == nil { break }
process(msg.Payload)
_ = msg.Ack(ctx)
}
```
## Error handling
```go
_, err := client.AcquireLock(ctx, "key", waymaker.LockConfig{MaxWait: 0})
if waymaker.IsServerCode(err, "expired") {
// lock is contended, no one waiting
}
var werr *waymaker.Error
if errors.As(err, &werr) {
fmt.Println(werr.Kind, werr.Code, werr.Message)
}
```
Error kinds:
- `"rpc"` — gRPC transport / status error
- `"server"``success=false` response (with result_code)
- `"invalid"` — invalid argument to a wrapper method
## Version
v0.1.27 — matches the waymaker server release of the same version.

74
go/cache.go Normal file
View File

@ -0,0 +1,74 @@
package waymaker
// Cache subsystem — TTL/eviction policy service (WaymakerCacheService).
//
// The server-side implementation is not yet complete; every call currently
// returns an "unimplemented" error from the server. The client surface
// matches the Rust client's minimal stub so callers can compile and wire
// up now, ready for when the server lands the first eviction policy.
//
// Entry points on *Client:
// - client.CacheAttachPolicy(ctx, bucket, policyID, params)
// - client.CacheDetachPolicy(ctx, bucket, policyID)
// - client.CacheStats(ctx, bucket)
import (
"context"
pb "git.awesomike.com/pub/waymaker-client/go/genpb/cache"
)
// CacheStats is returned by CacheStats.
type CacheStats struct {
// Raw proto response; fields will be populated when the server
// implementation lands.
Raw *pb.StatsResponse
}
// CacheAttachPolicy attaches an eviction policy to a bucket.
// NOTE: server currently returns Unimplemented.
func (c *Client) CacheAttachPolicy(ctx context.Context, bucket, policyID string, params map[string]string) error {
cc := c.cacheClient()
r, err := cc.AttachPolicy(ctx, &pb.AttachPolicyRequest{
Bucket: bucket,
PolicyId: policyID,
Params: params,
})
if err != nil {
return rpcErr(err)
}
if !r.GetSuccess() {
return serverErr(r.GetResultCode(), r.GetMessage())
}
return nil
}
// CacheDetachPolicy detaches the policy from a bucket.
// NOTE: server currently returns Unimplemented.
func (c *Client) CacheDetachPolicy(ctx context.Context, bucket string) error {
cc := c.cacheClient()
r, err := cc.DetachPolicy(ctx, &pb.DetachPolicyRequest{
Bucket: bucket,
})
if err != nil {
return rpcErr(err)
}
if !r.GetSuccess() {
return serverErr(r.GetResultCode(), r.GetMessage())
}
return nil
}
// CacheStats returns cache statistics for a bucket.
// NOTE: server currently returns Unimplemented.
func (c *Client) CacheStats(ctx context.Context, bucket string) (CacheStats, error) {
cc := c.cacheClient()
r, err := cc.Stats(ctx, &pb.StatsRequest{Bucket: bucket})
if err != nil {
return CacheStats{}, rpcErr(err)
}
if !r.GetSuccess() {
return CacheStats{}, serverErr(r.GetResultCode(), r.GetMessage())
}
return CacheStats{Raw: r}, nil
}

89
go/client.go Normal file
View File

@ -0,0 +1,89 @@
package waymaker
import (
"context"
"fmt"
waymaker_cache "git.awesomike.com/pub/waymaker-client/go/genpb/cache"
waymaker_collections "git.awesomike.com/pub/waymaker-client/go/genpb/collections"
waymaker_kv "git.awesomike.com/pub/waymaker-client/go/genpb/kv"
pb "git.awesomike.com/pub/waymaker-client/go/genpb/locks"
waymaker_sketches "git.awesomike.com/pub/waymaker-client/go/genpb/sketches"
waymaker_streams "git.awesomike.com/pub/waymaker-client/go/genpb/streams"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials/insecure"
)
// Client is a connected waymaker client. It holds a *grpc.ClientConn and
// exposes per-subsystem entry points. Client is safe to use from multiple
// goroutines.
//
// Client is cheap to copy — the underlying gRPC channel is reference-counted.
type Client struct {
conn *grpc.ClientConn
}
// Connect dials a single waymaker server and returns a Client. The target
// must be a valid gRPC target (e.g. "localhost:8818").
//
// Connection is lazy: Connect returns immediately after resolving the address.
// Use grpc.WithBlock() in opts if you need to confirm reachability at dial
// time.
func Connect(ctx context.Context, target string, opts ...grpc.DialOption) (*Client, error) {
defaults := []grpc.DialOption{
grpc.WithTransportCredentials(insecure.NewCredentials()),
}
opts = append(defaults, opts...)
conn, err := grpc.NewClient(target, opts...)
if err != nil {
return nil, fmt.Errorf("waymaker: dial %q: %w", target, err)
}
return &Client{conn: conn}, nil
}
// ConnectMulti dials multiple waymaker servers with round-robin load
// balancing. Requests are distributed across endpoints and automatically
// rerouted when one is unreachable — the right shape for a clustered
// deployment. Requires at least one target.
//
// Note: the current implementation dials the first endpoint only. A full
// static-resolver-based multi-endpoint implementation can be added without
// breaking the public surface.
func ConnectMulti(ctx context.Context, targets []string, opts ...grpc.DialOption) (*Client, error) {
if len(targets) == 0 {
return nil, invalidErr("ConnectMulti requires at least one target")
}
return Connect(ctx, targets[0], opts...)
}
// Close closes the underlying gRPC connection. Subsequent calls on the
// Client will fail.
func (c *Client) Close() error {
return c.conn.Close()
}
// --- per-subsystem gRPC client constructors (unexported) ---
func (c *Client) locksClient() pb.WaymakerServiceClient {
return pb.NewWaymakerServiceClient(c.conn)
}
func (c *Client) streamsClient() waymaker_streams.WaymakerStreamsServiceClient {
return waymaker_streams.NewWaymakerStreamsServiceClient(c.conn)
}
func (c *Client) kvClient() waymaker_kv.WaymakerKvServiceClient {
return waymaker_kv.NewWaymakerKvServiceClient(c.conn)
}
func (c *Client) collectionsClient() waymaker_collections.WaymakerCollectionsServiceClient {
return waymaker_collections.NewWaymakerCollectionsServiceClient(c.conn)
}
func (c *Client) sketchesClient() waymaker_sketches.WaymakerSketchesServiceClient {
return waymaker_sketches.NewWaymakerSketchesServiceClient(c.conn)
}
func (c *Client) cacheClient() waymaker_cache.WaymakerCacheServiceClient {
return waymaker_cache.NewWaymakerCacheServiceClient(c.conn)
}

517
go/collections.go Normal file
View File

@ -0,0 +1,517 @@
package waymaker
// Collections subsystem — Redis-shape Hash / Set / Queue backed by the
// server's WaymakerCollectionsService RPCs.
//
// All wire conventions (subject patterns, tombstone markers) live
// server-side. This client calls typed RPCs only.
//
// Entry points on *Client:
// Hash: CreateHashStore / GetOrCreateHashStore / HashStore(name) / DeleteHashStore
// Set: CreateSetStore / GetOrCreateSetStore / SetStore(name) / DeleteSetStore
// Queue: CreateQueue / GetOrCreateQueue / Queue(name) / DeleteQueue
import (
"context"
pb "git.awesomike.com/pub/waymaker-client/go/genpb/collections"
)
// ============================================================
// Hash
// ============================================================
// HashStoreConfig is the Hash store creation config.
type HashStoreConfig struct {
Name string
MaxBytes *uint64
Ephemeral bool
}
// HashStore is a reference to a named hash store. Cheap to copy.
type HashStore struct {
client *Client
Name string
}
func newHashStore(c *Client, name string) *HashStore {
return &HashStore{client: c, Name: name}
}
// Hash returns a handle to the hash identified by hashKey within this store.
func (s *HashStore) Hash(hashKey string) *Hash {
return &Hash{client: s.client, bucket: s.Name, hashKey: hashKey}
}
// Hash is a per-key hash within a HashStore.
type Hash struct {
client *Client
bucket string
hashKey string
}
// Set stores field → value. Returns the revision.
func (h *Hash) Set(ctx context.Context, field string, value []byte) (uint64, error) {
c := h.client.collectionsClient()
r, err := c.HashSet(ctx, &pb.HashSetRequest{
Bucket: h.bucket,
HashKey: h.hashKey,
Field: field,
Value: value,
})
if err != nil {
return 0, rpcErr(err)
}
if !r.GetSuccess() {
return 0, serverErr(r.GetResultCode(), r.GetMessage())
}
return r.GetRevision(), nil
}
// Get returns the value for field. Returns (nil, nil) when absent.
func (h *Hash) Get(ctx context.Context, field string) ([]byte, error) {
c := h.client.collectionsClient()
r, err := c.HashGet(ctx, &pb.HashGetRequest{
Bucket: h.bucket,
HashKey: h.hashKey,
Field: field,
})
if err != nil {
return nil, rpcErr(err)
}
if !r.GetSuccess() {
return nil, serverErr(r.GetResultCode(), r.GetMessage())
}
return r.GetValue(), nil
}
// Exists tests whether field exists.
func (h *Hash) Exists(ctx context.Context, field string) (bool, error) {
c := h.client.collectionsClient()
r, err := c.HashExists(ctx, &pb.HashExistsRequest{
Bucket: h.bucket,
HashKey: h.hashKey,
Field: field,
})
if err != nil {
return false, rpcErr(err)
}
if !r.GetSuccess() {
return false, serverErr(r.GetResultCode(), r.GetMessage())
}
return r.GetExists(), nil
}
// DeleteField removes field from the hash.
func (h *Hash) DeleteField(ctx context.Context, field string) error {
c := h.client.collectionsClient()
r, err := c.HashDelete(ctx, &pb.HashDeleteRequest{
Bucket: h.bucket,
HashKey: h.hashKey,
Field: field,
})
if err != nil {
return rpcErr(err)
}
if !r.GetSuccess() {
return serverErr(r.GetResultCode(), r.GetMessage())
}
return nil
}
// Fields returns all field names in the hash.
func (h *Hash) Fields(ctx context.Context) ([]string, error) {
c := h.client.collectionsClient()
r, err := c.HashFields(ctx, &pb.HashFieldsRequest{
Bucket: h.bucket,
HashKey: h.hashKey,
})
if err != nil {
return nil, rpcErr(err)
}
if !r.GetSuccess() {
return nil, serverErr(r.GetResultCode(), r.GetMessage())
}
return r.GetFields(), nil
}
// Len returns the number of fields in the hash.
func (h *Hash) Len(ctx context.Context) (uint64, error) {
c := h.client.collectionsClient()
r, err := c.HashLen(ctx, &pb.HashLenRequest{
Bucket: h.bucket,
HashKey: h.hashKey,
})
if err != nil {
return 0, rpcErr(err)
}
if !r.GetSuccess() {
return 0, serverErr(r.GetResultCode(), r.GetMessage())
}
return r.GetCount(), nil
}
// GetAll returns every field → value pair in the hash.
func (h *Hash) GetAll(ctx context.Context) (map[string][]byte, error) {
c := h.client.collectionsClient()
r, err := c.HashGetAll(ctx, &pb.HashGetAllRequest{
Bucket: h.bucket,
HashKey: h.hashKey,
})
if err != nil {
return nil, rpcErr(err)
}
if !r.GetSuccess() {
return nil, serverErr(r.GetResultCode(), r.GetMessage())
}
out := make(map[string][]byte, len(r.GetEntries()))
for _, e := range r.GetEntries() {
out[e.GetField()] = e.GetValue()
}
return out, nil
}
// --- Client HashStore entry points ---
// CreateHashStore creates a new hash store.
func (c *Client) CreateHashStore(ctx context.Context, config HashStoreConfig) (*HashStore, error) {
cc := c.collectionsClient()
r, err := cc.CreateHashStore(ctx, &pb.CreateHashStoreRequest{
Name: config.Name,
MaxBytes: uint64OrZero(config.MaxBytes),
Ephemeral: config.Ephemeral,
})
if err != nil {
return nil, rpcErr(err)
}
if !r.GetSuccess() {
return nil, serverErr(r.GetResultCode(), r.GetMessage())
}
return newHashStore(c, config.Name), nil
}
// GetOrCreateHashStore is idempotent.
func (c *Client) GetOrCreateHashStore(ctx context.Context, config HashStoreConfig) (*HashStore, error) {
s, err := c.CreateHashStore(ctx, config)
if err == nil {
return s, nil
}
if IsServerCode(err, "already_exists") {
return newHashStore(c, config.Name), nil
}
return nil, err
}
// HashStoreHandle returns a handle without verifying existence.
func (c *Client) HashStoreHandle(name string) *HashStore {
return newHashStore(c, name)
}
// DeleteHashStore deletes the hash store.
func (c *Client) DeleteHashStore(ctx context.Context, name string) error {
cc := c.collectionsClient()
r, err := cc.DeleteHashStore(ctx, &pb.DeleteHashStoreRequest{Name: name})
if err != nil {
return rpcErr(err)
}
if !r.GetSuccess() {
return serverErr(r.GetResultCode(), r.GetMessage())
}
return nil
}
// ============================================================
// Set
// ============================================================
// SetStoreConfig is the Set store creation config.
type SetStoreConfig struct {
Name string
MaxBytes *uint64
Ephemeral bool
}
// SetStore is a reference to a named set store. Cheap to copy.
type SetStore struct {
client *Client
Name string
}
func newSetStore(c *Client, name string) *SetStore {
return &SetStore{client: c, Name: name}
}
// Set returns a handle to the set identified by setKey.
func (s *SetStore) Set(setKey string) *Set {
return &Set{client: s.client, bucket: s.Name, setKey: setKey}
}
// Set is a per-key set within a SetStore.
type Set struct {
client *Client
bucket string
setKey string
}
// Add adds member to the set.
func (s *Set) Add(ctx context.Context, member string) error {
c := s.client.collectionsClient()
r, err := c.SetAdd(ctx, &pb.SetAddRequest{
Bucket: s.bucket,
SetKey: s.setKey,
Member: member,
})
if err != nil {
return rpcErr(err)
}
if !r.GetSuccess() {
return serverErr(r.GetResultCode(), r.GetMessage())
}
return nil
}
// Remove removes member from the set.
func (s *Set) Remove(ctx context.Context, member string) error {
c := s.client.collectionsClient()
r, err := c.SetRemove(ctx, &pb.SetRemoveRequest{
Bucket: s.bucket,
SetKey: s.setKey,
Member: member,
})
if err != nil {
return rpcErr(err)
}
if !r.GetSuccess() {
return serverErr(r.GetResultCode(), r.GetMessage())
}
return nil
}
// IsMember tests membership.
func (s *Set) IsMember(ctx context.Context, member string) (bool, error) {
c := s.client.collectionsClient()
r, err := c.SetIsMember(ctx, &pb.SetIsMemberRequest{
Bucket: s.bucket,
SetKey: s.setKey,
Member: member,
})
if err != nil {
return false, rpcErr(err)
}
if !r.GetSuccess() {
return false, serverErr(r.GetResultCode(), r.GetMessage())
}
return r.GetIsMember(), nil
}
// Members returns all members.
func (s *Set) Members(ctx context.Context) ([]string, error) {
c := s.client.collectionsClient()
r, err := c.SetMembers(ctx, &pb.SetMembersRequest{
Bucket: s.bucket,
SetKey: s.setKey,
})
if err != nil {
return nil, rpcErr(err)
}
if !r.GetSuccess() {
return nil, serverErr(r.GetResultCode(), r.GetMessage())
}
return r.GetMembers(), nil
}
// Len returns the number of members.
func (s *Set) Len(ctx context.Context) (uint64, error) {
c := s.client.collectionsClient()
r, err := c.SetLen(ctx, &pb.SetLenRequest{
Bucket: s.bucket,
SetKey: s.setKey,
})
if err != nil {
return 0, rpcErr(err)
}
if !r.GetSuccess() {
return 0, serverErr(r.GetResultCode(), r.GetMessage())
}
return r.GetCount(), nil
}
// --- Client SetStore entry points ---
// CreateSetStore creates a new set store.
func (c *Client) CreateSetStore(ctx context.Context, config SetStoreConfig) (*SetStore, error) {
cc := c.collectionsClient()
r, err := cc.CreateSetStore(ctx, &pb.CreateSetStoreRequest{
Name: config.Name,
MaxBytes: uint64OrZero(config.MaxBytes),
Ephemeral: config.Ephemeral,
})
if err != nil {
return nil, rpcErr(err)
}
if !r.GetSuccess() {
return nil, serverErr(r.GetResultCode(), r.GetMessage())
}
return newSetStore(c, config.Name), nil
}
// GetOrCreateSetStore is idempotent.
func (c *Client) GetOrCreateSetStore(ctx context.Context, config SetStoreConfig) (*SetStore, error) {
s, err := c.CreateSetStore(ctx, config)
if err == nil {
return s, nil
}
if IsServerCode(err, "already_exists") {
return newSetStore(c, config.Name), nil
}
return nil, err
}
// SetStoreHandle returns a handle without verifying existence.
func (c *Client) SetStoreHandle(name string) *SetStore {
return newSetStore(c, name)
}
// DeleteSetStore deletes the set store.
func (c *Client) DeleteSetStore(ctx context.Context, name string) error {
cc := c.collectionsClient()
r, err := cc.DeleteSetStore(ctx, &pb.DeleteSetStoreRequest{Name: name})
if err != nil {
return rpcErr(err)
}
if !r.GetSuccess() {
return serverErr(r.GetResultCode(), r.GetMessage())
}
return nil
}
// ============================================================
// Queue
// ============================================================
// QueueConfig is the Queue creation config.
type QueueConfig struct {
Name string
MaxBytes *uint64
MaxMessages *uint64
Ephemeral bool
}
// Queue is an RPUSH/LPOP-style append queue.
type Queue struct {
client *Client
Name string
}
func newQueue(c *Client, name string) *Queue {
return &Queue{client: c, Name: name}
}
// Push appends value to the queue. Returns the assigned sequence number.
func (q *Queue) Push(ctx context.Context, value []byte) (uint64, error) {
c := q.client.collectionsClient()
r, err := c.QueuePush(ctx, &pb.QueuePushRequest{
Bucket: q.Name,
Value: value,
})
if err != nil {
return 0, rpcErr(err)
}
if !r.GetSuccess() {
return 0, serverErr(r.GetResultCode(), r.GetMessage())
}
return r.GetSequence(), nil
}
// Pop removes and returns the front element. Returns (nil, nil) when empty.
func (q *Queue) Pop(ctx context.Context) ([]byte, error) {
c := q.client.collectionsClient()
r, err := c.QueuePop(ctx, &pb.QueuePopRequest{Bucket: q.Name})
if err != nil {
return nil, rpcErr(err)
}
if !r.GetSuccess() {
return nil, serverErr(r.GetResultCode(), r.GetMessage())
}
return r.GetValue(), nil
}
// Range returns up to limit items starting from fromSequence.
func (q *Queue) Range(ctx context.Context, from, limit uint64) ([][]byte, error) {
c := q.client.collectionsClient()
r, err := c.QueueRange(ctx, &pb.QueueRangeRequest{
Bucket: q.Name,
FromSequence: from,
Limit: limit,
})
if err != nil {
return nil, rpcErr(err)
}
if !r.GetSuccess() {
return nil, serverErr(r.GetResultCode(), r.GetMessage())
}
return r.GetValues(), nil
}
// Len returns the number of queued messages.
func (q *Queue) Len(ctx context.Context) (uint64, error) {
c := q.client.collectionsClient()
r, err := c.QueueLen(ctx, &pb.QueueLenRequest{Bucket: q.Name})
if err != nil {
return 0, rpcErr(err)
}
if !r.GetSuccess() {
return 0, serverErr(r.GetResultCode(), r.GetMessage())
}
return r.GetCount(), nil
}
// --- Client Queue entry points ---
// CreateQueue creates a new queue.
func (c *Client) CreateQueue(ctx context.Context, config QueueConfig) (*Queue, error) {
cc := c.collectionsClient()
r, err := cc.CreateQueue(ctx, &pb.CreateQueueRequest{
Name: config.Name,
MaxBytes: uint64OrZero(config.MaxBytes),
MaxMessages: uint64OrZero(config.MaxMessages),
Ephemeral: config.Ephemeral,
})
if err != nil {
return nil, rpcErr(err)
}
if !r.GetSuccess() {
return nil, serverErr(r.GetResultCode(), r.GetMessage())
}
return newQueue(c, config.Name), nil
}
// GetOrCreateQueue is idempotent.
func (c *Client) GetOrCreateQueue(ctx context.Context, config QueueConfig) (*Queue, error) {
q, err := c.CreateQueue(ctx, config)
if err == nil {
return q, nil
}
if IsServerCode(err, "already_exists") {
return newQueue(c, config.Name), nil
}
return nil, err
}
// QueueHandle returns a handle without verifying existence.
func (c *Client) QueueHandle(name string) *Queue {
return newQueue(c, name)
}
// DeleteQueue deletes the queue.
func (c *Client) DeleteQueue(ctx context.Context, name string) error {
cc := c.collectionsClient()
r, err := cc.DeleteQueue(ctx, &pb.DeleteQueueRequest{Name: name})
if err != nil {
return rpcErr(err)
}
if !r.GetSuccess() {
return serverErr(r.GetResultCode(), r.GetMessage())
}
return nil
}

58
go/errors.go Normal file
View File

@ -0,0 +1,58 @@
// Package waymaker is the official Go client for waymaker, a distributed
// coordination service. It wraps the gRPC stubs in genpb/ with ergonomic
// handles for locks, streams, KV, collections, sketches, object store and
// cache.
//
// Usage:
//
// client, err := waymaker.Connect(ctx, "http://localhost:8818")
// lock, err := client.AcquireLock(ctx, "my-key", waymaker.LockConfig{...})
package waymaker
import (
"fmt"
"google.golang.org/grpc/status"
)
// Error is the typed error surface for the waymaker Go client.
//
// There are three categories:
// - [ErrRPC] — gRPC transport / status error (network, server-side code).
// - [ErrServer] — server returned success=false with a result_code string.
// - [ErrInvalid] — caller passed a value the wrapper could not translate.
type Error struct {
// Kind is one of "rpc", "server", "invalid".
Kind string
// Code is the server result_code ("no_such_stream", "expired", …) for
// Kind=="server", or the gRPC status code string for Kind=="rpc".
Code string
// Message is the human-readable detail.
Message string
}
func (e *Error) Error() string {
return fmt.Sprintf("waymaker %s %s: %s", e.Kind, e.Code, e.Message)
}
// IsServerCode reports whether err is a server error whose result_code
// equals code.
func IsServerCode(err error, code string) bool {
e, ok := err.(*Error)
return ok && e.Kind == "server" && e.Code == code
}
func serverErr(code, message string) *Error {
return &Error{Kind: "server", Code: code, Message: message}
}
func invalidErr(msg string) *Error {
return &Error{Kind: "invalid", Code: "invalid_argument", Message: msg}
}
func rpcErr(err error) *Error {
if s, ok := status.FromError(err); ok {
return &Error{Kind: "rpc", Code: s.Code().String(), Message: s.Message()}
}
return &Error{Kind: "rpc", Code: "unknown", Message: err.Error()}
}

497
go/genpb/cache/cache.pb.go vendored Normal file
View File

@ -0,0 +1,497 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.36.11
// protoc v7.34.1
// source: cache.proto
package waymaker_cache
import (
protoreflect "google.golang.org/protobuf/reflect/protoreflect"
protoimpl "google.golang.org/protobuf/runtime/protoimpl"
reflect "reflect"
sync "sync"
unsafe "unsafe"
)
const (
// Verify that this generated code is sufficiently up-to-date.
_ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion)
// Verify that runtime/protoimpl is sufficiently up-to-date.
_ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20)
)
type AttachPolicyRequest struct {
state protoimpl.MessageState `protogen:"open.v1"`
Bucket string `protobuf:"bytes,1,opt,name=bucket,proto3" json:"bucket,omitempty"`
PolicyId string `protobuf:"bytes,2,opt,name=policy_id,json=policyId,proto3" json:"policy_id,omitempty"`
// Policy-specific knobs (e.g. `max_entries`, `default_ttl_ms`)
// — interpretation is server-side.
Params map[string]string `protobuf:"bytes,3,rep,name=params,proto3" json:"params,omitempty" protobuf_key:"bytes,1,opt,name=key" protobuf_val:"bytes,2,opt,name=value"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *AttachPolicyRequest) Reset() {
*x = AttachPolicyRequest{}
mi := &file_cache_proto_msgTypes[0]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *AttachPolicyRequest) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*AttachPolicyRequest) ProtoMessage() {}
func (x *AttachPolicyRequest) ProtoReflect() protoreflect.Message {
mi := &file_cache_proto_msgTypes[0]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use AttachPolicyRequest.ProtoReflect.Descriptor instead.
func (*AttachPolicyRequest) Descriptor() ([]byte, []int) {
return file_cache_proto_rawDescGZIP(), []int{0}
}
func (x *AttachPolicyRequest) GetBucket() string {
if x != nil {
return x.Bucket
}
return ""
}
func (x *AttachPolicyRequest) GetPolicyId() string {
if x != nil {
return x.PolicyId
}
return ""
}
func (x *AttachPolicyRequest) GetParams() map[string]string {
if x != nil {
return x.Params
}
return nil
}
type AttachPolicyResponse struct {
state protoimpl.MessageState `protogen:"open.v1"`
Success bool `protobuf:"varint,1,opt,name=success,proto3" json:"success,omitempty"`
ResultCode string `protobuf:"bytes,2,opt,name=result_code,json=resultCode,proto3" json:"result_code,omitempty"`
Message string `protobuf:"bytes,3,opt,name=message,proto3" json:"message,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *AttachPolicyResponse) Reset() {
*x = AttachPolicyResponse{}
mi := &file_cache_proto_msgTypes[1]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *AttachPolicyResponse) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*AttachPolicyResponse) ProtoMessage() {}
func (x *AttachPolicyResponse) ProtoReflect() protoreflect.Message {
mi := &file_cache_proto_msgTypes[1]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use AttachPolicyResponse.ProtoReflect.Descriptor instead.
func (*AttachPolicyResponse) Descriptor() ([]byte, []int) {
return file_cache_proto_rawDescGZIP(), []int{1}
}
func (x *AttachPolicyResponse) GetSuccess() bool {
if x != nil {
return x.Success
}
return false
}
func (x *AttachPolicyResponse) GetResultCode() string {
if x != nil {
return x.ResultCode
}
return ""
}
func (x *AttachPolicyResponse) GetMessage() string {
if x != nil {
return x.Message
}
return ""
}
type DetachPolicyRequest struct {
state protoimpl.MessageState `protogen:"open.v1"`
Bucket string `protobuf:"bytes,1,opt,name=bucket,proto3" json:"bucket,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *DetachPolicyRequest) Reset() {
*x = DetachPolicyRequest{}
mi := &file_cache_proto_msgTypes[2]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *DetachPolicyRequest) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*DetachPolicyRequest) ProtoMessage() {}
func (x *DetachPolicyRequest) ProtoReflect() protoreflect.Message {
mi := &file_cache_proto_msgTypes[2]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use DetachPolicyRequest.ProtoReflect.Descriptor instead.
func (*DetachPolicyRequest) Descriptor() ([]byte, []int) {
return file_cache_proto_rawDescGZIP(), []int{2}
}
func (x *DetachPolicyRequest) GetBucket() string {
if x != nil {
return x.Bucket
}
return ""
}
type DetachPolicyResponse struct {
state protoimpl.MessageState `protogen:"open.v1"`
Success bool `protobuf:"varint,1,opt,name=success,proto3" json:"success,omitempty"`
ResultCode string `protobuf:"bytes,2,opt,name=result_code,json=resultCode,proto3" json:"result_code,omitempty"`
Message string `protobuf:"bytes,3,opt,name=message,proto3" json:"message,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *DetachPolicyResponse) Reset() {
*x = DetachPolicyResponse{}
mi := &file_cache_proto_msgTypes[3]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *DetachPolicyResponse) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*DetachPolicyResponse) ProtoMessage() {}
func (x *DetachPolicyResponse) ProtoReflect() protoreflect.Message {
mi := &file_cache_proto_msgTypes[3]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use DetachPolicyResponse.ProtoReflect.Descriptor instead.
func (*DetachPolicyResponse) Descriptor() ([]byte, []int) {
return file_cache_proto_rawDescGZIP(), []int{3}
}
func (x *DetachPolicyResponse) GetSuccess() bool {
if x != nil {
return x.Success
}
return false
}
func (x *DetachPolicyResponse) GetResultCode() string {
if x != nil {
return x.ResultCode
}
return ""
}
func (x *DetachPolicyResponse) GetMessage() string {
if x != nil {
return x.Message
}
return ""
}
type StatsRequest struct {
state protoimpl.MessageState `protogen:"open.v1"`
Bucket string `protobuf:"bytes,1,opt,name=bucket,proto3" json:"bucket,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *StatsRequest) Reset() {
*x = StatsRequest{}
mi := &file_cache_proto_msgTypes[4]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *StatsRequest) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*StatsRequest) ProtoMessage() {}
func (x *StatsRequest) ProtoReflect() protoreflect.Message {
mi := &file_cache_proto_msgTypes[4]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use StatsRequest.ProtoReflect.Descriptor instead.
func (*StatsRequest) Descriptor() ([]byte, []int) {
return file_cache_proto_rawDescGZIP(), []int{4}
}
func (x *StatsRequest) GetBucket() string {
if x != nil {
return x.Bucket
}
return ""
}
type StatsResponse struct {
state protoimpl.MessageState `protogen:"open.v1"`
Success bool `protobuf:"varint,1,opt,name=success,proto3" json:"success,omitempty"`
ResultCode string `protobuf:"bytes,2,opt,name=result_code,json=resultCode,proto3" json:"result_code,omitempty"`
Message string `protobuf:"bytes,3,opt,name=message,proto3" json:"message,omitempty"`
HitCount uint64 `protobuf:"varint,4,opt,name=hit_count,json=hitCount,proto3" json:"hit_count,omitempty"`
MissCount uint64 `protobuf:"varint,5,opt,name=miss_count,json=missCount,proto3" json:"miss_count,omitempty"`
EvictionCount uint64 `protobuf:"varint,6,opt,name=eviction_count,json=evictionCount,proto3" json:"eviction_count,omitempty"`
SizeBytes uint64 `protobuf:"varint,7,opt,name=size_bytes,json=sizeBytes,proto3" json:"size_bytes,omitempty"`
EntryCount uint64 `protobuf:"varint,8,opt,name=entry_count,json=entryCount,proto3" json:"entry_count,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
func (x *StatsResponse) Reset() {
*x = StatsResponse{}
mi := &file_cache_proto_msgTypes[5]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
func (x *StatsResponse) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*StatsResponse) ProtoMessage() {}
func (x *StatsResponse) ProtoReflect() protoreflect.Message {
mi := &file_cache_proto_msgTypes[5]
if x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use StatsResponse.ProtoReflect.Descriptor instead.
func (*StatsResponse) Descriptor() ([]byte, []int) {
return file_cache_proto_rawDescGZIP(), []int{5}
}
func (x *StatsResponse) GetSuccess() bool {
if x != nil {
return x.Success
}
return false
}
func (x *StatsResponse) GetResultCode() string {
if x != nil {
return x.ResultCode
}
return ""
}
func (x *StatsResponse) GetMessage() string {
if x != nil {
return x.Message
}
return ""
}
func (x *StatsResponse) GetHitCount() uint64 {
if x != nil {
return x.HitCount
}
return 0
}
func (x *StatsResponse) GetMissCount() uint64 {
if x != nil {
return x.MissCount
}
return 0
}
func (x *StatsResponse) GetEvictionCount() uint64 {
if x != nil {
return x.EvictionCount
}
return 0
}
func (x *StatsResponse) GetSizeBytes() uint64 {
if x != nil {
return x.SizeBytes
}
return 0
}
func (x *StatsResponse) GetEntryCount() uint64 {
if x != nil {
return x.EntryCount
}
return 0
}
var File_cache_proto protoreflect.FileDescriptor
const file_cache_proto_rawDesc = "" +
"\n" +
"\vcache.proto\x12\x0ewaymaker.cache\"\xce\x01\n" +
"\x13AttachPolicyRequest\x12\x16\n" +
"\x06bucket\x18\x01 \x01(\tR\x06bucket\x12\x1b\n" +
"\tpolicy_id\x18\x02 \x01(\tR\bpolicyId\x12G\n" +
"\x06params\x18\x03 \x03(\v2/.waymaker.cache.AttachPolicyRequest.ParamsEntryR\x06params\x1a9\n" +
"\vParamsEntry\x12\x10\n" +
"\x03key\x18\x01 \x01(\tR\x03key\x12\x14\n" +
"\x05value\x18\x02 \x01(\tR\x05value:\x028\x01\"k\n" +
"\x14AttachPolicyResponse\x12\x18\n" +
"\asuccess\x18\x01 \x01(\bR\asuccess\x12\x1f\n" +
"\vresult_code\x18\x02 \x01(\tR\n" +
"resultCode\x12\x18\n" +
"\amessage\x18\x03 \x01(\tR\amessage\"-\n" +
"\x13DetachPolicyRequest\x12\x16\n" +
"\x06bucket\x18\x01 \x01(\tR\x06bucket\"k\n" +
"\x14DetachPolicyResponse\x12\x18\n" +
"\asuccess\x18\x01 \x01(\bR\asuccess\x12\x1f\n" +
"\vresult_code\x18\x02 \x01(\tR\n" +
"resultCode\x12\x18\n" +
"\amessage\x18\x03 \x01(\tR\amessage\"&\n" +
"\fStatsRequest\x12\x16\n" +
"\x06bucket\x18\x01 \x01(\tR\x06bucket\"\x87\x02\n" +
"\rStatsResponse\x12\x18\n" +
"\asuccess\x18\x01 \x01(\bR\asuccess\x12\x1f\n" +
"\vresult_code\x18\x02 \x01(\tR\n" +
"resultCode\x12\x18\n" +
"\amessage\x18\x03 \x01(\tR\amessage\x12\x1b\n" +
"\thit_count\x18\x04 \x01(\x04R\bhitCount\x12\x1d\n" +
"\n" +
"miss_count\x18\x05 \x01(\x04R\tmissCount\x12%\n" +
"\x0eeviction_count\x18\x06 \x01(\x04R\revictionCount\x12\x1d\n" +
"\n" +
"size_bytes\x18\a \x01(\x04R\tsizeBytes\x12\x1f\n" +
"\ventry_count\x18\b \x01(\x04R\n" +
"entryCount2\x92\x02\n" +
"\x14WaymakerCacheService\x12Y\n" +
"\fAttachPolicy\x12#.waymaker.cache.AttachPolicyRequest\x1a$.waymaker.cache.AttachPolicyResponse\x12Y\n" +
"\fDetachPolicy\x12#.waymaker.cache.DetachPolicyRequest\x1a$.waymaker.cache.DetachPolicyResponse\x12D\n" +
"\x05Stats\x12\x1c.waymaker.cache.StatsRequest\x1a\x1d.waymaker.cache.StatsResponseB\x16Z\x14/apis/waymaker_cacheb\x06proto3"
var (
file_cache_proto_rawDescOnce sync.Once
file_cache_proto_rawDescData []byte
)
func file_cache_proto_rawDescGZIP() []byte {
file_cache_proto_rawDescOnce.Do(func() {
file_cache_proto_rawDescData = protoimpl.X.CompressGZIP(unsafe.Slice(unsafe.StringData(file_cache_proto_rawDesc), len(file_cache_proto_rawDesc)))
})
return file_cache_proto_rawDescData
}
var file_cache_proto_msgTypes = make([]protoimpl.MessageInfo, 7)
var file_cache_proto_goTypes = []any{
(*AttachPolicyRequest)(nil), // 0: waymaker.cache.AttachPolicyRequest
(*AttachPolicyResponse)(nil), // 1: waymaker.cache.AttachPolicyResponse
(*DetachPolicyRequest)(nil), // 2: waymaker.cache.DetachPolicyRequest
(*DetachPolicyResponse)(nil), // 3: waymaker.cache.DetachPolicyResponse
(*StatsRequest)(nil), // 4: waymaker.cache.StatsRequest
(*StatsResponse)(nil), // 5: waymaker.cache.StatsResponse
nil, // 6: waymaker.cache.AttachPolicyRequest.ParamsEntry
}
var file_cache_proto_depIdxs = []int32{
6, // 0: waymaker.cache.AttachPolicyRequest.params:type_name -> waymaker.cache.AttachPolicyRequest.ParamsEntry
0, // 1: waymaker.cache.WaymakerCacheService.AttachPolicy:input_type -> waymaker.cache.AttachPolicyRequest
2, // 2: waymaker.cache.WaymakerCacheService.DetachPolicy:input_type -> waymaker.cache.DetachPolicyRequest
4, // 3: waymaker.cache.WaymakerCacheService.Stats:input_type -> waymaker.cache.StatsRequest
1, // 4: waymaker.cache.WaymakerCacheService.AttachPolicy:output_type -> waymaker.cache.AttachPolicyResponse
3, // 5: waymaker.cache.WaymakerCacheService.DetachPolicy:output_type -> waymaker.cache.DetachPolicyResponse
5, // 6: waymaker.cache.WaymakerCacheService.Stats:output_type -> waymaker.cache.StatsResponse
4, // [4:7] is the sub-list for method output_type
1, // [1:4] is the sub-list for method input_type
1, // [1:1] is the sub-list for extension type_name
1, // [1:1] is the sub-list for extension extendee
0, // [0:1] is the sub-list for field type_name
}
func init() { file_cache_proto_init() }
func file_cache_proto_init() {
if File_cache_proto != nil {
return
}
type x struct{}
out := protoimpl.TypeBuilder{
File: protoimpl.DescBuilder{
GoPackagePath: reflect.TypeOf(x{}).PkgPath(),
RawDescriptor: unsafe.Slice(unsafe.StringData(file_cache_proto_rawDesc), len(file_cache_proto_rawDesc)),
NumEnums: 0,
NumMessages: 7,
NumExtensions: 0,
NumServices: 1,
},
GoTypes: file_cache_proto_goTypes,
DependencyIndexes: file_cache_proto_depIdxs,
MessageInfos: file_cache_proto_msgTypes,
}.Build()
File_cache_proto = out.File
file_cache_proto_goTypes = nil
file_cache_proto_depIdxs = nil
}

209
go/genpb/cache/cache_grpc.pb.go vendored Normal file
View File

@ -0,0 +1,209 @@
// Code generated by protoc-gen-go-grpc. DO NOT EDIT.
// versions:
// - protoc-gen-go-grpc v1.6.2
// - protoc v7.34.1
// source: cache.proto
package waymaker_cache
import (
context "context"
grpc "google.golang.org/grpc"
codes "google.golang.org/grpc/codes"
status "google.golang.org/grpc/status"
)
// This is a compile-time assertion to ensure that this generated file
// is compatible with the grpc package it is being compiled against.
// Requires gRPC-Go v1.64.0 or later.
const _ = grpc.SupportPackageIsVersion9
const (
WaymakerCacheService_AttachPolicy_FullMethodName = "/waymaker.cache.WaymakerCacheService/AttachPolicy"
WaymakerCacheService_DetachPolicy_FullMethodName = "/waymaker.cache.WaymakerCacheService/DetachPolicy"
WaymakerCacheService_Stats_FullMethodName = "/waymaker.cache.WaymakerCacheService/Stats"
)
// WaymakerCacheServiceClient is the client API for WaymakerCacheService service.
//
// For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream.
type WaymakerCacheServiceClient interface {
// Apply a TTL policy to a bucket. `policy_id` selects from
// server-configured policies (initially: `lru`, `expiry`).
AttachPolicy(ctx context.Context, in *AttachPolicyRequest, opts ...grpc.CallOption) (*AttachPolicyResponse, error)
// Detach the policy currently bound to `bucket` (no-op if
// none).
DetachPolicy(ctx context.Context, in *DetachPolicyRequest, opts ...grpc.CallOption) (*DetachPolicyResponse, error)
// Report current cache stats (hit/miss/eviction counters,
// memory footprint) for a bucket.
Stats(ctx context.Context, in *StatsRequest, opts ...grpc.CallOption) (*StatsResponse, error)
}
type waymakerCacheServiceClient struct {
cc grpc.ClientConnInterface
}
func NewWaymakerCacheServiceClient(cc grpc.ClientConnInterface) WaymakerCacheServiceClient {
return &waymakerCacheServiceClient{cc}
}
func (c *waymakerCacheServiceClient) AttachPolicy(ctx context.Context, in *AttachPolicyRequest, opts ...grpc.CallOption) (*AttachPolicyResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(AttachPolicyResponse)
err := c.cc.Invoke(ctx, WaymakerCacheService_AttachPolicy_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerCacheServiceClient) DetachPolicy(ctx context.Context, in *DetachPolicyRequest, opts ...grpc.CallOption) (*DetachPolicyResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(DetachPolicyResponse)
err := c.cc.Invoke(ctx, WaymakerCacheService_DetachPolicy_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerCacheServiceClient) Stats(ctx context.Context, in *StatsRequest, opts ...grpc.CallOption) (*StatsResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(StatsResponse)
err := c.cc.Invoke(ctx, WaymakerCacheService_Stats_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
// WaymakerCacheServiceServer is the server API for WaymakerCacheService service.
// All implementations must embed UnimplementedWaymakerCacheServiceServer
// for forward compatibility.
type WaymakerCacheServiceServer interface {
// Apply a TTL policy to a bucket. `policy_id` selects from
// server-configured policies (initially: `lru`, `expiry`).
AttachPolicy(context.Context, *AttachPolicyRequest) (*AttachPolicyResponse, error)
// Detach the policy currently bound to `bucket` (no-op if
// none).
DetachPolicy(context.Context, *DetachPolicyRequest) (*DetachPolicyResponse, error)
// Report current cache stats (hit/miss/eviction counters,
// memory footprint) for a bucket.
Stats(context.Context, *StatsRequest) (*StatsResponse, error)
mustEmbedUnimplementedWaymakerCacheServiceServer()
}
// UnimplementedWaymakerCacheServiceServer must be embedded to have
// forward compatible implementations.
//
// NOTE: this should be embedded by value instead of pointer to avoid a nil
// pointer dereference when methods are called.
type UnimplementedWaymakerCacheServiceServer struct{}
func (UnimplementedWaymakerCacheServiceServer) AttachPolicy(context.Context, *AttachPolicyRequest) (*AttachPolicyResponse, error) {
return nil, status.Error(codes.Unimplemented, "method AttachPolicy not implemented")
}
func (UnimplementedWaymakerCacheServiceServer) DetachPolicy(context.Context, *DetachPolicyRequest) (*DetachPolicyResponse, error) {
return nil, status.Error(codes.Unimplemented, "method DetachPolicy not implemented")
}
func (UnimplementedWaymakerCacheServiceServer) Stats(context.Context, *StatsRequest) (*StatsResponse, error) {
return nil, status.Error(codes.Unimplemented, "method Stats not implemented")
}
func (UnimplementedWaymakerCacheServiceServer) mustEmbedUnimplementedWaymakerCacheServiceServer() {}
func (UnimplementedWaymakerCacheServiceServer) testEmbeddedByValue() {}
// UnsafeWaymakerCacheServiceServer may be embedded to opt out of forward compatibility for this service.
// Use of this interface is not recommended, as added methods to WaymakerCacheServiceServer will
// result in compilation errors.
type UnsafeWaymakerCacheServiceServer interface {
mustEmbedUnimplementedWaymakerCacheServiceServer()
}
func RegisterWaymakerCacheServiceServer(s grpc.ServiceRegistrar, srv WaymakerCacheServiceServer) {
// If the following call panics, it indicates UnimplementedWaymakerCacheServiceServer was
// embedded by pointer and is nil. This will cause panics if an
// unimplemented method is ever invoked, so we test this at initialization
// time to prevent it from happening at runtime later due to I/O.
if t, ok := srv.(interface{ testEmbeddedByValue() }); ok {
t.testEmbeddedByValue()
}
s.RegisterService(&WaymakerCacheService_ServiceDesc, srv)
}
func _WaymakerCacheService_AttachPolicy_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(AttachPolicyRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerCacheServiceServer).AttachPolicy(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerCacheService_AttachPolicy_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerCacheServiceServer).AttachPolicy(ctx, req.(*AttachPolicyRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerCacheService_DetachPolicy_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(DetachPolicyRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerCacheServiceServer).DetachPolicy(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerCacheService_DetachPolicy_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerCacheServiceServer).DetachPolicy(ctx, req.(*DetachPolicyRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerCacheService_Stats_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(StatsRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerCacheServiceServer).Stats(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerCacheService_Stats_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerCacheServiceServer).Stats(ctx, req.(*StatsRequest))
}
return interceptor(ctx, in, info, handler)
}
// WaymakerCacheService_ServiceDesc is the grpc.ServiceDesc for WaymakerCacheService service.
// It's only intended for direct use with grpc.RegisterService,
// and not to be introspected or modified (even as a copy)
var WaymakerCacheService_ServiceDesc = grpc.ServiceDesc{
ServiceName: "waymaker.cache.WaymakerCacheService",
HandlerType: (*WaymakerCacheServiceServer)(nil),
Methods: []grpc.MethodDesc{
{
MethodName: "AttachPolicy",
Handler: _WaymakerCacheService_AttachPolicy_Handler,
},
{
MethodName: "DetachPolicy",
Handler: _WaymakerCacheService_DetachPolicy_Handler,
},
{
MethodName: "Stats",
Handler: _WaymakerCacheService_Stats_Handler,
},
},
Streams: []grpc.StreamDesc{},
Metadata: "cache.proto",
}

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,926 @@
// Code generated by protoc-gen-go-grpc. DO NOT EDIT.
// versions:
// - protoc-gen-go-grpc v1.6.2
// - protoc v7.34.1
// source: collections.proto
package waymaker_collections
import (
context "context"
grpc "google.golang.org/grpc"
codes "google.golang.org/grpc/codes"
status "google.golang.org/grpc/status"
)
// This is a compile-time assertion to ensure that this generated file
// is compatible with the grpc package it is being compiled against.
// Requires gRPC-Go v1.64.0 or later.
const _ = grpc.SupportPackageIsVersion9
const (
WaymakerCollectionsService_CreateHashStore_FullMethodName = "/waymaker.collections.WaymakerCollectionsService/CreateHashStore"
WaymakerCollectionsService_DeleteHashStore_FullMethodName = "/waymaker.collections.WaymakerCollectionsService/DeleteHashStore"
WaymakerCollectionsService_HashSet_FullMethodName = "/waymaker.collections.WaymakerCollectionsService/HashSet"
WaymakerCollectionsService_HashGet_FullMethodName = "/waymaker.collections.WaymakerCollectionsService/HashGet"
WaymakerCollectionsService_HashExists_FullMethodName = "/waymaker.collections.WaymakerCollectionsService/HashExists"
WaymakerCollectionsService_HashDelete_FullMethodName = "/waymaker.collections.WaymakerCollectionsService/HashDelete"
WaymakerCollectionsService_HashGetAll_FullMethodName = "/waymaker.collections.WaymakerCollectionsService/HashGetAll"
WaymakerCollectionsService_HashFields_FullMethodName = "/waymaker.collections.WaymakerCollectionsService/HashFields"
WaymakerCollectionsService_HashLen_FullMethodName = "/waymaker.collections.WaymakerCollectionsService/HashLen"
WaymakerCollectionsService_CreateSetStore_FullMethodName = "/waymaker.collections.WaymakerCollectionsService/CreateSetStore"
WaymakerCollectionsService_DeleteSetStore_FullMethodName = "/waymaker.collections.WaymakerCollectionsService/DeleteSetStore"
WaymakerCollectionsService_SetAdd_FullMethodName = "/waymaker.collections.WaymakerCollectionsService/SetAdd"
WaymakerCollectionsService_SetRemove_FullMethodName = "/waymaker.collections.WaymakerCollectionsService/SetRemove"
WaymakerCollectionsService_SetIsMember_FullMethodName = "/waymaker.collections.WaymakerCollectionsService/SetIsMember"
WaymakerCollectionsService_SetMembers_FullMethodName = "/waymaker.collections.WaymakerCollectionsService/SetMembers"
WaymakerCollectionsService_SetLen_FullMethodName = "/waymaker.collections.WaymakerCollectionsService/SetLen"
WaymakerCollectionsService_CreateQueue_FullMethodName = "/waymaker.collections.WaymakerCollectionsService/CreateQueue"
WaymakerCollectionsService_DeleteQueue_FullMethodName = "/waymaker.collections.WaymakerCollectionsService/DeleteQueue"
WaymakerCollectionsService_QueuePush_FullMethodName = "/waymaker.collections.WaymakerCollectionsService/QueuePush"
WaymakerCollectionsService_QueuePop_FullMethodName = "/waymaker.collections.WaymakerCollectionsService/QueuePop"
WaymakerCollectionsService_QueueRange_FullMethodName = "/waymaker.collections.WaymakerCollectionsService/QueueRange"
WaymakerCollectionsService_QueueLen_FullMethodName = "/waymaker.collections.WaymakerCollectionsService/QueueLen"
)
// WaymakerCollectionsServiceClient is the client API for WaymakerCollectionsService service.
//
// For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream.
type WaymakerCollectionsServiceClient interface {
// ----- Hash -------------------------------------------------
CreateHashStore(ctx context.Context, in *CreateHashStoreRequest, opts ...grpc.CallOption) (*CreateHashStoreResponse, error)
DeleteHashStore(ctx context.Context, in *DeleteHashStoreRequest, opts ...grpc.CallOption) (*DeleteHashStoreResponse, error)
HashSet(ctx context.Context, in *HashSetRequest, opts ...grpc.CallOption) (*HashSetResponse, error)
HashGet(ctx context.Context, in *HashGetRequest, opts ...grpc.CallOption) (*HashGetResponse, error)
HashExists(ctx context.Context, in *HashExistsRequest, opts ...grpc.CallOption) (*HashExistsResponse, error)
HashDelete(ctx context.Context, in *HashDeleteRequest, opts ...grpc.CallOption) (*HashDeleteResponse, error)
HashGetAll(ctx context.Context, in *HashGetAllRequest, opts ...grpc.CallOption) (*HashGetAllResponse, error)
HashFields(ctx context.Context, in *HashFieldsRequest, opts ...grpc.CallOption) (*HashFieldsResponse, error)
HashLen(ctx context.Context, in *HashLenRequest, opts ...grpc.CallOption) (*HashLenResponse, error)
// ----- Set --------------------------------------------------
CreateSetStore(ctx context.Context, in *CreateSetStoreRequest, opts ...grpc.CallOption) (*CreateSetStoreResponse, error)
DeleteSetStore(ctx context.Context, in *DeleteSetStoreRequest, opts ...grpc.CallOption) (*DeleteSetStoreResponse, error)
SetAdd(ctx context.Context, in *SetAddRequest, opts ...grpc.CallOption) (*SetAddResponse, error)
SetRemove(ctx context.Context, in *SetRemoveRequest, opts ...grpc.CallOption) (*SetRemoveResponse, error)
SetIsMember(ctx context.Context, in *SetIsMemberRequest, opts ...grpc.CallOption) (*SetIsMemberResponse, error)
SetMembers(ctx context.Context, in *SetMembersRequest, opts ...grpc.CallOption) (*SetMembersResponse, error)
SetLen(ctx context.Context, in *SetLenRequest, opts ...grpc.CallOption) (*SetLenResponse, error)
// ----- Queue ------------------------------------------------
CreateQueue(ctx context.Context, in *CreateQueueRequest, opts ...grpc.CallOption) (*CreateQueueResponse, error)
DeleteQueue(ctx context.Context, in *DeleteQueueRequest, opts ...grpc.CallOption) (*DeleteQueueResponse, error)
QueuePush(ctx context.Context, in *QueuePushRequest, opts ...grpc.CallOption) (*QueuePushResponse, error)
QueuePop(ctx context.Context, in *QueuePopRequest, opts ...grpc.CallOption) (*QueuePopResponse, error)
QueueRange(ctx context.Context, in *QueueRangeRequest, opts ...grpc.CallOption) (*QueueRangeResponse, error)
QueueLen(ctx context.Context, in *QueueLenRequest, opts ...grpc.CallOption) (*QueueLenResponse, error)
}
type waymakerCollectionsServiceClient struct {
cc grpc.ClientConnInterface
}
func NewWaymakerCollectionsServiceClient(cc grpc.ClientConnInterface) WaymakerCollectionsServiceClient {
return &waymakerCollectionsServiceClient{cc}
}
func (c *waymakerCollectionsServiceClient) CreateHashStore(ctx context.Context, in *CreateHashStoreRequest, opts ...grpc.CallOption) (*CreateHashStoreResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(CreateHashStoreResponse)
err := c.cc.Invoke(ctx, WaymakerCollectionsService_CreateHashStore_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerCollectionsServiceClient) DeleteHashStore(ctx context.Context, in *DeleteHashStoreRequest, opts ...grpc.CallOption) (*DeleteHashStoreResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(DeleteHashStoreResponse)
err := c.cc.Invoke(ctx, WaymakerCollectionsService_DeleteHashStore_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerCollectionsServiceClient) HashSet(ctx context.Context, in *HashSetRequest, opts ...grpc.CallOption) (*HashSetResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(HashSetResponse)
err := c.cc.Invoke(ctx, WaymakerCollectionsService_HashSet_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerCollectionsServiceClient) HashGet(ctx context.Context, in *HashGetRequest, opts ...grpc.CallOption) (*HashGetResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(HashGetResponse)
err := c.cc.Invoke(ctx, WaymakerCollectionsService_HashGet_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerCollectionsServiceClient) HashExists(ctx context.Context, in *HashExistsRequest, opts ...grpc.CallOption) (*HashExistsResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(HashExistsResponse)
err := c.cc.Invoke(ctx, WaymakerCollectionsService_HashExists_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerCollectionsServiceClient) HashDelete(ctx context.Context, in *HashDeleteRequest, opts ...grpc.CallOption) (*HashDeleteResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(HashDeleteResponse)
err := c.cc.Invoke(ctx, WaymakerCollectionsService_HashDelete_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerCollectionsServiceClient) HashGetAll(ctx context.Context, in *HashGetAllRequest, opts ...grpc.CallOption) (*HashGetAllResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(HashGetAllResponse)
err := c.cc.Invoke(ctx, WaymakerCollectionsService_HashGetAll_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerCollectionsServiceClient) HashFields(ctx context.Context, in *HashFieldsRequest, opts ...grpc.CallOption) (*HashFieldsResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(HashFieldsResponse)
err := c.cc.Invoke(ctx, WaymakerCollectionsService_HashFields_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerCollectionsServiceClient) HashLen(ctx context.Context, in *HashLenRequest, opts ...grpc.CallOption) (*HashLenResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(HashLenResponse)
err := c.cc.Invoke(ctx, WaymakerCollectionsService_HashLen_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerCollectionsServiceClient) CreateSetStore(ctx context.Context, in *CreateSetStoreRequest, opts ...grpc.CallOption) (*CreateSetStoreResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(CreateSetStoreResponse)
err := c.cc.Invoke(ctx, WaymakerCollectionsService_CreateSetStore_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerCollectionsServiceClient) DeleteSetStore(ctx context.Context, in *DeleteSetStoreRequest, opts ...grpc.CallOption) (*DeleteSetStoreResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(DeleteSetStoreResponse)
err := c.cc.Invoke(ctx, WaymakerCollectionsService_DeleteSetStore_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerCollectionsServiceClient) SetAdd(ctx context.Context, in *SetAddRequest, opts ...grpc.CallOption) (*SetAddResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(SetAddResponse)
err := c.cc.Invoke(ctx, WaymakerCollectionsService_SetAdd_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerCollectionsServiceClient) SetRemove(ctx context.Context, in *SetRemoveRequest, opts ...grpc.CallOption) (*SetRemoveResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(SetRemoveResponse)
err := c.cc.Invoke(ctx, WaymakerCollectionsService_SetRemove_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerCollectionsServiceClient) SetIsMember(ctx context.Context, in *SetIsMemberRequest, opts ...grpc.CallOption) (*SetIsMemberResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(SetIsMemberResponse)
err := c.cc.Invoke(ctx, WaymakerCollectionsService_SetIsMember_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerCollectionsServiceClient) SetMembers(ctx context.Context, in *SetMembersRequest, opts ...grpc.CallOption) (*SetMembersResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(SetMembersResponse)
err := c.cc.Invoke(ctx, WaymakerCollectionsService_SetMembers_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerCollectionsServiceClient) SetLen(ctx context.Context, in *SetLenRequest, opts ...grpc.CallOption) (*SetLenResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(SetLenResponse)
err := c.cc.Invoke(ctx, WaymakerCollectionsService_SetLen_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerCollectionsServiceClient) CreateQueue(ctx context.Context, in *CreateQueueRequest, opts ...grpc.CallOption) (*CreateQueueResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(CreateQueueResponse)
err := c.cc.Invoke(ctx, WaymakerCollectionsService_CreateQueue_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerCollectionsServiceClient) DeleteQueue(ctx context.Context, in *DeleteQueueRequest, opts ...grpc.CallOption) (*DeleteQueueResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(DeleteQueueResponse)
err := c.cc.Invoke(ctx, WaymakerCollectionsService_DeleteQueue_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerCollectionsServiceClient) QueuePush(ctx context.Context, in *QueuePushRequest, opts ...grpc.CallOption) (*QueuePushResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(QueuePushResponse)
err := c.cc.Invoke(ctx, WaymakerCollectionsService_QueuePush_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerCollectionsServiceClient) QueuePop(ctx context.Context, in *QueuePopRequest, opts ...grpc.CallOption) (*QueuePopResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(QueuePopResponse)
err := c.cc.Invoke(ctx, WaymakerCollectionsService_QueuePop_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerCollectionsServiceClient) QueueRange(ctx context.Context, in *QueueRangeRequest, opts ...grpc.CallOption) (*QueueRangeResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(QueueRangeResponse)
err := c.cc.Invoke(ctx, WaymakerCollectionsService_QueueRange_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerCollectionsServiceClient) QueueLen(ctx context.Context, in *QueueLenRequest, opts ...grpc.CallOption) (*QueueLenResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(QueueLenResponse)
err := c.cc.Invoke(ctx, WaymakerCollectionsService_QueueLen_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
// WaymakerCollectionsServiceServer is the server API for WaymakerCollectionsService service.
// All implementations must embed UnimplementedWaymakerCollectionsServiceServer
// for forward compatibility.
type WaymakerCollectionsServiceServer interface {
// ----- Hash -------------------------------------------------
CreateHashStore(context.Context, *CreateHashStoreRequest) (*CreateHashStoreResponse, error)
DeleteHashStore(context.Context, *DeleteHashStoreRequest) (*DeleteHashStoreResponse, error)
HashSet(context.Context, *HashSetRequest) (*HashSetResponse, error)
HashGet(context.Context, *HashGetRequest) (*HashGetResponse, error)
HashExists(context.Context, *HashExistsRequest) (*HashExistsResponse, error)
HashDelete(context.Context, *HashDeleteRequest) (*HashDeleteResponse, error)
HashGetAll(context.Context, *HashGetAllRequest) (*HashGetAllResponse, error)
HashFields(context.Context, *HashFieldsRequest) (*HashFieldsResponse, error)
HashLen(context.Context, *HashLenRequest) (*HashLenResponse, error)
// ----- Set --------------------------------------------------
CreateSetStore(context.Context, *CreateSetStoreRequest) (*CreateSetStoreResponse, error)
DeleteSetStore(context.Context, *DeleteSetStoreRequest) (*DeleteSetStoreResponse, error)
SetAdd(context.Context, *SetAddRequest) (*SetAddResponse, error)
SetRemove(context.Context, *SetRemoveRequest) (*SetRemoveResponse, error)
SetIsMember(context.Context, *SetIsMemberRequest) (*SetIsMemberResponse, error)
SetMembers(context.Context, *SetMembersRequest) (*SetMembersResponse, error)
SetLen(context.Context, *SetLenRequest) (*SetLenResponse, error)
// ----- Queue ------------------------------------------------
CreateQueue(context.Context, *CreateQueueRequest) (*CreateQueueResponse, error)
DeleteQueue(context.Context, *DeleteQueueRequest) (*DeleteQueueResponse, error)
QueuePush(context.Context, *QueuePushRequest) (*QueuePushResponse, error)
QueuePop(context.Context, *QueuePopRequest) (*QueuePopResponse, error)
QueueRange(context.Context, *QueueRangeRequest) (*QueueRangeResponse, error)
QueueLen(context.Context, *QueueLenRequest) (*QueueLenResponse, error)
mustEmbedUnimplementedWaymakerCollectionsServiceServer()
}
// UnimplementedWaymakerCollectionsServiceServer must be embedded to have
// forward compatible implementations.
//
// NOTE: this should be embedded by value instead of pointer to avoid a nil
// pointer dereference when methods are called.
type UnimplementedWaymakerCollectionsServiceServer struct{}
func (UnimplementedWaymakerCollectionsServiceServer) CreateHashStore(context.Context, *CreateHashStoreRequest) (*CreateHashStoreResponse, error) {
return nil, status.Error(codes.Unimplemented, "method CreateHashStore not implemented")
}
func (UnimplementedWaymakerCollectionsServiceServer) DeleteHashStore(context.Context, *DeleteHashStoreRequest) (*DeleteHashStoreResponse, error) {
return nil, status.Error(codes.Unimplemented, "method DeleteHashStore not implemented")
}
func (UnimplementedWaymakerCollectionsServiceServer) HashSet(context.Context, *HashSetRequest) (*HashSetResponse, error) {
return nil, status.Error(codes.Unimplemented, "method HashSet not implemented")
}
func (UnimplementedWaymakerCollectionsServiceServer) HashGet(context.Context, *HashGetRequest) (*HashGetResponse, error) {
return nil, status.Error(codes.Unimplemented, "method HashGet not implemented")
}
func (UnimplementedWaymakerCollectionsServiceServer) HashExists(context.Context, *HashExistsRequest) (*HashExistsResponse, error) {
return nil, status.Error(codes.Unimplemented, "method HashExists not implemented")
}
func (UnimplementedWaymakerCollectionsServiceServer) HashDelete(context.Context, *HashDeleteRequest) (*HashDeleteResponse, error) {
return nil, status.Error(codes.Unimplemented, "method HashDelete not implemented")
}
func (UnimplementedWaymakerCollectionsServiceServer) HashGetAll(context.Context, *HashGetAllRequest) (*HashGetAllResponse, error) {
return nil, status.Error(codes.Unimplemented, "method HashGetAll not implemented")
}
func (UnimplementedWaymakerCollectionsServiceServer) HashFields(context.Context, *HashFieldsRequest) (*HashFieldsResponse, error) {
return nil, status.Error(codes.Unimplemented, "method HashFields not implemented")
}
func (UnimplementedWaymakerCollectionsServiceServer) HashLen(context.Context, *HashLenRequest) (*HashLenResponse, error) {
return nil, status.Error(codes.Unimplemented, "method HashLen not implemented")
}
func (UnimplementedWaymakerCollectionsServiceServer) CreateSetStore(context.Context, *CreateSetStoreRequest) (*CreateSetStoreResponse, error) {
return nil, status.Error(codes.Unimplemented, "method CreateSetStore not implemented")
}
func (UnimplementedWaymakerCollectionsServiceServer) DeleteSetStore(context.Context, *DeleteSetStoreRequest) (*DeleteSetStoreResponse, error) {
return nil, status.Error(codes.Unimplemented, "method DeleteSetStore not implemented")
}
func (UnimplementedWaymakerCollectionsServiceServer) SetAdd(context.Context, *SetAddRequest) (*SetAddResponse, error) {
return nil, status.Error(codes.Unimplemented, "method SetAdd not implemented")
}
func (UnimplementedWaymakerCollectionsServiceServer) SetRemove(context.Context, *SetRemoveRequest) (*SetRemoveResponse, error) {
return nil, status.Error(codes.Unimplemented, "method SetRemove not implemented")
}
func (UnimplementedWaymakerCollectionsServiceServer) SetIsMember(context.Context, *SetIsMemberRequest) (*SetIsMemberResponse, error) {
return nil, status.Error(codes.Unimplemented, "method SetIsMember not implemented")
}
func (UnimplementedWaymakerCollectionsServiceServer) SetMembers(context.Context, *SetMembersRequest) (*SetMembersResponse, error) {
return nil, status.Error(codes.Unimplemented, "method SetMembers not implemented")
}
func (UnimplementedWaymakerCollectionsServiceServer) SetLen(context.Context, *SetLenRequest) (*SetLenResponse, error) {
return nil, status.Error(codes.Unimplemented, "method SetLen not implemented")
}
func (UnimplementedWaymakerCollectionsServiceServer) CreateQueue(context.Context, *CreateQueueRequest) (*CreateQueueResponse, error) {
return nil, status.Error(codes.Unimplemented, "method CreateQueue not implemented")
}
func (UnimplementedWaymakerCollectionsServiceServer) DeleteQueue(context.Context, *DeleteQueueRequest) (*DeleteQueueResponse, error) {
return nil, status.Error(codes.Unimplemented, "method DeleteQueue not implemented")
}
func (UnimplementedWaymakerCollectionsServiceServer) QueuePush(context.Context, *QueuePushRequest) (*QueuePushResponse, error) {
return nil, status.Error(codes.Unimplemented, "method QueuePush not implemented")
}
func (UnimplementedWaymakerCollectionsServiceServer) QueuePop(context.Context, *QueuePopRequest) (*QueuePopResponse, error) {
return nil, status.Error(codes.Unimplemented, "method QueuePop not implemented")
}
func (UnimplementedWaymakerCollectionsServiceServer) QueueRange(context.Context, *QueueRangeRequest) (*QueueRangeResponse, error) {
return nil, status.Error(codes.Unimplemented, "method QueueRange not implemented")
}
func (UnimplementedWaymakerCollectionsServiceServer) QueueLen(context.Context, *QueueLenRequest) (*QueueLenResponse, error) {
return nil, status.Error(codes.Unimplemented, "method QueueLen not implemented")
}
func (UnimplementedWaymakerCollectionsServiceServer) mustEmbedUnimplementedWaymakerCollectionsServiceServer() {
}
func (UnimplementedWaymakerCollectionsServiceServer) testEmbeddedByValue() {}
// UnsafeWaymakerCollectionsServiceServer may be embedded to opt out of forward compatibility for this service.
// Use of this interface is not recommended, as added methods to WaymakerCollectionsServiceServer will
// result in compilation errors.
type UnsafeWaymakerCollectionsServiceServer interface {
mustEmbedUnimplementedWaymakerCollectionsServiceServer()
}
func RegisterWaymakerCollectionsServiceServer(s grpc.ServiceRegistrar, srv WaymakerCollectionsServiceServer) {
// If the following call panics, it indicates UnimplementedWaymakerCollectionsServiceServer was
// embedded by pointer and is nil. This will cause panics if an
// unimplemented method is ever invoked, so we test this at initialization
// time to prevent it from happening at runtime later due to I/O.
if t, ok := srv.(interface{ testEmbeddedByValue() }); ok {
t.testEmbeddedByValue()
}
s.RegisterService(&WaymakerCollectionsService_ServiceDesc, srv)
}
func _WaymakerCollectionsService_CreateHashStore_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(CreateHashStoreRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerCollectionsServiceServer).CreateHashStore(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerCollectionsService_CreateHashStore_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerCollectionsServiceServer).CreateHashStore(ctx, req.(*CreateHashStoreRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerCollectionsService_DeleteHashStore_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(DeleteHashStoreRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerCollectionsServiceServer).DeleteHashStore(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerCollectionsService_DeleteHashStore_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerCollectionsServiceServer).DeleteHashStore(ctx, req.(*DeleteHashStoreRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerCollectionsService_HashSet_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(HashSetRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerCollectionsServiceServer).HashSet(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerCollectionsService_HashSet_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerCollectionsServiceServer).HashSet(ctx, req.(*HashSetRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerCollectionsService_HashGet_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(HashGetRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerCollectionsServiceServer).HashGet(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerCollectionsService_HashGet_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerCollectionsServiceServer).HashGet(ctx, req.(*HashGetRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerCollectionsService_HashExists_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(HashExistsRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerCollectionsServiceServer).HashExists(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerCollectionsService_HashExists_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerCollectionsServiceServer).HashExists(ctx, req.(*HashExistsRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerCollectionsService_HashDelete_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(HashDeleteRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerCollectionsServiceServer).HashDelete(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerCollectionsService_HashDelete_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerCollectionsServiceServer).HashDelete(ctx, req.(*HashDeleteRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerCollectionsService_HashGetAll_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(HashGetAllRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerCollectionsServiceServer).HashGetAll(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerCollectionsService_HashGetAll_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerCollectionsServiceServer).HashGetAll(ctx, req.(*HashGetAllRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerCollectionsService_HashFields_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(HashFieldsRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerCollectionsServiceServer).HashFields(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerCollectionsService_HashFields_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerCollectionsServiceServer).HashFields(ctx, req.(*HashFieldsRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerCollectionsService_HashLen_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(HashLenRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerCollectionsServiceServer).HashLen(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerCollectionsService_HashLen_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerCollectionsServiceServer).HashLen(ctx, req.(*HashLenRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerCollectionsService_CreateSetStore_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(CreateSetStoreRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerCollectionsServiceServer).CreateSetStore(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerCollectionsService_CreateSetStore_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerCollectionsServiceServer).CreateSetStore(ctx, req.(*CreateSetStoreRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerCollectionsService_DeleteSetStore_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(DeleteSetStoreRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerCollectionsServiceServer).DeleteSetStore(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerCollectionsService_DeleteSetStore_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerCollectionsServiceServer).DeleteSetStore(ctx, req.(*DeleteSetStoreRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerCollectionsService_SetAdd_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(SetAddRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerCollectionsServiceServer).SetAdd(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerCollectionsService_SetAdd_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerCollectionsServiceServer).SetAdd(ctx, req.(*SetAddRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerCollectionsService_SetRemove_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(SetRemoveRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerCollectionsServiceServer).SetRemove(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerCollectionsService_SetRemove_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerCollectionsServiceServer).SetRemove(ctx, req.(*SetRemoveRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerCollectionsService_SetIsMember_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(SetIsMemberRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerCollectionsServiceServer).SetIsMember(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerCollectionsService_SetIsMember_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerCollectionsServiceServer).SetIsMember(ctx, req.(*SetIsMemberRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerCollectionsService_SetMembers_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(SetMembersRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerCollectionsServiceServer).SetMembers(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerCollectionsService_SetMembers_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerCollectionsServiceServer).SetMembers(ctx, req.(*SetMembersRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerCollectionsService_SetLen_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(SetLenRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerCollectionsServiceServer).SetLen(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerCollectionsService_SetLen_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerCollectionsServiceServer).SetLen(ctx, req.(*SetLenRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerCollectionsService_CreateQueue_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(CreateQueueRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerCollectionsServiceServer).CreateQueue(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerCollectionsService_CreateQueue_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerCollectionsServiceServer).CreateQueue(ctx, req.(*CreateQueueRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerCollectionsService_DeleteQueue_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(DeleteQueueRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerCollectionsServiceServer).DeleteQueue(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerCollectionsService_DeleteQueue_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerCollectionsServiceServer).DeleteQueue(ctx, req.(*DeleteQueueRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerCollectionsService_QueuePush_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(QueuePushRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerCollectionsServiceServer).QueuePush(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerCollectionsService_QueuePush_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerCollectionsServiceServer).QueuePush(ctx, req.(*QueuePushRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerCollectionsService_QueuePop_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(QueuePopRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerCollectionsServiceServer).QueuePop(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerCollectionsService_QueuePop_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerCollectionsServiceServer).QueuePop(ctx, req.(*QueuePopRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerCollectionsService_QueueRange_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(QueueRangeRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerCollectionsServiceServer).QueueRange(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerCollectionsService_QueueRange_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerCollectionsServiceServer).QueueRange(ctx, req.(*QueueRangeRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerCollectionsService_QueueLen_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(QueueLenRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerCollectionsServiceServer).QueueLen(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerCollectionsService_QueueLen_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerCollectionsServiceServer).QueueLen(ctx, req.(*QueueLenRequest))
}
return interceptor(ctx, in, info, handler)
}
// WaymakerCollectionsService_ServiceDesc is the grpc.ServiceDesc for WaymakerCollectionsService service.
// It's only intended for direct use with grpc.RegisterService,
// and not to be introspected or modified (even as a copy)
var WaymakerCollectionsService_ServiceDesc = grpc.ServiceDesc{
ServiceName: "waymaker.collections.WaymakerCollectionsService",
HandlerType: (*WaymakerCollectionsServiceServer)(nil),
Methods: []grpc.MethodDesc{
{
MethodName: "CreateHashStore",
Handler: _WaymakerCollectionsService_CreateHashStore_Handler,
},
{
MethodName: "DeleteHashStore",
Handler: _WaymakerCollectionsService_DeleteHashStore_Handler,
},
{
MethodName: "HashSet",
Handler: _WaymakerCollectionsService_HashSet_Handler,
},
{
MethodName: "HashGet",
Handler: _WaymakerCollectionsService_HashGet_Handler,
},
{
MethodName: "HashExists",
Handler: _WaymakerCollectionsService_HashExists_Handler,
},
{
MethodName: "HashDelete",
Handler: _WaymakerCollectionsService_HashDelete_Handler,
},
{
MethodName: "HashGetAll",
Handler: _WaymakerCollectionsService_HashGetAll_Handler,
},
{
MethodName: "HashFields",
Handler: _WaymakerCollectionsService_HashFields_Handler,
},
{
MethodName: "HashLen",
Handler: _WaymakerCollectionsService_HashLen_Handler,
},
{
MethodName: "CreateSetStore",
Handler: _WaymakerCollectionsService_CreateSetStore_Handler,
},
{
MethodName: "DeleteSetStore",
Handler: _WaymakerCollectionsService_DeleteSetStore_Handler,
},
{
MethodName: "SetAdd",
Handler: _WaymakerCollectionsService_SetAdd_Handler,
},
{
MethodName: "SetRemove",
Handler: _WaymakerCollectionsService_SetRemove_Handler,
},
{
MethodName: "SetIsMember",
Handler: _WaymakerCollectionsService_SetIsMember_Handler,
},
{
MethodName: "SetMembers",
Handler: _WaymakerCollectionsService_SetMembers_Handler,
},
{
MethodName: "SetLen",
Handler: _WaymakerCollectionsService_SetLen_Handler,
},
{
MethodName: "CreateQueue",
Handler: _WaymakerCollectionsService_CreateQueue_Handler,
},
{
MethodName: "DeleteQueue",
Handler: _WaymakerCollectionsService_DeleteQueue_Handler,
},
{
MethodName: "QueuePush",
Handler: _WaymakerCollectionsService_QueuePush_Handler,
},
{
MethodName: "QueuePop",
Handler: _WaymakerCollectionsService_QueuePop_Handler,
},
{
MethodName: "QueueRange",
Handler: _WaymakerCollectionsService_QueueRange_Handler,
},
{
MethodName: "QueueLen",
Handler: _WaymakerCollectionsService_QueueLen_Handler,
},
},
Streams: []grpc.StreamDesc{},
Metadata: "collections.proto",
}

1797
go/genpb/kv/kv.pb.go Normal file

File diff suppressed because it is too large Load Diff

529
go/genpb/kv/kv_grpc.pb.go Normal file
View File

@ -0,0 +1,529 @@
// Code generated by protoc-gen-go-grpc. DO NOT EDIT.
// versions:
// - protoc-gen-go-grpc v1.6.2
// - protoc v7.34.1
// source: kv.proto
package waymaker_kv
import (
context "context"
grpc "google.golang.org/grpc"
codes "google.golang.org/grpc/codes"
status "google.golang.org/grpc/status"
)
// This is a compile-time assertion to ensure that this generated file
// is compatible with the grpc package it is being compiled against.
// Requires gRPC-Go v1.64.0 or later.
const _ = grpc.SupportPackageIsVersion9
const (
WaymakerKvService_CreateBucket_FullMethodName = "/waymaker.kv.WaymakerKvService/CreateBucket"
WaymakerKvService_DeleteBucket_FullMethodName = "/waymaker.kv.WaymakerKvService/DeleteBucket"
WaymakerKvService_Put_FullMethodName = "/waymaker.kv.WaymakerKvService/Put"
WaymakerKvService_Create_FullMethodName = "/waymaker.kv.WaymakerKvService/Create"
WaymakerKvService_Update_FullMethodName = "/waymaker.kv.WaymakerKvService/Update"
WaymakerKvService_Delete_FullMethodName = "/waymaker.kv.WaymakerKvService/Delete"
WaymakerKvService_Get_FullMethodName = "/waymaker.kv.WaymakerKvService/Get"
WaymakerKvService_Keys_FullMethodName = "/waymaker.kv.WaymakerKvService/Keys"
WaymakerKvService_History_FullMethodName = "/waymaker.kv.WaymakerKvService/History"
WaymakerKvService_Touch_FullMethodName = "/waymaker.kv.WaymakerKvService/Touch"
WaymakerKvService_Watch_FullMethodName = "/waymaker.kv.WaymakerKvService/Watch"
)
// WaymakerKvServiceClient is the client API for WaymakerKvService service.
//
// For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream.
type WaymakerKvServiceClient interface {
// ----- Bucket lifecycle -------------------------------------
CreateBucket(ctx context.Context, in *KvCreateBucketRequest, opts ...grpc.CallOption) (*KvCreateBucketResponse, error)
DeleteBucket(ctx context.Context, in *KvDeleteBucketRequest, opts ...grpc.CallOption) (*KvDeleteBucketResponse, error)
// ----- Mutations --------------------------------------------
Put(ctx context.Context, in *KvPutRequest, opts ...grpc.CallOption) (*KvPutResponse, error)
// CAS create — succeeds only when the key has never been
// written or its current value is a tombstone.
Create(ctx context.Context, in *KvCreateRequest, opts ...grpc.CallOption) (*KvPutResponse, error)
// CAS update — succeeds only when `expected_revision`
// matches the server-side revision.
Update(ctx context.Context, in *KvUpdateRequest, opts ...grpc.CallOption) (*KvPutResponse, error)
Delete(ctx context.Context, in *KvDeleteRequest, opts ...grpc.CallOption) (*KvDeleteResponse, error)
// ----- Reads ------------------------------------------------
Get(ctx context.Context, in *KvGetRequest, opts ...grpc.CallOption) (*KvGetResponse, error)
Keys(ctx context.Context, in *KvKeysRequest, opts ...grpc.CallOption) (*KvKeysResponse, error)
History(ctx context.Context, in *KvHistoryRequest, opts ...grpc.CallOption) (*KvHistoryResponse, error)
// ----- TTL refresh ------------------------------------------
Touch(ctx context.Context, in *KvTouchRequest, opts ...grpc.CallOption) (*KvPutResponse, error)
// ----- Watch ------------------------------------------------
// Server-streamed event flow for a single bucket. When `key`
// is empty, every put/delete in the bucket fans out; when
// `key` is set, only events at that key are emitted.
Watch(ctx context.Context, in *KvWatchRequest, opts ...grpc.CallOption) (grpc.ServerStreamingClient[KvWatchEvent], error)
}
type waymakerKvServiceClient struct {
cc grpc.ClientConnInterface
}
func NewWaymakerKvServiceClient(cc grpc.ClientConnInterface) WaymakerKvServiceClient {
return &waymakerKvServiceClient{cc}
}
func (c *waymakerKvServiceClient) CreateBucket(ctx context.Context, in *KvCreateBucketRequest, opts ...grpc.CallOption) (*KvCreateBucketResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(KvCreateBucketResponse)
err := c.cc.Invoke(ctx, WaymakerKvService_CreateBucket_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerKvServiceClient) DeleteBucket(ctx context.Context, in *KvDeleteBucketRequest, opts ...grpc.CallOption) (*KvDeleteBucketResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(KvDeleteBucketResponse)
err := c.cc.Invoke(ctx, WaymakerKvService_DeleteBucket_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerKvServiceClient) Put(ctx context.Context, in *KvPutRequest, opts ...grpc.CallOption) (*KvPutResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(KvPutResponse)
err := c.cc.Invoke(ctx, WaymakerKvService_Put_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerKvServiceClient) Create(ctx context.Context, in *KvCreateRequest, opts ...grpc.CallOption) (*KvPutResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(KvPutResponse)
err := c.cc.Invoke(ctx, WaymakerKvService_Create_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerKvServiceClient) Update(ctx context.Context, in *KvUpdateRequest, opts ...grpc.CallOption) (*KvPutResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(KvPutResponse)
err := c.cc.Invoke(ctx, WaymakerKvService_Update_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerKvServiceClient) Delete(ctx context.Context, in *KvDeleteRequest, opts ...grpc.CallOption) (*KvDeleteResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(KvDeleteResponse)
err := c.cc.Invoke(ctx, WaymakerKvService_Delete_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerKvServiceClient) Get(ctx context.Context, in *KvGetRequest, opts ...grpc.CallOption) (*KvGetResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(KvGetResponse)
err := c.cc.Invoke(ctx, WaymakerKvService_Get_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerKvServiceClient) Keys(ctx context.Context, in *KvKeysRequest, opts ...grpc.CallOption) (*KvKeysResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(KvKeysResponse)
err := c.cc.Invoke(ctx, WaymakerKvService_Keys_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerKvServiceClient) History(ctx context.Context, in *KvHistoryRequest, opts ...grpc.CallOption) (*KvHistoryResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(KvHistoryResponse)
err := c.cc.Invoke(ctx, WaymakerKvService_History_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerKvServiceClient) Touch(ctx context.Context, in *KvTouchRequest, opts ...grpc.CallOption) (*KvPutResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(KvPutResponse)
err := c.cc.Invoke(ctx, WaymakerKvService_Touch_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerKvServiceClient) Watch(ctx context.Context, in *KvWatchRequest, opts ...grpc.CallOption) (grpc.ServerStreamingClient[KvWatchEvent], error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
stream, err := c.cc.NewStream(ctx, &WaymakerKvService_ServiceDesc.Streams[0], WaymakerKvService_Watch_FullMethodName, cOpts...)
if err != nil {
return nil, err
}
x := &grpc.GenericClientStream[KvWatchRequest, KvWatchEvent]{ClientStream: stream}
if err := x.ClientStream.SendMsg(in); err != nil {
return nil, err
}
if err := x.ClientStream.CloseSend(); err != nil {
return nil, err
}
return x, nil
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type WaymakerKvService_WatchClient = grpc.ServerStreamingClient[KvWatchEvent]
// WaymakerKvServiceServer is the server API for WaymakerKvService service.
// All implementations must embed UnimplementedWaymakerKvServiceServer
// for forward compatibility.
type WaymakerKvServiceServer interface {
// ----- Bucket lifecycle -------------------------------------
CreateBucket(context.Context, *KvCreateBucketRequest) (*KvCreateBucketResponse, error)
DeleteBucket(context.Context, *KvDeleteBucketRequest) (*KvDeleteBucketResponse, error)
// ----- Mutations --------------------------------------------
Put(context.Context, *KvPutRequest) (*KvPutResponse, error)
// CAS create — succeeds only when the key has never been
// written or its current value is a tombstone.
Create(context.Context, *KvCreateRequest) (*KvPutResponse, error)
// CAS update — succeeds only when `expected_revision`
// matches the server-side revision.
Update(context.Context, *KvUpdateRequest) (*KvPutResponse, error)
Delete(context.Context, *KvDeleteRequest) (*KvDeleteResponse, error)
// ----- Reads ------------------------------------------------
Get(context.Context, *KvGetRequest) (*KvGetResponse, error)
Keys(context.Context, *KvKeysRequest) (*KvKeysResponse, error)
History(context.Context, *KvHistoryRequest) (*KvHistoryResponse, error)
// ----- TTL refresh ------------------------------------------
Touch(context.Context, *KvTouchRequest) (*KvPutResponse, error)
// ----- Watch ------------------------------------------------
// Server-streamed event flow for a single bucket. When `key`
// is empty, every put/delete in the bucket fans out; when
// `key` is set, only events at that key are emitted.
Watch(*KvWatchRequest, grpc.ServerStreamingServer[KvWatchEvent]) error
mustEmbedUnimplementedWaymakerKvServiceServer()
}
// UnimplementedWaymakerKvServiceServer must be embedded to have
// forward compatible implementations.
//
// NOTE: this should be embedded by value instead of pointer to avoid a nil
// pointer dereference when methods are called.
type UnimplementedWaymakerKvServiceServer struct{}
func (UnimplementedWaymakerKvServiceServer) CreateBucket(context.Context, *KvCreateBucketRequest) (*KvCreateBucketResponse, error) {
return nil, status.Error(codes.Unimplemented, "method CreateBucket not implemented")
}
func (UnimplementedWaymakerKvServiceServer) DeleteBucket(context.Context, *KvDeleteBucketRequest) (*KvDeleteBucketResponse, error) {
return nil, status.Error(codes.Unimplemented, "method DeleteBucket not implemented")
}
func (UnimplementedWaymakerKvServiceServer) Put(context.Context, *KvPutRequest) (*KvPutResponse, error) {
return nil, status.Error(codes.Unimplemented, "method Put not implemented")
}
func (UnimplementedWaymakerKvServiceServer) Create(context.Context, *KvCreateRequest) (*KvPutResponse, error) {
return nil, status.Error(codes.Unimplemented, "method Create not implemented")
}
func (UnimplementedWaymakerKvServiceServer) Update(context.Context, *KvUpdateRequest) (*KvPutResponse, error) {
return nil, status.Error(codes.Unimplemented, "method Update not implemented")
}
func (UnimplementedWaymakerKvServiceServer) Delete(context.Context, *KvDeleteRequest) (*KvDeleteResponse, error) {
return nil, status.Error(codes.Unimplemented, "method Delete not implemented")
}
func (UnimplementedWaymakerKvServiceServer) Get(context.Context, *KvGetRequest) (*KvGetResponse, error) {
return nil, status.Error(codes.Unimplemented, "method Get not implemented")
}
func (UnimplementedWaymakerKvServiceServer) Keys(context.Context, *KvKeysRequest) (*KvKeysResponse, error) {
return nil, status.Error(codes.Unimplemented, "method Keys not implemented")
}
func (UnimplementedWaymakerKvServiceServer) History(context.Context, *KvHistoryRequest) (*KvHistoryResponse, error) {
return nil, status.Error(codes.Unimplemented, "method History not implemented")
}
func (UnimplementedWaymakerKvServiceServer) Touch(context.Context, *KvTouchRequest) (*KvPutResponse, error) {
return nil, status.Error(codes.Unimplemented, "method Touch not implemented")
}
func (UnimplementedWaymakerKvServiceServer) Watch(*KvWatchRequest, grpc.ServerStreamingServer[KvWatchEvent]) error {
return status.Error(codes.Unimplemented, "method Watch not implemented")
}
func (UnimplementedWaymakerKvServiceServer) mustEmbedUnimplementedWaymakerKvServiceServer() {}
func (UnimplementedWaymakerKvServiceServer) testEmbeddedByValue() {}
// UnsafeWaymakerKvServiceServer may be embedded to opt out of forward compatibility for this service.
// Use of this interface is not recommended, as added methods to WaymakerKvServiceServer will
// result in compilation errors.
type UnsafeWaymakerKvServiceServer interface {
mustEmbedUnimplementedWaymakerKvServiceServer()
}
func RegisterWaymakerKvServiceServer(s grpc.ServiceRegistrar, srv WaymakerKvServiceServer) {
// If the following call panics, it indicates UnimplementedWaymakerKvServiceServer was
// embedded by pointer and is nil. This will cause panics if an
// unimplemented method is ever invoked, so we test this at initialization
// time to prevent it from happening at runtime later due to I/O.
if t, ok := srv.(interface{ testEmbeddedByValue() }); ok {
t.testEmbeddedByValue()
}
s.RegisterService(&WaymakerKvService_ServiceDesc, srv)
}
func _WaymakerKvService_CreateBucket_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(KvCreateBucketRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerKvServiceServer).CreateBucket(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerKvService_CreateBucket_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerKvServiceServer).CreateBucket(ctx, req.(*KvCreateBucketRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerKvService_DeleteBucket_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(KvDeleteBucketRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerKvServiceServer).DeleteBucket(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerKvService_DeleteBucket_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerKvServiceServer).DeleteBucket(ctx, req.(*KvDeleteBucketRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerKvService_Put_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(KvPutRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerKvServiceServer).Put(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerKvService_Put_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerKvServiceServer).Put(ctx, req.(*KvPutRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerKvService_Create_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(KvCreateRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerKvServiceServer).Create(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerKvService_Create_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerKvServiceServer).Create(ctx, req.(*KvCreateRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerKvService_Update_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(KvUpdateRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerKvServiceServer).Update(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerKvService_Update_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerKvServiceServer).Update(ctx, req.(*KvUpdateRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerKvService_Delete_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(KvDeleteRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerKvServiceServer).Delete(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerKvService_Delete_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerKvServiceServer).Delete(ctx, req.(*KvDeleteRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerKvService_Get_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(KvGetRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerKvServiceServer).Get(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerKvService_Get_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerKvServiceServer).Get(ctx, req.(*KvGetRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerKvService_Keys_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(KvKeysRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerKvServiceServer).Keys(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerKvService_Keys_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerKvServiceServer).Keys(ctx, req.(*KvKeysRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerKvService_History_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(KvHistoryRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerKvServiceServer).History(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerKvService_History_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerKvServiceServer).History(ctx, req.(*KvHistoryRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerKvService_Touch_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(KvTouchRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerKvServiceServer).Touch(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerKvService_Touch_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerKvServiceServer).Touch(ctx, req.(*KvTouchRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerKvService_Watch_Handler(srv interface{}, stream grpc.ServerStream) error {
m := new(KvWatchRequest)
if err := stream.RecvMsg(m); err != nil {
return err
}
return srv.(WaymakerKvServiceServer).Watch(m, &grpc.GenericServerStream[KvWatchRequest, KvWatchEvent]{ServerStream: stream})
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type WaymakerKvService_WatchServer = grpc.ServerStreamingServer[KvWatchEvent]
// WaymakerKvService_ServiceDesc is the grpc.ServiceDesc for WaymakerKvService service.
// It's only intended for direct use with grpc.RegisterService,
// and not to be introspected or modified (even as a copy)
var WaymakerKvService_ServiceDesc = grpc.ServiceDesc{
ServiceName: "waymaker.kv.WaymakerKvService",
HandlerType: (*WaymakerKvServiceServer)(nil),
Methods: []grpc.MethodDesc{
{
MethodName: "CreateBucket",
Handler: _WaymakerKvService_CreateBucket_Handler,
},
{
MethodName: "DeleteBucket",
Handler: _WaymakerKvService_DeleteBucket_Handler,
},
{
MethodName: "Put",
Handler: _WaymakerKvService_Put_Handler,
},
{
MethodName: "Create",
Handler: _WaymakerKvService_Create_Handler,
},
{
MethodName: "Update",
Handler: _WaymakerKvService_Update_Handler,
},
{
MethodName: "Delete",
Handler: _WaymakerKvService_Delete_Handler,
},
{
MethodName: "Get",
Handler: _WaymakerKvService_Get_Handler,
},
{
MethodName: "Keys",
Handler: _WaymakerKvService_Keys_Handler,
},
{
MethodName: "History",
Handler: _WaymakerKvService_History_Handler,
},
{
MethodName: "Touch",
Handler: _WaymakerKvService_Touch_Handler,
},
},
Streams: []grpc.StreamDesc{
{
StreamName: "Watch",
Handler: _WaymakerKvService_Watch_Handler,
ServerStreams: true,
},
},
Metadata: "kv.proto",
}

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,414 @@
// Code generated by protoc-gen-go-grpc. DO NOT EDIT.
// versions:
// - protoc-gen-go-grpc v1.6.2
// - protoc v7.34.1
// source: waymaker_locks.proto
package waymaker
import (
context "context"
grpc "google.golang.org/grpc"
codes "google.golang.org/grpc/codes"
status "google.golang.org/grpc/status"
)
// This is a compile-time assertion to ensure that this generated file
// is compatible with the grpc package it is being compiled against.
// Requires gRPC-Go v1.64.0 or later.
const _ = grpc.SupportPackageIsVersion9
const (
WaymakerService_Lock_FullMethodName = "/waymaker.WaymakerService/Lock"
WaymakerService_ReadLock_FullMethodName = "/waymaker.WaymakerService/ReadLock"
WaymakerService_UnLock_FullMethodName = "/waymaker.WaymakerService/UnLock"
WaymakerService_LeaseStatus_FullMethodName = "/waymaker.WaymakerService/LeaseStatus"
WaymakerService_ExtendLease_FullMethodName = "/waymaker.WaymakerService/ExtendLease"
WaymakerService_MultiLock_FullMethodName = "/waymaker.WaymakerService/MultiLock"
WaymakerService_ListAcquiredLocks_FullMethodName = "/waymaker.WaymakerService/ListAcquiredLocks"
)
// WaymakerServiceClient is the client API for WaymakerService service.
//
// For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream.
//
// WaymakerService defines a gRPC service for managing distributed locks.
type WaymakerServiceClient interface {
// Lock attempts to acquire a lock based on the provided LockRequest.
// The response is a stream of LockEvent messages that provide updates
// on the status of the lock acquisition.
Lock(ctx context.Context, in *LockRequest, opts ...grpc.CallOption) (grpc.ServerStreamingClient[LockEvent], error)
// ReadLock attempts to acquire a read lock, which allows multiple readers
// but no writers to hold the lock simultaneously. The response is a stream
// of LockEvent messages that provide updates on the status of the lock acquisition.
ReadLock(ctx context.Context, in *LockRequest, opts ...grpc.CallOption) (grpc.ServerStreamingClient[LockEvent], error)
// UnLock releases a previously acquired lock based on the provided UnLockRequest.
// The response is an UnLockResponse indicating the success or failure of the operation.
UnLock(ctx context.Context, in *UnLockRequest, opts ...grpc.CallOption) (*UnLockResponse, error)
// LeaseStatus retrieves the current status of a lock lease based on the provided
// LeaseStatusRequest. The response is a LeaseStatusResponse containing details
// about the lease.
LeaseStatus(ctx context.Context, in *LeaseStatusRequest, opts ...grpc.CallOption) (*LeaseStatusResponse, error)
// ExtendLease extends the lease of an already acquired lock based on the provided
// ExtendLeaseRequest. The response is an ExtendLeaseResponse indicating the success
// or failure of the operation and the updated lease information.
ExtendLease(ctx context.Context, in *ExtendLeaseRequest, opts ...grpc.CallOption) (*ExtendLeaseResponse, error)
// MultiLock acquires N locks atomically — all keys are granted or none are.
// The server sorts keys lexicographically to guarantee deadlock-free
// ordering between any two MultiLock callers (without this, callers asking
// for (k1,k2) and (k2,k1) could deadlock under contention). `max_wait_period`
// applies to the whole batch as a single deadline, not per key. On any
// failure the server releases every key it already acquired in this batch
// before returning. Returns a unary response — no streaming.
MultiLock(ctx context.Context, in *MultiLockRequest, opts ...grpc.CallOption) (*MultiLockResponse, error)
// ListAcquiredLocks returns every lock CURRENTLY HELD on the node serving the
// request (the node that owns each key via the consistent-hash ring). It is a
// read-only operator/introspection surface (waymaker-ctl `locks list`); it does
// NOT include waiters. An optional `key_prefix` filters the result server-side.
// In a multi-node cluster, call it on each node to see the full picture, since
// each node only holds the locks for the keys it owns.
ListAcquiredLocks(ctx context.Context, in *ListAcquiredLocksRequest, opts ...grpc.CallOption) (*ListAcquiredLocksResponse, error)
}
type waymakerServiceClient struct {
cc grpc.ClientConnInterface
}
func NewWaymakerServiceClient(cc grpc.ClientConnInterface) WaymakerServiceClient {
return &waymakerServiceClient{cc}
}
func (c *waymakerServiceClient) Lock(ctx context.Context, in *LockRequest, opts ...grpc.CallOption) (grpc.ServerStreamingClient[LockEvent], error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
stream, err := c.cc.NewStream(ctx, &WaymakerService_ServiceDesc.Streams[0], WaymakerService_Lock_FullMethodName, cOpts...)
if err != nil {
return nil, err
}
x := &grpc.GenericClientStream[LockRequest, LockEvent]{ClientStream: stream}
if err := x.ClientStream.SendMsg(in); err != nil {
return nil, err
}
if err := x.ClientStream.CloseSend(); err != nil {
return nil, err
}
return x, nil
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type WaymakerService_LockClient = grpc.ServerStreamingClient[LockEvent]
func (c *waymakerServiceClient) ReadLock(ctx context.Context, in *LockRequest, opts ...grpc.CallOption) (grpc.ServerStreamingClient[LockEvent], error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
stream, err := c.cc.NewStream(ctx, &WaymakerService_ServiceDesc.Streams[1], WaymakerService_ReadLock_FullMethodName, cOpts...)
if err != nil {
return nil, err
}
x := &grpc.GenericClientStream[LockRequest, LockEvent]{ClientStream: stream}
if err := x.ClientStream.SendMsg(in); err != nil {
return nil, err
}
if err := x.ClientStream.CloseSend(); err != nil {
return nil, err
}
return x, nil
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type WaymakerService_ReadLockClient = grpc.ServerStreamingClient[LockEvent]
func (c *waymakerServiceClient) UnLock(ctx context.Context, in *UnLockRequest, opts ...grpc.CallOption) (*UnLockResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(UnLockResponse)
err := c.cc.Invoke(ctx, WaymakerService_UnLock_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerServiceClient) LeaseStatus(ctx context.Context, in *LeaseStatusRequest, opts ...grpc.CallOption) (*LeaseStatusResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(LeaseStatusResponse)
err := c.cc.Invoke(ctx, WaymakerService_LeaseStatus_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerServiceClient) ExtendLease(ctx context.Context, in *ExtendLeaseRequest, opts ...grpc.CallOption) (*ExtendLeaseResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(ExtendLeaseResponse)
err := c.cc.Invoke(ctx, WaymakerService_ExtendLease_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerServiceClient) MultiLock(ctx context.Context, in *MultiLockRequest, opts ...grpc.CallOption) (*MultiLockResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(MultiLockResponse)
err := c.cc.Invoke(ctx, WaymakerService_MultiLock_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *waymakerServiceClient) ListAcquiredLocks(ctx context.Context, in *ListAcquiredLocksRequest, opts ...grpc.CallOption) (*ListAcquiredLocksResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(ListAcquiredLocksResponse)
err := c.cc.Invoke(ctx, WaymakerService_ListAcquiredLocks_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
// WaymakerServiceServer is the server API for WaymakerService service.
// All implementations must embed UnimplementedWaymakerServiceServer
// for forward compatibility.
//
// WaymakerService defines a gRPC service for managing distributed locks.
type WaymakerServiceServer interface {
// Lock attempts to acquire a lock based on the provided LockRequest.
// The response is a stream of LockEvent messages that provide updates
// on the status of the lock acquisition.
Lock(*LockRequest, grpc.ServerStreamingServer[LockEvent]) error
// ReadLock attempts to acquire a read lock, which allows multiple readers
// but no writers to hold the lock simultaneously. The response is a stream
// of LockEvent messages that provide updates on the status of the lock acquisition.
ReadLock(*LockRequest, grpc.ServerStreamingServer[LockEvent]) error
// UnLock releases a previously acquired lock based on the provided UnLockRequest.
// The response is an UnLockResponse indicating the success or failure of the operation.
UnLock(context.Context, *UnLockRequest) (*UnLockResponse, error)
// LeaseStatus retrieves the current status of a lock lease based on the provided
// LeaseStatusRequest. The response is a LeaseStatusResponse containing details
// about the lease.
LeaseStatus(context.Context, *LeaseStatusRequest) (*LeaseStatusResponse, error)
// ExtendLease extends the lease of an already acquired lock based on the provided
// ExtendLeaseRequest. The response is an ExtendLeaseResponse indicating the success
// or failure of the operation and the updated lease information.
ExtendLease(context.Context, *ExtendLeaseRequest) (*ExtendLeaseResponse, error)
// MultiLock acquires N locks atomically — all keys are granted or none are.
// The server sorts keys lexicographically to guarantee deadlock-free
// ordering between any two MultiLock callers (without this, callers asking
// for (k1,k2) and (k2,k1) could deadlock under contention). `max_wait_period`
// applies to the whole batch as a single deadline, not per key. On any
// failure the server releases every key it already acquired in this batch
// before returning. Returns a unary response — no streaming.
MultiLock(context.Context, *MultiLockRequest) (*MultiLockResponse, error)
// ListAcquiredLocks returns every lock CURRENTLY HELD on the node serving the
// request (the node that owns each key via the consistent-hash ring). It is a
// read-only operator/introspection surface (waymaker-ctl `locks list`); it does
// NOT include waiters. An optional `key_prefix` filters the result server-side.
// In a multi-node cluster, call it on each node to see the full picture, since
// each node only holds the locks for the keys it owns.
ListAcquiredLocks(context.Context, *ListAcquiredLocksRequest) (*ListAcquiredLocksResponse, error)
mustEmbedUnimplementedWaymakerServiceServer()
}
// UnimplementedWaymakerServiceServer must be embedded to have
// forward compatible implementations.
//
// NOTE: this should be embedded by value instead of pointer to avoid a nil
// pointer dereference when methods are called.
type UnimplementedWaymakerServiceServer struct{}
func (UnimplementedWaymakerServiceServer) Lock(*LockRequest, grpc.ServerStreamingServer[LockEvent]) error {
return status.Error(codes.Unimplemented, "method Lock not implemented")
}
func (UnimplementedWaymakerServiceServer) ReadLock(*LockRequest, grpc.ServerStreamingServer[LockEvent]) error {
return status.Error(codes.Unimplemented, "method ReadLock not implemented")
}
func (UnimplementedWaymakerServiceServer) UnLock(context.Context, *UnLockRequest) (*UnLockResponse, error) {
return nil, status.Error(codes.Unimplemented, "method UnLock not implemented")
}
func (UnimplementedWaymakerServiceServer) LeaseStatus(context.Context, *LeaseStatusRequest) (*LeaseStatusResponse, error) {
return nil, status.Error(codes.Unimplemented, "method LeaseStatus not implemented")
}
func (UnimplementedWaymakerServiceServer) ExtendLease(context.Context, *ExtendLeaseRequest) (*ExtendLeaseResponse, error) {
return nil, status.Error(codes.Unimplemented, "method ExtendLease not implemented")
}
func (UnimplementedWaymakerServiceServer) MultiLock(context.Context, *MultiLockRequest) (*MultiLockResponse, error) {
return nil, status.Error(codes.Unimplemented, "method MultiLock not implemented")
}
func (UnimplementedWaymakerServiceServer) ListAcquiredLocks(context.Context, *ListAcquiredLocksRequest) (*ListAcquiredLocksResponse, error) {
return nil, status.Error(codes.Unimplemented, "method ListAcquiredLocks not implemented")
}
func (UnimplementedWaymakerServiceServer) mustEmbedUnimplementedWaymakerServiceServer() {}
func (UnimplementedWaymakerServiceServer) testEmbeddedByValue() {}
// UnsafeWaymakerServiceServer may be embedded to opt out of forward compatibility for this service.
// Use of this interface is not recommended, as added methods to WaymakerServiceServer will
// result in compilation errors.
type UnsafeWaymakerServiceServer interface {
mustEmbedUnimplementedWaymakerServiceServer()
}
func RegisterWaymakerServiceServer(s grpc.ServiceRegistrar, srv WaymakerServiceServer) {
// If the following call panics, it indicates UnimplementedWaymakerServiceServer was
// embedded by pointer and is nil. This will cause panics if an
// unimplemented method is ever invoked, so we test this at initialization
// time to prevent it from happening at runtime later due to I/O.
if t, ok := srv.(interface{ testEmbeddedByValue() }); ok {
t.testEmbeddedByValue()
}
s.RegisterService(&WaymakerService_ServiceDesc, srv)
}
func _WaymakerService_Lock_Handler(srv interface{}, stream grpc.ServerStream) error {
m := new(LockRequest)
if err := stream.RecvMsg(m); err != nil {
return err
}
return srv.(WaymakerServiceServer).Lock(m, &grpc.GenericServerStream[LockRequest, LockEvent]{ServerStream: stream})
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type WaymakerService_LockServer = grpc.ServerStreamingServer[LockEvent]
func _WaymakerService_ReadLock_Handler(srv interface{}, stream grpc.ServerStream) error {
m := new(LockRequest)
if err := stream.RecvMsg(m); err != nil {
return err
}
return srv.(WaymakerServiceServer).ReadLock(m, &grpc.GenericServerStream[LockRequest, LockEvent]{ServerStream: stream})
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type WaymakerService_ReadLockServer = grpc.ServerStreamingServer[LockEvent]
func _WaymakerService_UnLock_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(UnLockRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerServiceServer).UnLock(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerService_UnLock_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerServiceServer).UnLock(ctx, req.(*UnLockRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerService_LeaseStatus_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(LeaseStatusRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerServiceServer).LeaseStatus(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerService_LeaseStatus_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerServiceServer).LeaseStatus(ctx, req.(*LeaseStatusRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerService_ExtendLease_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(ExtendLeaseRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerServiceServer).ExtendLease(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerService_ExtendLease_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerServiceServer).ExtendLease(ctx, req.(*ExtendLeaseRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerService_MultiLock_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(MultiLockRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerServiceServer).MultiLock(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerService_MultiLock_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerServiceServer).MultiLock(ctx, req.(*MultiLockRequest))
}
return interceptor(ctx, in, info, handler)
}
func _WaymakerService_ListAcquiredLocks_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(ListAcquiredLocksRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(WaymakerServiceServer).ListAcquiredLocks(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: WaymakerService_ListAcquiredLocks_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(WaymakerServiceServer).ListAcquiredLocks(ctx, req.(*ListAcquiredLocksRequest))
}
return interceptor(ctx, in, info, handler)
}
// WaymakerService_ServiceDesc is the grpc.ServiceDesc for WaymakerService service.
// It's only intended for direct use with grpc.RegisterService,
// and not to be introspected or modified (even as a copy)
var WaymakerService_ServiceDesc = grpc.ServiceDesc{
ServiceName: "waymaker.WaymakerService",
HandlerType: (*WaymakerServiceServer)(nil),
Methods: []grpc.MethodDesc{
{
MethodName: "UnLock",
Handler: _WaymakerService_UnLock_Handler,
},
{
MethodName: "LeaseStatus",
Handler: _WaymakerService_LeaseStatus_Handler,
},
{
MethodName: "ExtendLease",
Handler: _WaymakerService_ExtendLease_Handler,
},
{
MethodName: "MultiLock",
Handler: _WaymakerService_MultiLock_Handler,
},
{
MethodName: "ListAcquiredLocks",
Handler: _WaymakerService_ListAcquiredLocks_Handler,
},
},
Streams: []grpc.StreamDesc{
{
StreamName: "Lock",
Handler: _WaymakerService_Lock_Handler,
ServerStreams: true,
},
{
StreamName: "ReadLock",
Handler: _WaymakerService_ReadLock_Handler,
ServerStreams: true,
},
},
Metadata: "waymaker_locks.proto",
}

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

15
go/go.mod Normal file
View File

@ -0,0 +1,15 @@
module git.awesomike.com/pub/waymaker-client/go
go 1.26.2
require (
google.golang.org/grpc v1.81.1
google.golang.org/protobuf v1.36.11
)
require (
golang.org/x/net v0.51.0 // indirect
golang.org/x/sys v0.42.0 // indirect
golang.org/x/text v0.34.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20260226221140-a57be14db171 // indirect
)

38
go/go.sum Normal file
View File

@ -0,0 +1,38 @@
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=
go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=
go.opentelemetry.io/otel v1.43.0 h1:mYIM03dnh5zfN7HautFE4ieIig9amkNANT+xcVxAj9I=
go.opentelemetry.io/otel v1.43.0/go.mod h1:JuG+u74mvjvcm8vj8pI5XiHy1zDeoCS2LB1spIq7Ay0=
go.opentelemetry.io/otel/metric v1.43.0 h1:d7638QeInOnuwOONPp4JAOGfbCEpYb+K6DVWvdxGzgM=
go.opentelemetry.io/otel/metric v1.43.0/go.mod h1:RDnPtIxvqlgO8GRW18W6Z/4P462ldprJtfxHxyKd2PY=
go.opentelemetry.io/otel/sdk v1.43.0 h1:pi5mE86i5rTeLXqoF/hhiBtUNcrAGHLKQdhg4h4V9Dg=
go.opentelemetry.io/otel/sdk v1.43.0/go.mod h1:P+IkVU3iWukmiit/Yf9AWvpyRDlUeBaRg6Y+C58QHzg=
go.opentelemetry.io/otel/sdk/metric v1.43.0 h1:S88dyqXjJkuBNLeMcVPRFXpRw2fuwdvfCGLEo89fDkw=
go.opentelemetry.io/otel/sdk/metric v1.43.0/go.mod h1:C/RJtwSEJ5hzTiUz5pXF1kILHStzb9zFlIEe85bhj6A=
go.opentelemetry.io/otel/trace v1.43.0 h1:BkNrHpup+4k4w+ZZ86CZoHHEkohws8AY+WTX09nk+3A=
go.opentelemetry.io/otel/trace v1.43.0/go.mod h1:/QJhyVBUUswCphDVxq+8mld+AvhXZLhe+8WVFxiFff0=
golang.org/x/net v0.51.0 h1:94R/GTO7mt3/4wIKpcR5gkGmRLOuE/2hNGeWq/GBIFo=
golang.org/x/net v0.51.0/go.mod h1:aamm+2QF5ogm02fjy5Bb7CQ0WMt1/WVM7FtyaTLlA9Y=
golang.org/x/sys v0.42.0 h1:omrd2nAlyT5ESRdCLYdm3+fMfNFE/+Rf4bDIQImRJeo=
golang.org/x/sys v0.42.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/text v0.34.0 h1:oL/Qq0Kdaqxa1KbNeMKwQq0reLCCaFtqu2eNuSeNHbk=
golang.org/x/text v0.34.0/go.mod h1:homfLqTYRFyVYemLBFl5GgL/DWEiH5wcsQ5gSh1yziA=
gonum.org/v1/gonum v0.17.0 h1:VbpOemQlsSMrYmn7T2OUvQ4dqxQXU+ouZFQsZOx50z4=
gonum.org/v1/gonum v0.17.0/go.mod h1:El3tOrEuMpv2UdMrbNlKEh9vd86bmQ6vqIcDwxEOc1E=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260226221140-a57be14db171 h1:ggcbiqK8WWh6l1dnltU4BgWGIGo+EVYxCaAPih/zQXQ=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260226221140-a57be14db171/go.mod h1:4Hqkh8ycfw05ld/3BWL7rJOSfebL2Q+DVDeRgYgxUU8=
google.golang.org/grpc v1.81.1 h1:VnnIIZ88UzOOKLukQi+ImGz8O1Wdp8nAGGnvOfEIWQQ=
google.golang.org/grpc v1.81.1/go.mod h1:xGH9GfzOyMTGIOXBJmXt+BX/V0kcdQbdcuwQ/zNw42I=
google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=
google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=

348
go/kv.go Normal file
View File

@ -0,0 +1,348 @@
package waymaker
// KV subsystem — thin RPC binding over the server's Kv* RPCs.
// All conventions (subject patterns, tombstone marker, TTL header) live
// server-side in waymaker_streams::wire_conventions. This client just
// calls the typed RPCs — no subject-level knowledge required.
//
// Entry points on *Client:
// - client.CreateKV(ctx, KVConfig{…})
// - client.GetOrCreateKV(ctx, KVConfig{…})
// - client.KV(name) — handle without creation
// - client.DeleteKV(ctx, name)
import (
"context"
"io"
"time"
pb "git.awesomike.com/pub/waymaker-client/go/genpb/kv"
)
// KVConfig is the bucket creation config.
type KVConfig struct {
Name string
MaxBytes *uint64
MaxValueSize *uint64
MaxAge *time.Duration
Ephemeral bool
// MaxRevisions caps per-key revision count. 0 = unbounded.
MaxRevisions uint64
}
// Bucket is a reference to a KV bucket. Cheap to copy.
type Bucket struct {
client *Client
Name string
}
func newBucket(c *Client, name string) *Bucket {
return &Bucket{client: c, Name: name}
}
// Put stores value under key. Latest-write-wins. Returns the new revision.
func (b *Bucket) Put(ctx context.Context, key string, value []byte) (uint64, error) {
return b.putInternal(ctx, key, value, 0)
}
// PutWithTTL stores value under key with a per-key TTL.
func (b *Bucket) PutWithTTL(ctx context.Context, key string, value []byte, ttl time.Duration) (uint64, error) {
return b.putInternal(ctx, key, value, uint64(ttl.Milliseconds()))
}
func (b *Bucket) putInternal(ctx context.Context, key string, value []byte, ttlMs uint64) (uint64, error) {
c := b.client.kvClient()
r, err := c.Put(ctx, &pb.KvPutRequest{
Bucket: b.Name,
Key: key,
Value: value,
TtlMs: ttlMs,
})
if err != nil {
return 0, rpcErr(err)
}
if !r.GetSuccess() {
return 0, serverErr(r.GetResultCode(), r.GetMessage())
}
return r.GetRevision(), nil
}
// Create is an atomic create — fails with code "wrong_revision" if the key
// already exists.
func (b *Bucket) Create(ctx context.Context, key string, value []byte) (uint64, error) {
c := b.client.kvClient()
r, err := c.Create(ctx, &pb.KvCreateRequest{
Bucket: b.Name,
Key: key,
Value: value,
TtlMs: 0,
})
if err != nil {
return 0, rpcErr(err)
}
if !r.GetSuccess() {
return 0, serverErr(r.GetResultCode(), r.GetMessage())
}
return r.GetRevision(), nil
}
// Update is a CAS update — succeeds only if current revision matches
// expectedRevision.
func (b *Bucket) Update(ctx context.Context, key string, value []byte, expectedRevision uint64) (uint64, error) {
c := b.client.kvClient()
r, err := c.Update(ctx, &pb.KvUpdateRequest{
Bucket: b.Name,
Key: key,
Value: value,
ExpectedRevision: expectedRevision,
TtlMs: 0,
})
if err != nil {
return 0, rpcErr(err)
}
if !r.GetSuccess() {
return 0, serverErr(r.GetResultCode(), r.GetMessage())
}
return r.GetRevision(), nil
}
// Get returns the latest value. Returns (nil, nil) when absent or
// tombstoned.
func (b *Bucket) Get(ctx context.Context, key string) ([]byte, error) {
v, _, err := b.GetWithRevision(ctx, key)
return v, err
}
// GetWithRevision returns the latest value + revision (for chaining CAS).
// Returns (nil, 0, nil) when absent or tombstoned.
func (b *Bucket) GetWithRevision(ctx context.Context, key string) ([]byte, uint64, error) {
c := b.client.kvClient()
r, err := c.Get(ctx, &pb.KvGetRequest{Bucket: b.Name, Key: key})
if err != nil {
return nil, 0, rpcErr(err)
}
if !r.GetSuccess() {
return nil, 0, serverErr(r.GetResultCode(), r.GetMessage())
}
e := r.GetEntry()
if e == nil {
return nil, 0, nil
}
return e.GetValue(), e.GetRevision(), nil
}
// Delete tombstones key.
func (b *Bucket) Delete(ctx context.Context, key string) error {
c := b.client.kvClient()
r, err := c.Delete(ctx, &pb.KvDeleteRequest{Bucket: b.Name, Key: key})
if err != nil {
return rpcErr(err)
}
if !r.GetSuccess() {
return serverErr(r.GetResultCode(), r.GetMessage())
}
return nil
}
// Touch extends the TTL on key without changing its value.
func (b *Bucket) Touch(ctx context.Context, key string, ttl time.Duration) (uint64, error) {
c := b.client.kvClient()
r, err := c.Touch(ctx, &pb.KvTouchRequest{
Bucket: b.Name,
Key: key,
TtlMs: uint64(ttl.Milliseconds()),
})
if err != nil {
return 0, rpcErr(err)
}
if !r.GetSuccess() {
return 0, serverErr(r.GetResultCode(), r.GetMessage())
}
return r.GetRevision(), nil
}
// Keys lists every non-tombstoned key in the bucket.
func (b *Bucket) Keys(ctx context.Context) ([]string, error) {
c := b.client.kvClient()
r, err := c.Keys(ctx, &pb.KvKeysRequest{Bucket: b.Name})
if err != nil {
return nil, rpcErr(err)
}
if !r.GetSuccess() {
return nil, serverErr(r.GetResultCode(), r.GetMessage())
}
var out []string
for _, e := range r.GetEntries() {
if !e.GetDeleted() {
out = append(out, e.GetKey())
}
}
return out, nil
}
// HistoryEntry is one revision of a key.
type HistoryEntry struct {
Value []byte
Revision uint64
TsMs int64
Tombstone bool
}
// History returns historical values at key in publish order.
func (b *Bucket) History(ctx context.Context, key string) ([]HistoryEntry, error) {
c := b.client.kvClient()
r, err := c.History(ctx, &pb.KvHistoryRequest{
Bucket: b.Name,
Key: key,
FromRevision: 0,
Limit: 0,
})
if err != nil {
return nil, rpcErr(err)
}
if !r.GetSuccess() {
return nil, serverErr(r.GetResultCode(), r.GetMessage())
}
out := make([]HistoryEntry, len(r.GetEntries()))
for i, e := range r.GetEntries() {
out[i] = HistoryEntry{
Value: e.GetValue(),
Revision: e.GetRevision(),
TsMs: e.GetTsMs(),
Tombstone: e.GetDeleted(),
}
}
return out, nil
}
// KVEvent is a live change event delivered by Watch.
type KVEvent struct {
// Put is non-nil for a put event.
Put *KVPutEvent
// Delete is non-nil for a delete event.
Delete *KVDeleteEvent
}
// KVPutEvent carries a put notification.
type KVPutEvent struct {
Key string
Value []byte
Revision uint64
TsMs int64
}
// KVDeleteEvent carries a delete/tombstone notification.
type KVDeleteEvent struct {
Key string
Revision uint64
TsMs int64
}
// WatchStream is a live watch on a KV bucket.
type WatchStream struct {
inner pb.WaymakerKvService_WatchClient
}
// Next blocks until the next event arrives or an error occurs.
// Returns (zero, nil) on normal end-of-stream.
func (w *WatchStream) Next() (KVEvent, error) {
for {
ev, err := w.inner.Recv()
if err != nil {
if err == io.EOF {
return KVEvent{}, nil
}
return KVEvent{}, rpcErr(err)
}
switch e := ev.GetEvent().(type) {
case *pb.KvWatchEvent_Put:
return KVEvent{Put: &KVPutEvent{
Key: e.Put.GetKey(),
Value: e.Put.GetValue(),
Revision: e.Put.GetRevision(),
TsMs: e.Put.GetTsMs(),
}}, nil
case *pb.KvWatchEvent_Delete:
return KVEvent{Delete: &KVDeleteEvent{
Key: e.Delete.GetKey(),
Revision: e.Delete.GetRevision(),
TsMs: e.Delete.GetTsMs(),
}}, nil
}
}
}
// Watch opens a live watch on key.
func (b *Bucket) Watch(ctx context.Context, key string) (*WatchStream, error) {
return b.watchInner(ctx, key)
}
// WatchAll opens a live watch on every key in the bucket.
func (b *Bucket) WatchAll(ctx context.Context) (*WatchStream, error) {
return b.watchInner(ctx, "")
}
func (b *Bucket) watchInner(ctx context.Context, key string) (*WatchStream, error) {
c := b.client.kvClient()
stream, err := c.Watch(ctx, &pb.KvWatchRequest{Bucket: b.Name, Key: key})
if err != nil {
return nil, rpcErr(err)
}
return &WatchStream{inner: stream}, nil
}
// --- Client entry points ---
// CreateKV creates a new KV bucket.
func (c *Client) CreateKV(ctx context.Context, config KVConfig) (*Bucket, error) {
kvc := c.kvClient()
var maxAgeMs uint64
if config.MaxAge != nil {
maxAgeMs = uint64(config.MaxAge.Milliseconds())
}
r, err := kvc.CreateBucket(ctx, &pb.KvCreateBucketRequest{
Bucket: config.Name,
MaxBytes: uint64OrZero(config.MaxBytes),
MaxValueSize: uint64OrZero(config.MaxValueSize),
MaxAgeMs: maxAgeMs,
Ephemeral: config.Ephemeral,
MaxRevisions: config.MaxRevisions,
})
if err != nil {
return nil, rpcErr(err)
}
if !r.GetSuccess() {
return nil, serverErr(r.GetResultCode(), r.GetMessage())
}
return newBucket(c, config.Name), nil
}
// GetOrCreateKV is the idempotent create-or-get.
func (c *Client) GetOrCreateKV(ctx context.Context, config KVConfig) (*Bucket, error) {
b, err := c.CreateKV(ctx, config)
if err == nil {
return b, nil
}
if IsServerCode(err, "already_exists") {
return newBucket(c, config.Name), nil
}
return nil, err
}
// KV returns a bucket handle without verifying existence.
func (c *Client) KV(name string) *Bucket {
return newBucket(c, name)
}
// DeleteKV deletes the KV bucket.
func (c *Client) DeleteKV(ctx context.Context, name string) error {
kvc := c.kvClient()
r, err := kvc.DeleteBucket(ctx, &pb.KvDeleteBucketRequest{Bucket: name})
if err != nil {
return rpcErr(err)
}
if !r.GetSuccess() {
return serverErr(r.GetResultCode(), r.GetMessage())
}
return nil
}

774
go/lock.go Normal file
View File

@ -0,0 +1,774 @@
package waymaker
// Locks subsystem — wraps the rwlock RPCs (Lock / ReadLock / UnLock /
// LeaseStatus / ExtendLease / MultiLock).
//
// Entry points live on *Client:
// - client.AcquireLock(ctx, key, LockConfig{…}) — exclusive (write) lock
// - client.AcquireReadLock(ctx, key, LockConfig{…}) — shared (read) lock
// - client.LeaseStatus(ctx, key, id)
// - client.MultiLock(ctx, keys, LockConfig{…})
//
// The returned *Lock keeps a background goroutine that holds the server
// event stream open and, if that stream drops (e.g. the key's primary
// bounces), transparently re-binds it and re-confirms ownership — reusing
// the original RequestID so a still-held lease is recovered rather than
// re-contended. The lock's live state is published through Lock.Watch().
import (
"context"
"fmt"
"io"
"sync"
"time"
pb "git.awesomike.com/pub/waymaker-client/go/genpb/locks"
"google.golang.org/grpc"
)
// Ensure grpc is used (it's imported for CallOption in ListAcquiredLocks).
var _ = grpc.EmptyCallOption{}
// --- Scope ---
// Scope is the fence-token durability tier. It controls ONE thing: how
// durable the per-key fence_token counter (a monotonic uint64) is across
// failures. It does NOT control whether the lease survives node failures
// (that is replication_factor), nor mutual exclusion (you must still
// fence at the side-effect level).
type Scope int32
const (
// ScopeEphemeral — counter in RAM on the owning node. Resets on
// process restart or hash-ring rebalance. Fastest; no I/O.
ScopeEphemeral Scope = 1
// ScopeLocal — counter persisted to disk on the owning node.
// Survives process restart; resets on ring rebalance.
ScopeLocal Scope = 2
// ScopeQuorum — Raft-replicated per-key counter, cluster-wide
// monotonic. Survives any single-node failure.
ScopeQuorum Scope = 3
)
// --- LockConfig ---
// LockConfig is the acquire configuration. Zero value is valid:
// MaxWait defaults to 1 hour, LeaseTTL to 30 s, RequesterApplication
// to "waymaker-client-go". RequestID is auto-filled with a UUID if empty.
type LockConfig struct {
// MaxWait is how long the server will block waiting for the lock.
// Zero means try-acquire (fail immediately if contended).
// Default: 1 hour.
MaxWait time.Duration
// LeaseTTL is how long the lease lives once acquired.
// Default: 30 s.
LeaseTTL time.Duration
// Priority class — higher values jump the wait queue.
Priority uint32
// Scope controls fence-token durability.
Scope Scope
// RequesterInfo is free-form metadata for server-side audit.
RequesterInfo string
// RequesterApplication is the application name.
// Default: "waymaker-client-go".
RequesterApplication string
// RequestID is the idempotency key for retries of the same acquire.
// Auto-filled with a UUID if empty.
RequestID string
}
func (c LockConfig) withDefaults() LockConfig {
if c.MaxWait == 0 {
c.MaxWait = time.Hour
}
if c.LeaseTTL == 0 {
c.LeaseTTL = 30 * time.Second
}
if c.RequesterApplication == "" {
c.RequesterApplication = "waymaker-client-go"
}
if c.RequestID == "" {
c.RequestID = newUUID()
}
if c.Scope == 0 {
c.Scope = ScopeEphemeral
}
return c
}
func durationToMs(d time.Duration) uint32 {
ms := d.Milliseconds()
if ms < 0 {
return 0
}
if ms > int64(^uint32(0)) {
return ^uint32(0)
}
return uint32(ms)
}
func (c LockConfig) toLockRequest(key string) *pb.LockRequest {
return &pb.LockRequest{
Key: key,
MaxWaitPeriod: durationToMs(c.MaxWait),
MaxLeasePeriod: durationToMs(c.LeaseTTL),
Priority: c.Priority,
RequesterInfo: c.RequesterInfo,
RequesterApplication: c.RequesterApplication,
RequestId: c.RequestID,
FenceScope: pb.FenceScope(c.Scope),
}
}
// --- Lease ---
// Lease holds the details of a currently-held lock lease.
type Lease struct {
ID string
Key string
AcquiredAtMs int64
LeaseExpiresAtMs int64
FenceToken uint64
Priority uint32
}
func leaseFromPB(l *pb.Lease) Lease {
return Lease{
ID: l.GetId(),
Key: l.GetKey(),
AcquiredAtMs: l.GetCreatedAt(),
LeaseExpiresAtMs: l.GetLeaseExpiresAt(),
FenceToken: l.GetFenceToken(),
Priority: l.GetPriority(),
}
}
// --- LockState ---
// LockState is a live snapshot of a held lock, delivered through
// Lock.Watch(). FenceToken and ID change only if the lock was lost
// and transparently re-won after a primary failure; Lost flips to true
// once the client gives up re-establishing ownership.
type LockState struct {
// ID is the lease id. Stable across transparent re-binds.
ID string
// FenceToken is the current fence token. Re-read before every
// fenced side effect.
FenceToken uint64
// LeaseExpiresAtMs is the lease expiry epoch (ms).
LeaseExpiresAtMs int64
// Lost is true once the client could no longer prove ownership.
// A lost holder MUST stop acting as the holder.
Lost bool
}
// --- Lock ---
// Lock is an acquired lock handle. Dropping the handle does NOT
// auto-release the lock — call Unlock explicitly (or let the lease
// expire). This matches the underlying RPC semantics.
//
// A background goroutine keeps the server event stream open. If the
// stream drops — typically because the key's primary bounced — the
// goroutine transparently re-binds it, reusing the original RequestID
// so a still-held lease is recovered rather than re-contended. The
// lease itself is kept alive by the server's TTL plus SpawnRenewal,
// independent of the stream.
type Lock struct {
client *Client
Key string
mu sync.RWMutex
state LockState
// stateCh is closed and replaced whenever state changes, so
// Watch() receivers can block on the current channel.
stateCh chan struct{}
// stopCh is closed to signal the hold goroutine to stop.
stopCh chan struct{}
// stopOnce guards the single close of stopCh.
stopOnce sync.Once
holdDone chan struct{}
}
// ID returns the current lease id (live).
func (l *Lock) ID() string {
l.mu.RLock()
defer l.mu.RUnlock()
return l.state.ID
}
// FenceToken returns the current fence token (live). Re-read before
// every fenced side effect.
func (l *Lock) FenceToken() uint64 {
l.mu.RLock()
defer l.mu.RUnlock()
return l.state.FenceToken
}
// LeaseExpiresAtMs returns the lease expiry epoch ms (live).
func (l *Lock) LeaseExpiresAtMs() int64 {
l.mu.RLock()
defer l.mu.RUnlock()
return l.state.LeaseExpiresAtMs
}
// IsLost returns true once the client has lost the lock and could not
// re-win it. A lost holder must stop acting as the holder.
func (l *Lock) IsLost() bool {
l.mu.RLock()
defer l.mu.RUnlock()
return l.state.Lost
}
// State returns a copy of the current LockState.
func (l *Lock) State() LockState {
l.mu.RLock()
defer l.mu.RUnlock()
return l.state
}
// Watch returns a channel that is closed each time the lock state changes.
// Callers should call State() (or the individual accessors) after receiving
// from the channel to get the updated values. A new channel is returned on
// each call; each caller gets independent notification.
//
// Usage:
//
// ch := lock.Watch()
// for {
// <-ch
// if lock.IsLost() { ... }
// ch = lock.Watch() // re-subscribe for next change
// }
func (l *Lock) Watch() <-chan struct{} {
l.mu.RLock()
defer l.mu.RUnlock()
return l.stateCh
}
// publishState updates the internal state and notifies watchers.
// Must be called with mu held for write.
func (l *Lock) publishLocked(next LockState) {
if l.state == next {
return
}
l.state = next
old := l.stateCh
l.stateCh = make(chan struct{})
close(old)
}
// Extend the lease by additional duration.
func (l *Lock) Extend(ctx context.Context, additional time.Duration) (Lease, error) {
c := l.client.locksClient()
r, err := c.ExtendLease(ctx, &pb.ExtendLeaseRequest{
Key: l.Key,
Id: l.ID(),
LeaseTimeout: durationToMs(additional),
})
if err != nil {
return Lease{}, rpcErr(err)
}
if !r.GetSuccess() {
return Lease{}, serverErr(r.GetResultCode(), r.GetMessage())
}
lease := r.GetLease()
if lease == nil {
return Lease{}, serverErr("internal", "missing lease in ExtendLease response")
}
return leaseFromPB(lease), nil
}
// Unlock releases the lock. After this returns, the server-side state
// is gone. The background hold goroutine is stopped before the UnLock
// RPC is sent so it cannot re-acquire a lock the caller is releasing.
func (l *Lock) Unlock(ctx context.Context) error {
// Signal the hold goroutine to stop FIRST so it cannot race with
// the UnLock RPC and resurrect a just-released lock.
l.stop()
// Wait for the hold goroutine to finish before sending UnLock, so
// we don't race a re-acquire with the explicit release.
<-l.holdDone
id := l.ID()
c := l.client.locksClient()
r, err := c.UnLock(ctx, &pb.UnLockRequest{
Key: l.Key,
Id: id,
})
if err != nil {
return rpcErr(err)
}
if !r.GetSuccess() {
return serverErr(r.GetResultCode(), r.GetMessage())
}
return nil
}
func (l *Lock) stop() {
l.stopOnce.Do(func() {
close(l.stopCh)
})
}
// RenewalHandle keeps a background renewal task alive. Drop it (or call
// Stop) to halt renewal cleanly.
type RenewalHandle struct {
stopCh chan struct{}
stopOnce sync.Once
done chan struct{}
}
// Stop halts the renewal task. After this returns, no further
// ExtendLease RPCs will be sent.
func (h *RenewalHandle) Stop() {
h.stopOnce.Do(func() {
close(h.stopCh)
})
<-h.done
}
// SpawnRenewal spawns a background goroutine that periodically extends
// the lease. Returns a RenewalHandle — call Stop() or let it be
// garbage-collected (which aborts the goroutine) to halt renewal.
//
// The renewal reads the live id each tick so a transparent re-acquire
// after a primary failure renews the correct lease.
//
// The timer fires the TTL renewal at every after the first tick (which
// is skipped — the caller has the freshly-acquired lease). The extend
// call is bounded to every duration so a hung server can't freeze the
// renewal task.
func (l *Lock) SpawnRenewal(every time.Duration) *RenewalHandle {
h := &RenewalHandle{
stopCh: make(chan struct{}),
done: make(chan struct{}),
}
ttl := every * 2
if ttl > time.Duration(^uint32(0))*time.Millisecond {
ttl = time.Duration(^uint32(0)) * time.Millisecond
}
go func() {
defer close(h.done)
ticker := time.NewTicker(every)
defer ticker.Stop()
// Skip the first tick so we don't extend immediately after acquire.
select {
case <-ticker.C:
// discard the first tick
case <-h.stopCh:
return
case <-l.stopCh:
return
}
for {
select {
case <-h.stopCh:
return
case <-l.stopCh:
return
case <-ticker.C:
state := l.State()
if state.Lost {
continue
}
ctx, cancel := context.WithTimeout(context.Background(), every)
c := l.client.locksClient()
_, _ = c.ExtendLease(ctx, &pb.ExtendLeaseRequest{
Key: l.Key,
Id: state.ID,
LeaseTimeout: durationToMs(ttl),
})
cancel()
}
}
}()
return h
}
// --- background hold-loop internals ---
const (
holdBaseBackoff = 200 * time.Millisecond
holdMaxBackoff = 10 * time.Second
holdRPCTimeout = 10 * time.Second
)
type drainResult int
const (
drainStopped drainResult = iota // stop signal received
drainDisconnected // stream ended; try to re-establish
)
type reboundResult struct {
stream pb.WaymakerService_LockClient
id string
fence uint64
exp int64
ok bool
}
type ownershipResult int
const (
ownershipHeld ownershipResult = iota
ownershipLost
ownershipUnknown
)
// drainStream reads events from stream until it closes, stop is signalled,
// or the lease is reported gone. Heartbeat / Acquired events update lock state.
func (l *Lock) drainStream(stream pb.WaymakerService_LockClient) drainResult {
for {
// Check stop before blocking on Recv.
select {
case <-l.stopCh:
return drainStopped
default:
}
// Use a channel to race Recv against stopCh.
type recvResult struct {
ev *pb.LockEvent
err error
}
ch := make(chan recvResult, 1)
go func() {
ev, err := stream.Recv()
ch <- recvResult{ev, err}
}()
select {
case <-l.stopCh:
return drainStopped
case res := <-ch:
if res.err != nil {
// io.EOF or any other error: stream ended.
return drainDisconnected
}
ev := res.ev
switch ev.GetEventType() {
case pb.LockEventType_Heartbeat:
l.mu.Lock()
next := l.state
next.LeaseExpiresAtMs = ev.GetLeaseExpiresAt()
l.publishLocked(next)
l.mu.Unlock()
case pb.LockEventType_Acquired:
l.mu.Lock()
next := LockState{
ID: ev.GetId(),
FenceToken: ev.GetFenceToken(),
LeaseExpiresAtMs: ev.GetLeaseExpiresAt(),
Lost: false,
}
l.publishLocked(next)
l.mu.Unlock()
case pb.LockEventType_Expired, pb.LockEventType_Failed:
return drainDisconnected
default:
// Waiting, Unknown: ignore.
}
}
}
}
// reacquire tries one idempotent re-acquire (max_wait=0) using the
// original request_id. Returns reboundResult.ok=true on success.
func (l *Lock) reacquire(read bool, req *pb.LockRequest) reboundResult {
ctx, cancel := context.WithTimeout(context.Background(), holdRPCTimeout)
defer cancel()
var stream pb.WaymakerService_LockClient
var err error
c := l.client.locksClient()
if read {
stream, err = c.ReadLock(ctx, req)
} else {
stream, err = c.Lock(ctx, req)
}
if err != nil {
return reboundResult{}
}
for {
ev, err := stream.Recv()
if err != nil {
return reboundResult{}
}
switch ev.GetEventType() {
case pb.LockEventType_Acquired:
return reboundResult{
stream: stream,
id: ev.GetId(),
fence: ev.GetFenceToken(),
exp: ev.GetLeaseExpiresAt(),
ok: true,
}
case pb.LockEventType_Failed, pb.LockEventType_Expired:
return reboundResult{}
default:
// Waiting, Heartbeat — keep reading.
}
}
}
// confirmOwnership checks whether the lock's current id still owns the key.
func (l *Lock) confirmOwnership(key, id string) (ownershipResult, int64) {
ctx, cancel := context.WithTimeout(context.Background(), holdRPCTimeout)
defer cancel()
c := l.client.locksClient()
r, err := c.LeaseStatus(ctx, &pb.LeaseStatusRequest{Key: key, Id: id})
if err != nil {
return ownershipUnknown, 0
}
if !r.GetSuccess() || r.GetLease() == nil {
return ownershipLost, 0
}
return ownershipHeld, r.GetLease().GetLeaseExpiresAt()
}
// holdLoop is the background goroutine. It drains the stream, re-binds on
// drop, and marks the lock lost only when ownership cannot be confirmed.
func (l *Lock) holdLoop(read bool, stream pb.WaymakerService_LockClient, reacquireReq *pb.LockRequest) {
defer close(l.holdDone)
for {
switch l.drainStream(stream) {
case drainStopped:
return
case drainDisconnected:
// Fall through to re-establish.
}
backoff := holdBaseBackoff
for {
select {
case <-l.stopCh:
return
default:
}
rb := l.reacquire(read, reacquireReq)
if rb.ok {
l.mu.Lock()
l.publishLocked(LockState{
ID: rb.id,
FenceToken: rb.fence,
LeaseExpiresAtMs: rb.exp,
Lost: false,
})
l.mu.Unlock()
stream = rb.stream
break // resume draining the fresh stream
}
// Could not re-acquire. Check if we still hold the lease.
curID := l.ID()
own, exp := l.confirmOwnership(l.Key, curID)
switch own {
case ownershipHeld:
l.mu.Lock()
next := l.state
next.LeaseExpiresAtMs = exp
l.publishLocked(next)
l.mu.Unlock()
case ownershipUnknown:
// Transient — back off and retry.
case ownershipLost:
l.mu.Lock()
next := l.state
next.Lost = true
l.publishLocked(next)
l.mu.Unlock()
return
}
select {
case <-l.stopCh:
return
case <-time.After(backoff):
}
backoff *= 2
if backoff > holdMaxBackoff {
backoff = holdMaxBackoff
}
continue
}
// Broke out of inner loop — resume outer drain loop.
}
}
// --- Client entry points ---
// AcquireLock acquires an exclusive (write) lock. Blocks for at most
// config.MaxWait; returns an error with Code=="expired" if the wait
// elapses without a grant.
func (c *Client) AcquireLock(ctx context.Context, key string, config LockConfig) (*Lock, error) {
return c.acquireLockInner(ctx, key, config, false)
}
// AcquireReadLock acquires a shared (read) lock.
func (c *Client) AcquireReadLock(ctx context.Context, key string, config LockConfig) (*Lock, error) {
return c.acquireLockInner(ctx, key, config, true)
}
func (c *Client) acquireLockInner(ctx context.Context, key string, config LockConfig, read bool) (*Lock, error) {
config = config.withDefaults()
req := config.toLockRequest(key)
// Template for the re-acquire: same request_id (idempotent recovery)
// but max_wait = 0 so it never blocks.
// Construct fresh to avoid copying a proto message (contains sync.Mutex).
reacquireReq := &pb.LockRequest{
Key: req.Key,
MaxWaitPeriod: 0,
MaxLeasePeriod: req.MaxLeasePeriod,
Priority: req.Priority,
RequesterInfo: req.RequesterInfo,
RequesterApplication: req.RequesterApplication,
RequestId: req.RequestId,
FenceScope: req.FenceScope,
}
lc := c.locksClient()
var stream pb.WaymakerService_LockClient
var err error
if read {
stream, err = lc.ReadLock(ctx, req)
} else {
stream, err = lc.Lock(ctx, req)
}
if err != nil {
return nil, rpcErr(err)
}
// Consume events until Acquired / Failed / Expired.
for {
ev, err := stream.Recv()
if err != nil {
if err == io.EOF {
return nil, serverErr("stream_closed", "lock stream closed before acquire")
}
return nil, rpcErr(err)
}
switch ev.GetEventType() {
case pb.LockEventType_Acquired:
init := LockState{
ID: ev.GetId(),
FenceToken: ev.GetFenceToken(),
LeaseExpiresAtMs: ev.GetLeaseExpiresAt(),
Lost: false,
}
lock := &Lock{
client: c,
Key: key,
state: init,
stateCh: make(chan struct{}),
stopCh: make(chan struct{}),
holdDone: make(chan struct{}),
}
go lock.holdLoop(read, stream, reacquireReq)
return lock, nil
case pb.LockEventType_Failed:
return nil, serverErr("failed", ev.GetMessage())
case pb.LockEventType_Expired:
return nil, serverErr("expired", ev.GetMessage())
default:
// Waiting / Heartbeat / Unknown — keep reading.
}
}
}
// LeaseStatus queries the current state of a lock by id.
func (c *Client) LeaseStatus(ctx context.Context, key, id string) (Lease, error) {
lc := c.locksClient()
r, err := lc.LeaseStatus(ctx, &pb.LeaseStatusRequest{Key: key, Id: id})
if err != nil {
return Lease{}, rpcErr(err)
}
if !r.GetSuccess() {
return Lease{}, serverErr(r.GetResultCode(), r.GetMessage())
}
lease := r.GetLease()
if lease == nil {
return Lease{}, serverErr("internal", "missing lease in LeaseStatus response")
}
return leaseFromPB(lease), nil
}
// MultiLockKey is one entry in a MultiLock request.
type MultiLockKey struct {
Key string
WriteLock bool
}
// MultiLock acquires N locks atomically. The server sorts keys to
// guarantee deadlock-free ordering. On any failure every partial lock is
// released before the call returns.
//
// Returns leases in the server's acquisition order (lexicographic by key).
func (c *Client) MultiLock(ctx context.Context, keys []MultiLockKey, config LockConfig) ([]Lease, error) {
config = config.withDefaults()
pbKeys := make([]*pb.MultiLockKey, len(keys))
for i, k := range keys {
pbKeys[i] = &pb.MultiLockKey{Key: k.Key, WriteLock: k.WriteLock}
}
lc := c.locksClient()
r, err := lc.MultiLock(ctx, &pb.MultiLockRequest{
Keys: pbKeys,
MaxWaitPeriod: durationToMs(config.MaxWait),
MaxLeasePeriod: durationToMs(config.LeaseTTL),
Priority: config.Priority,
RequesterInfo: config.RequesterInfo,
RequesterApplication: config.RequesterApplication,
RequestId: config.RequestID,
FenceScope: pb.FenceScope(config.Scope),
})
if err != nil {
return nil, rpcErr(err)
}
if !r.GetSuccess() {
return nil, serverErr(r.GetResultCode(), r.GetMessage())
}
leases := make([]Lease, len(r.GetLeases()))
for i, l := range r.GetLeases() {
leases[i] = leaseFromPB(l)
}
return leases, nil
}
// ListAcquiredLocks returns every lock currently held on the node
// serving the request. Optionally filtered by key prefix.
func (c *Client) ListAcquiredLocks(ctx context.Context, keyPrefix string, opts ...grpc.CallOption) ([]*pb.AcquiredLock, error) {
lc := c.locksClient()
r, err := lc.ListAcquiredLocks(ctx, &pb.ListAcquiredLocksRequest{KeyPrefix: keyPrefix}, opts...)
if err != nil {
return nil, rpcErr(err)
}
if !r.GetSuccess() {
return nil, serverErr("list_failed", "ListAcquiredLocks returned success=false")
}
return r.GetLocks(), nil
}
// --- UUID helper (no external dependency) ---
func newUUID() string {
// Simple time-based UUID v4 using crypto/rand.
b := make([]byte, 16)
_, _ = cryptoRandRead(b)
b[6] = (b[6] & 0x0f) | 0x40
b[8] = (b[8] & 0x3f) | 0x80
return fmt.Sprintf("%08x-%04x-%04x-%04x-%012x",
b[0:4], b[4:6], b[6:8], b[8:10], b[10:16])
}

302
go/object.go Normal file
View File

@ -0,0 +1,302 @@
package waymaker
// Object store subsystem — chunked Put/Get/Delete/List/Revisions backed by
// the streams service's PutObject/GetObject RPCs.
//
// An object-store bucket maps to a stream. The wrapper hides the wire
// convention behind a per-bucket *ObjectStore handle.
//
// Entry points on *Client:
// - client.CreateObjectStore(ctx, ObjectStoreConfig{…})
// - client.GetOrCreateObjectStore(ctx, ObjectStoreConfig{…})
// - client.ObjectStore(name) — handle without creation
import (
"context"
pb "git.awesomike.com/pub/waymaker-client/go/genpb/streams"
)
// DefaultChunkSize is the server-side default chunk size (1 MiB).
const DefaultChunkSize uint64 = 1024 * 1024
// ObjectStoreConfig is the bucket creation config.
type ObjectStoreConfig struct {
Name string
MaxBytes *uint64
Ephemeral bool
}
// ObjectInfo is the metadata of a stored object.
type ObjectInfo struct {
Name string
TotalBytes uint64
ChunkCount uint64
ChunkSize uint64
SHA256 string
TsMs int64
Headers [][2]string
Revision uint64 // metadata sequence
Deduped bool
}
func objectInfoFromPB(i *pb.ObjectInfo) ObjectInfo {
hdrs := make([][2]string, len(i.GetHeaders()))
for j, h := range i.GetHeaders() {
hdrs[j] = [2]string{h.GetKey(), h.GetValue()}
}
return ObjectInfo{
Name: i.GetName(),
TotalBytes: i.GetTotalBytes(),
ChunkCount: i.GetChunkCount(),
ChunkSize: i.GetChunkSize(),
SHA256: i.GetSha256(),
TsMs: i.GetTsMs(),
Headers: hdrs,
Revision: i.GetMetadataSeq(),
Deduped: i.GetDeduped(),
}
}
// ObjectEntry is one row returned by ObjectStore.List.
type ObjectEntry struct {
Name string
TotalBytes uint64
Deleted bool
}
// ObjectRevision is one metadata revision returned by ObjectStore.Revisions.
type ObjectRevision struct {
MetadataSeq uint64
TotalBytes uint64
SHA256 string
TsMs int64
Deleted bool
}
// PutOptions controls optional Put parameters.
type PutOptions struct {
ChunkSize uint64 // 0 = server default
Headers [][2]string
SHA256 string // optional pre-computed SHA-256 hex
Dedupe bool
}
// ObjectStore is a reference to an object-store bucket.
type ObjectStore struct {
client *Client
Name string
}
func newObjectStore(c *Client, name string) *ObjectStore {
return &ObjectStore{client: c, Name: name}
}
// Put uploads a small-to-medium object (whole payload in one RPC).
func (s *ObjectStore) Put(ctx context.Context, name string, payload []byte) (ObjectInfo, error) {
return s.PutWith(ctx, name, payload, PutOptions{})
}
// PutWith uploads with explicit options.
func (s *ObjectStore) PutWith(ctx context.Context, name string, payload []byte, opts PutOptions) (ObjectInfo, error) {
pbHeaders := make([]*pb.MessageHeader, len(opts.Headers))
for i, h := range opts.Headers {
pbHeaders[i] = &pb.MessageHeader{Key: h[0], Value: h[1]}
}
c := s.client.streamsClient()
r, err := c.PutObject(ctx, &pb.PutObjectRequest{
Bucket: s.Name,
Name: name,
Payload: payload,
ChunkSize: opts.ChunkSize,
Headers: pbHeaders,
Sha256: opts.SHA256,
Dedupe: opts.Dedupe,
})
if err != nil {
return ObjectInfo{}, rpcErr(err)
}
if !r.GetSuccess() {
return ObjectInfo{}, serverErr(r.GetResultCode(), r.GetMessage())
}
info := r.GetInfo()
if info == nil {
return ObjectInfo{}, serverErr("internal", "missing info in PutObject response")
}
return objectInfoFromPB(info), nil
}
// Get retrieves an object's payload and metadata.
func (s *ObjectStore) Get(ctx context.Context, name string) (ObjectInfo, []byte, error) {
c := s.client.streamsClient()
r, err := c.GetObject(ctx, &pb.GetObjectRequest{Bucket: s.Name, Name: name})
if err != nil {
return ObjectInfo{}, nil, rpcErr(err)
}
if !r.GetSuccess() {
return ObjectInfo{}, nil, serverErr(r.GetResultCode(), r.GetMessage())
}
info := r.GetInfo()
if info == nil {
return ObjectInfo{}, nil, serverErr("internal", "missing info in GetObject response")
}
return objectInfoFromPB(info), r.GetPayload(), nil
}
// GetRange reads a byte range of an object's payload. len=0 reads to EOF.
func (s *ObjectStore) GetRange(ctx context.Context, name string, offset, length uint64) (ObjectInfo, uint64, []byte, error) {
c := s.client.streamsClient()
r, err := c.GetObjectRange(ctx, &pb.GetObjectRangeRequest{
Bucket: s.Name,
Name: name,
Offset: offset,
Len: length,
})
if err != nil {
return ObjectInfo{}, 0, nil, rpcErr(err)
}
if !r.GetSuccess() {
return ObjectInfo{}, 0, nil, serverErr(r.GetResultCode(), r.GetMessage())
}
info := r.GetInfo()
if info == nil {
return ObjectInfo{}, 0, nil, serverErr("internal", "missing info in GetObjectRange response")
}
return objectInfoFromPB(info), r.GetActualOffset(), r.GetPayload(), nil
}
// Info returns object metadata without the payload. Returns (zero, nil) if
// the object has been deleted.
func (s *ObjectStore) Info(ctx context.Context, name string) (*ObjectInfo, error) {
c := s.client.streamsClient()
r, err := c.GetObjectInfo(ctx, &pb.GetObjectInfoRequest{Bucket: s.Name, Name: name})
if err != nil {
return nil, rpcErr(err)
}
if !r.GetSuccess() {
return nil, serverErr(r.GetResultCode(), r.GetMessage())
}
if r.GetDeleted() {
return nil, nil
}
if r.GetInfo() == nil {
return nil, nil
}
info := objectInfoFromPB(r.GetInfo())
return &info, nil
}
// Delete tombstones an object. Returns the tombstone sequence number.
func (s *ObjectStore) Delete(ctx context.Context, name string) (uint64, error) {
c := s.client.streamsClient()
r, err := c.DeleteObject(ctx, &pb.DeleteObjectRequest{Bucket: s.Name, Name: name})
if err != nil {
return 0, rpcErr(err)
}
if !r.GetSuccess() {
return 0, serverErr(r.GetResultCode(), r.GetMessage())
}
return r.GetTombstoneSeq(), nil
}
// List lists objects in the bucket matching namePrefix. Tombstoned entries
// are excluded.
func (s *ObjectStore) List(ctx context.Context, namePrefix string) ([]ObjectEntry, error) {
return s.listInner(ctx, namePrefix, false)
}
// ListWithDeleted lists objects including tombstoned entries.
func (s *ObjectStore) ListWithDeleted(ctx context.Context, namePrefix string) ([]ObjectEntry, error) {
return s.listInner(ctx, namePrefix, true)
}
func (s *ObjectStore) listInner(ctx context.Context, namePrefix string, includeDeleted bool) ([]ObjectEntry, error) {
c := s.client.streamsClient()
r, err := c.ListObjects(ctx, &pb.ListObjectsRequest{
Bucket: s.Name,
NamePrefix: namePrefix,
IncludeDeleted: includeDeleted,
})
if err != nil {
return nil, rpcErr(err)
}
if !r.GetSuccess() {
return nil, serverErr(r.GetResultCode(), r.GetMessage())
}
out := make([]ObjectEntry, len(r.GetEntries()))
for i, e := range r.GetEntries() {
out[i] = ObjectEntry{
Name: e.GetName(),
TotalBytes: e.GetTotalBytes(),
Deleted: e.GetDeleted(),
}
}
return out, nil
}
// Revisions lists every metadata revision of name in sequence order.
func (s *ObjectStore) Revisions(ctx context.Context, name string) ([]ObjectRevision, error) {
c := s.client.streamsClient()
r, err := c.ListObjectRevisions(ctx, &pb.ListObjectRevisionsRequest{
Bucket: s.Name,
Name: name,
FromSeq: 0,
Limit: 0,
})
if err != nil {
return nil, rpcErr(err)
}
if !r.GetSuccess() {
return nil, serverErr(r.GetResultCode(), r.GetMessage())
}
out := make([]ObjectRevision, len(r.GetRevisions()))
for i, rev := range r.GetRevisions() {
out[i] = ObjectRevision{
MetadataSeq: rev.GetMetadataSeq(),
TotalBytes: rev.GetTotalBytes(),
SHA256: rev.GetSha256(),
TsMs: rev.GetTsMs(),
Deleted: rev.GetDeleted(),
}
}
return out, nil
}
// --- Client entry points ---
// CreateObjectStore creates a new object-store bucket.
func (c *Client) CreateObjectStore(ctx context.Context, config ObjectStoreConfig) (*ObjectStore, error) {
sc := StreamConfig{
Name: config.Name,
Subjects: []string{"objm.>", "objc.>"},
Retention: RetentionLimits,
MaxBytes: config.MaxBytes,
Ephemeral: config.Ephemeral,
}
_, err := c.CreateStream(ctx, sc)
if err != nil {
return nil, err
}
return newObjectStore(c, config.Name), nil
}
// GetOrCreateObjectStore is the idempotent create-or-get.
func (c *Client) GetOrCreateObjectStore(ctx context.Context, config ObjectStoreConfig) (*ObjectStore, error) {
sc := StreamConfig{
Name: config.Name,
Subjects: []string{"objm.>", "objc.>"},
Retention: RetentionLimits,
MaxBytes: config.MaxBytes,
Ephemeral: config.Ephemeral,
}
_, err := c.GetOrCreateStream(ctx, sc)
if err != nil {
return nil, err
}
return newObjectStore(c, config.Name), nil
}
// ObjectStoreHandle returns a handle without verifying existence.
func (c *Client) ObjectStoreHandle(name string) *ObjectStore {
return newObjectStore(c, name)
}

501
go/sketches.go Normal file
View File

@ -0,0 +1,501 @@
package waymaker
// Sketches subsystem — Bloom filter, HyperLogLog, Count-Min Sketch, Top-K,
// t-digest. Thin RPC bindings over WaymakerSketchesService.
//
// Entry points on *Client:
// Bloom: CreateBloom / Bloom(name) / DeleteBloom
// HLL: CreateHLL / HLL(name) / DeleteHLL
// CMS: CreateCMS / CMS(name) / DeleteCMS
// TopK: CreateTopK / TopK(name) / DeleteTopK
// TDigest: CreateTDigest / TDigest(name) / DeleteTDigest
import (
"context"
pb "git.awesomike.com/pub/waymaker-client/go/genpb/sketches"
)
// ============================================================
// Bloom filter
// ============================================================
// BloomConfig is the Bloom filter creation config.
type BloomConfig struct {
Name string
Capacity uint64
// ErrorRate is the target false-positive rate (e.g. 0.01 for 1%).
// 0 = server default (0.01).
ErrorRate float64
}
// BloomInfo is diagnostic information about a Bloom filter.
type BloomInfo struct {
Capacity uint64
ErrorRate float64
BitsSet uint64
BitCount uint64
HashCount uint32
ItemsAdded uint64
}
// Bloom is a reference to a Bloom filter.
type Bloom struct {
client *Client
Name string
}
func newBloom(c *Client, name string) *Bloom { return &Bloom{client: c, Name: name} }
// Add adds item to the filter.
func (b *Bloom) Add(ctx context.Context, item []byte) error {
c := b.client.sketchesClient()
r, err := c.BloomAdd(ctx, &pb.BloomAddRequest{Name: b.Name, Item: item})
if err != nil {
return rpcErr(err)
}
if !r.GetSuccess() {
return serverErr(r.GetResultCode(), r.GetMessage())
}
return nil
}
// AddMany adds multiple items.
func (b *Bloom) AddMany(ctx context.Context, items [][]byte) error {
c := b.client.sketchesClient()
r, err := c.BloomMultiAdd(ctx, &pb.BloomMultiAddRequest{Name: b.Name, Items: items})
if err != nil {
return rpcErr(err)
}
if !r.GetSuccess() {
return serverErr(r.GetResultCode(), r.GetMessage())
}
return nil
}
// Exists tests membership. true = probably present; false = definitely absent.
func (b *Bloom) Exists(ctx context.Context, item []byte) (bool, error) {
c := b.client.sketchesClient()
r, err := c.BloomExists(ctx, &pb.BloomExistsRequest{Name: b.Name, Item: item})
if err != nil {
return false, rpcErr(err)
}
if !r.GetSuccess() {
return false, serverErr(r.GetResultCode(), r.GetMessage())
}
return r.GetExists(), nil
}
// ExistsMany tests membership for multiple items.
func (b *Bloom) ExistsMany(ctx context.Context, items [][]byte) ([]bool, error) {
c := b.client.sketchesClient()
r, err := c.BloomMultiExists(ctx, &pb.BloomMultiExistsRequest{Name: b.Name, Items: items})
if err != nil {
return nil, rpcErr(err)
}
if !r.GetSuccess() {
return nil, serverErr(r.GetResultCode(), r.GetMessage())
}
return r.GetExists(), nil
}
// Info returns diagnostic info about the filter.
func (b *Bloom) Info(ctx context.Context) (BloomInfo, error) {
c := b.client.sketchesClient()
r, err := c.BloomInfo(ctx, &pb.BloomInfoRequest{Name: b.Name})
if err != nil {
return BloomInfo{}, rpcErr(err)
}
if !r.GetSuccess() {
return BloomInfo{}, serverErr(r.GetResultCode(), r.GetMessage())
}
return BloomInfo{
Capacity: r.GetCapacity(),
ErrorRate: r.GetErrorRate(),
BitsSet: r.GetBitsSet(),
BitCount: r.GetBitCount(),
HashCount: r.GetHashCount(),
ItemsAdded: r.GetItemsAdded(),
}, nil
}
func (c *Client) CreateBloom(ctx context.Context, config BloomConfig) (*Bloom, error) {
sc := c.sketchesClient()
r, err := sc.BloomReserve(ctx, &pb.BloomReserveRequest{
Name: config.Name,
Capacity: config.Capacity,
ErrorRate: config.ErrorRate,
})
if err != nil {
return nil, rpcErr(err)
}
if !r.GetSuccess() {
return nil, serverErr(r.GetResultCode(), r.GetMessage())
}
return newBloom(c, config.Name), nil
}
func (c *Client) BloomHandle(name string) *Bloom { return newBloom(c, name) }
func (c *Client) DeleteBloom(ctx context.Context, name string) error {
sc := c.sketchesClient()
r, err := sc.BloomDelete(ctx, &pb.BloomDeleteRequest{Name: name})
if err != nil {
return rpcErr(err)
}
if !r.GetSuccess() {
return serverErr(r.GetResultCode(), r.GetMessage())
}
return nil
}
// ============================================================
// HyperLogLog
// ============================================================
// HLLConfig is the HLL creation config.
type HLLConfig struct {
Name string
// Precision controls register count: 2^Precision. Valid range 4..18.
// 0 = server default (14, ~1% error).
Precision uint32
}
// HLL is a reference to a HyperLogLog.
type HLL struct {
client *Client
Name string
}
func newHLL(c *Client, name string) *HLL { return &HLL{client: c, Name: name} }
// Add adds items to the estimator.
func (h *HLL) Add(ctx context.Context, items [][]byte) error {
c := h.client.sketchesClient()
r, err := c.HllAdd(ctx, &pb.HllAddRequest{Name: h.Name, Items: items})
if err != nil {
return rpcErr(err)
}
if !r.GetSuccess() {
return serverErr(r.GetResultCode(), r.GetMessage())
}
return nil
}
// Count returns the estimated cardinality.
func (h *HLL) Count(ctx context.Context) (uint64, error) {
c := h.client.sketchesClient()
r, err := c.HllCount(ctx, &pb.HllCountRequest{Name: h.Name})
if err != nil {
return 0, rpcErr(err)
}
if !r.GetSuccess() {
return 0, serverErr(r.GetResultCode(), r.GetMessage())
}
return r.GetEstimate(), nil
}
// MergeFrom merges sources into this HLL (union of registers).
func (h *HLL) MergeFrom(ctx context.Context, sources []string) error {
c := h.client.sketchesClient()
r, err := c.HllMerge(ctx, &pb.HllMergeRequest{Destination: h.Name, Sources: sources})
if err != nil {
return rpcErr(err)
}
if !r.GetSuccess() {
return serverErr(r.GetResultCode(), r.GetMessage())
}
return nil
}
func (c *Client) CreateHLL(ctx context.Context, config HLLConfig) (*HLL, error) {
sc := c.sketchesClient()
r, err := sc.HllReserve(ctx, &pb.HllReserveRequest{Name: config.Name, Precision: config.Precision})
if err != nil {
return nil, rpcErr(err)
}
if !r.GetSuccess() {
return nil, serverErr(r.GetResultCode(), r.GetMessage())
}
return newHLL(c, config.Name), nil
}
func (c *Client) HLLHandle(name string) *HLL { return newHLL(c, name) }
func (c *Client) DeleteHLL(ctx context.Context, name string) error {
sc := c.sketchesClient()
r, err := sc.HllDelete(ctx, &pb.HllDeleteRequest{Name: name})
if err != nil {
return rpcErr(err)
}
if !r.GetSuccess() {
return serverErr(r.GetResultCode(), r.GetMessage())
}
return nil
}
// ============================================================
// Count-Min Sketch
// ============================================================
// CMSConfig is the CMS creation config.
type CMSConfig struct {
Name string
Width uint64 // 0 = server default
Depth uint64 // 0 = server default
}
// CMSIncrItem is one (item, count) pair for CMS.Incr.
type CMSIncrItem struct {
Item []byte
Count uint64
}
// CMS is a reference to a Count-Min Sketch.
type CMS struct {
client *Client
Name string
}
func newCMS(c *Client, name string) *CMS { return &CMS{client: c, Name: name} }
// Incr increments counts. Returns one estimate per input item.
func (c *CMS) Incr(ctx context.Context, items []CMSIncrItem) ([]uint64, error) {
sc := c.client.sketchesClient()
pbItems := make([]*pb.CmsIncrByItem, len(items))
for i, it := range items {
pbItems[i] = &pb.CmsIncrByItem{Item: it.Item, Count: it.Count}
}
r, err := sc.CmsIncrBy(ctx, &pb.CmsIncrByRequest{Name: c.Name, Items: pbItems})
if err != nil {
return nil, rpcErr(err)
}
if !r.GetSuccess() {
return nil, serverErr(r.GetResultCode(), r.GetMessage())
}
return r.GetCounts(), nil
}
// Query returns estimated frequencies for items.
func (c *CMS) Query(ctx context.Context, items [][]byte) ([]uint64, error) {
sc := c.client.sketchesClient()
r, err := sc.CmsQuery(ctx, &pb.CmsQueryRequest{Name: c.Name, Items: items})
if err != nil {
return nil, rpcErr(err)
}
if !r.GetSuccess() {
return nil, serverErr(r.GetResultCode(), r.GetMessage())
}
return r.GetCounts(), nil
}
func (c *Client) CreateCMS(ctx context.Context, config CMSConfig) (*CMS, error) {
sc := c.sketchesClient()
r, err := sc.CmsReserve(ctx, &pb.CmsReserveRequest{Name: config.Name, Width: config.Width, Depth: config.Depth})
if err != nil {
return nil, rpcErr(err)
}
if !r.GetSuccess() {
return nil, serverErr(r.GetResultCode(), r.GetMessage())
}
return newCMS(c, config.Name), nil
}
func (c *Client) CMSHandle(name string) *CMS { return newCMS(c, name) }
func (c *Client) DeleteCMS(ctx context.Context, name string) error {
sc := c.sketchesClient()
r, err := sc.CmsDelete(ctx, &pb.CmsDeleteRequest{Name: name})
if err != nil {
return rpcErr(err)
}
if !r.GetSuccess() {
return serverErr(r.GetResultCode(), r.GetMessage())
}
return nil
}
// ============================================================
// Top-K
// ============================================================
// TopKConfig is the Top-K creation config.
type TopKConfig struct {
Name string
K uint32
Width uint64 // 0 = server default
Depth uint64 // 0 = server default
Decay float64 // 0 = server default
}
// TopKListEntry is one (item, count) in the top-K list.
type TopKListEntry struct {
Item []byte
Count uint64
}
// TopK is a reference to a Top-K sketch.
type TopK struct {
client *Client
Name string
}
func newTopK(c *Client, name string) *TopK { return &TopK{client: c, Name: name} }
// Add adds items. Returns one evicted item per slot (empty bytes if none).
func (t *TopK) Add(ctx context.Context, items [][]byte) ([][]byte, error) {
c := t.client.sketchesClient()
r, err := c.TopKAdd(ctx, &pb.TopKAddRequest{Name: t.Name, Items: items})
if err != nil {
return nil, rpcErr(err)
}
if !r.GetSuccess() {
return nil, serverErr(r.GetResultCode(), r.GetMessage())
}
return r.GetEvicted(), nil
}
// Query tests whether items are in the top-K list.
func (t *TopK) Query(ctx context.Context, items [][]byte) ([]bool, error) {
c := t.client.sketchesClient()
r, err := c.TopKQuery(ctx, &pb.TopKQueryRequest{Name: t.Name, Items: items})
if err != nil {
return nil, rpcErr(err)
}
if !r.GetSuccess() {
return nil, serverErr(r.GetResultCode(), r.GetMessage())
}
return r.GetInTopK(), nil
}
// List returns the current top-K list.
func (t *TopK) List(ctx context.Context) ([]TopKListEntry, error) {
c := t.client.sketchesClient()
r, err := c.TopKList(ctx, &pb.TopKListRequest{Name: t.Name})
if err != nil {
return nil, rpcErr(err)
}
if !r.GetSuccess() {
return nil, serverErr(r.GetResultCode(), r.GetMessage())
}
out := make([]TopKListEntry, len(r.GetEntries()))
for i, e := range r.GetEntries() {
out[i] = TopKListEntry{Item: e.GetItem(), Count: e.GetCount()}
}
return out, nil
}
func (c *Client) CreateTopK(ctx context.Context, config TopKConfig) (*TopK, error) {
sc := c.sketchesClient()
r, err := sc.TopKReserve(ctx, &pb.TopKReserveRequest{
Name: config.Name,
K: config.K,
Width: config.Width,
Depth: config.Depth,
Decay: config.Decay,
})
if err != nil {
return nil, rpcErr(err)
}
if !r.GetSuccess() {
return nil, serverErr(r.GetResultCode(), r.GetMessage())
}
return newTopK(c, config.Name), nil
}
func (c *Client) TopKHandle(name string) *TopK { return newTopK(c, name) }
func (c *Client) DeleteTopK(ctx context.Context, name string) error {
sc := c.sketchesClient()
r, err := sc.TopKDelete(ctx, &pb.TopKDeleteRequest{Name: name})
if err != nil {
return rpcErr(err)
}
if !r.GetSuccess() {
return serverErr(r.GetResultCode(), r.GetMessage())
}
return nil
}
// ============================================================
// t-digest
// ============================================================
// TDigestConfig is the t-digest creation config.
type TDigestConfig struct {
Name string
Compression uint32 // 0 = server default
}
// TDigest is a reference to a t-digest quantile sketch.
type TDigest struct {
client *Client
Name string
}
func newTDigest(c *Client, name string) *TDigest { return &TDigest{client: c, Name: name} }
// Add adds values to the sketch.
func (t *TDigest) Add(ctx context.Context, values []float64) error {
c := t.client.sketchesClient()
r, err := c.TDigestAdd(ctx, &pb.TDigestAddRequest{Name: t.Name, Values: values})
if err != nil {
return rpcErr(err)
}
if !r.GetSuccess() {
return serverErr(r.GetResultCode(), r.GetMessage())
}
return nil
}
// Quantile computes quantile estimates. One result per input quantile.
func (t *TDigest) Quantile(ctx context.Context, quantiles []float64) ([]float64, error) {
c := t.client.sketchesClient()
r, err := c.TDigestQuantile(ctx, &pb.TDigestQuantileRequest{Name: t.Name, Quantiles: quantiles})
if err != nil {
return nil, rpcErr(err)
}
if !r.GetSuccess() {
return nil, serverErr(r.GetResultCode(), r.GetMessage())
}
return r.GetValues(), nil
}
// MinMax returns the minimum and maximum observed values.
func (t *TDigest) MinMax(ctx context.Context) (min, max float64, err error) {
c := t.client.sketchesClient()
r, err := c.TDigestMinMax(ctx, &pb.TDigestMinMaxRequest{Name: t.Name})
if err != nil {
return 0, 0, rpcErr(err)
}
if !r.GetSuccess() {
return 0, 0, serverErr(r.GetResultCode(), r.GetMessage())
}
return r.GetMin(), r.GetMax(), nil
}
func (c *Client) CreateTDigest(ctx context.Context, config TDigestConfig) (*TDigest, error) {
sc := c.sketchesClient()
r, err := sc.TDigestCreate(ctx, &pb.TDigestCreateRequest{Name: config.Name, Compression: config.Compression})
if err != nil {
return nil, rpcErr(err)
}
if !r.GetSuccess() {
return nil, serverErr(r.GetResultCode(), r.GetMessage())
}
return newTDigest(c, config.Name), nil
}
func (c *Client) TDigestHandle(name string) *TDigest { return newTDigest(c, name) }
func (c *Client) DeleteTDigest(ctx context.Context, name string) error {
sc := c.sketchesClient()
r, err := sc.TDigestDelete(ctx, &pb.TDigestDeleteRequest{Name: name})
if err != nil {
return rpcErr(err)
}
if !r.GetSuccess() {
return serverErr(r.GetResultCode(), r.GetMessage())
}
return nil
}

673
go/stream.go Normal file
View File

@ -0,0 +1,673 @@
package waymaker
// Streams subsystem — wraps the stream lifecycle, publish, fetch, and
// consumer (push + pull) RPCs.
//
// Entry points on *Client:
// - client.CreateStream(ctx, StreamConfig{…})
// - client.GetStream(ctx, name)
// - client.GetOrCreateStream(ctx, StreamConfig{…})
// - client.UpdateStream(ctx, name, StreamUpdate{…})
// - client.DeleteStream(ctx, name)
import (
"context"
"io"
"time"
pb "git.awesomike.com/pub/waymaker-client/go/genpb/streams"
)
// --- Retention policy ---
// RetentionPolicy controls how the stream prunes old messages.
type RetentionPolicy int
const (
RetentionLimits RetentionPolicy = iota // size/age limits; default
RetentionWorkQueue // exactly-once work queue
RetentionInterest // retain while consumers exist
)
// --- StreamSource ---
// OnDropPolicy controls what a cross-stream tail does when the source
// drops messages past the last-sourced seq.
type OnDropPolicy int
const (
OnDropHalt OnDropPolicy = 0
OnDropSkipToFirstAvailable OnDropPolicy = 1
)
// SubjectTransform is a NATS-style subject rewrite for cross-stream sources.
type SubjectTransform struct {
SourcePattern string
Destination string
}
// StreamSource identifies a stream to tail from.
type StreamSource struct {
SourceStream string
FilterSubject string
StartSeq uint64
StartTimeMs int64
MaxInitialBackfill uint64
SubjectTransform *SubjectTransform
OnDrop OnDropPolicy
DlqStream string
}
// --- StreamConfig ---
// StreamConfig is the creation config for a stream.
type StreamConfig struct {
Name string
Subjects []string
Retention RetentionPolicy
MaxAge time.Duration
MaxMessages *uint64
MaxBytes *uint64
MaxMessageSize *uint64
BlockSize uint64
StrictLimits bool
Ephemeral bool
MaxMsgsPerSubject uint64
Sources []StreamSource
}
func (c StreamConfig) toPB() *pb.StreamConfigPb {
var retention *pb.Retention
switch c.Retention {
case RetentionWorkQueue:
retention = &pb.Retention{Policy: &pb.Retention_WorkQueue{WorkQueue: &pb.WorkQueueRetention{}}}
case RetentionInterest:
retention = &pb.Retention{Policy: &pb.Retention_Interest{Interest: &pb.InterestRetention{}}}
default: // RetentionLimits
var maxAgeMs *uint64
if c.MaxAge > 0 {
v := uint64(c.MaxAge.Milliseconds())
maxAgeMs = &v
}
retention = &pb.Retention{
Policy: &pb.Retention_Limits{
Limits: &pb.LimitsRetention{
MaxAgeMs: maxAgeMs,
MaxMsgs: c.MaxMessages,
MaxBytes: c.MaxBytes,
StrictLimits: c.StrictLimits,
},
},
}
}
sources := make([]*pb.StreamSourceConfigPb, len(c.Sources))
for i, s := range c.Sources {
var onDrop pb.OnDropPolicy
if s.OnDrop == OnDropSkipToFirstAvailable {
onDrop = pb.OnDropPolicy_ON_DROP_SKIP_TO_FIRST_AVAILABLE
}
src := &pb.StreamSourceConfigPb{
SourceStream: s.SourceStream,
FilterSubject: s.FilterSubject,
StartSeq: s.StartSeq,
StartTimeMs: s.StartTimeMs,
MaxInitialBackfill: s.MaxInitialBackfill,
OnDrop: onDrop,
DlqStream: s.DlqStream,
}
if s.SubjectTransform != nil {
src.SubjectTransform = &pb.SubjectTransformPb{
SourcePattern: s.SubjectTransform.SourcePattern,
Destination: s.SubjectTransform.Destination,
}
}
sources[i] = src
}
return &pb.StreamConfigPb{
Name: c.Name,
SubjectsFilter: c.Subjects,
Retention: retention,
BlockSize: c.BlockSize,
MaxMsgBytes: uint64OrZero(c.MaxMessageSize),
Ephemeral: c.Ephemeral,
MaxMsgsPerSubject: c.MaxMsgsPerSubject,
Sources: sources,
}
}
func uint64OrZero(p *uint64) uint64 {
if p == nil {
return 0
}
return *p
}
// --- StreamUpdate ---
// StreamUpdate carries the mutable subset of a stream's config.
// Nil fields are not touched server-side.
type StreamUpdate struct {
MaxAge *time.Duration
MaxMessages *uint64
MaxBytes *uint64
MaxMsgBytes *uint64
StrictLimits *bool
}
// --- SourceStatus ---
// SourceStatus is one row of per-(sourcing, source) status.
type SourceStatus struct {
SourcingStream string
SourceStream string
LastSourcedSeq uint64
PulledTotal uint64
LastError string
LastErrorTsMs int64
}
// --- PublishAck ---
// PublishAck is returned by Stream.Publish.
type PublishAck struct {
Sequence uint64
}
// --- Stream handle ---
// Stream is a reference to a stream on the server. Cheap to copy.
type Stream struct {
client *Client
Name string
}
func newStream(c *Client, name string) *Stream {
return &Stream{client: c, Name: name}
}
// Publish sends a message into this stream.
func (s *Stream) Publish(ctx context.Context, subject string, payload []byte) (PublishAck, error) {
return s.PublishWithHeaders(ctx, subject, nil, payload)
}
// PublishWithHeaders publishes with explicit headers.
func (s *Stream) PublishWithHeaders(ctx context.Context, subject string, headers [][2]string, payload []byte) (PublishAck, error) {
pbHeaders := make([]*pb.MessageHeader, len(headers))
for i, h := range headers {
pbHeaders[i] = &pb.MessageHeader{Key: h[0], Value: h[1]}
}
c := s.client.streamsClient()
r, err := c.Publish(ctx, &pb.PublishRequest{
Stream: s.Name,
Subject: subject,
Payload: payload,
Headers: pbHeaders,
})
if err != nil {
return PublishAck{}, rpcErr(err)
}
if !r.GetSuccess() {
return PublishAck{}, serverErr(r.GetResultCode(), r.GetMessage())
}
return PublishAck{Sequence: r.GetSeq()}, nil
}
// SourcesStatus returns the per-source tail status for this stream.
func (s *Stream) SourcesStatus(ctx context.Context) ([]SourceStatus, error) {
c := s.client.streamsClient()
r, err := c.GetStreamInfo(ctx, &pb.GetStreamInfoRequest{Name: s.Name})
if err != nil {
return nil, rpcErr(err)
}
if !r.GetSuccess() {
return nil, serverErr(r.GetResultCode(), r.GetMessage())
}
out := make([]SourceStatus, len(r.GetSourcesStatus()))
for i, ss := range r.GetSourcesStatus() {
out[i] = SourceStatus{
SourcingStream: s.Name,
SourceStream: ss.GetSourceStream(),
LastSourcedSeq: ss.GetLastSourcedSeq(),
PulledTotal: ss.GetPulledTotal(),
LastError: ss.GetLastError(),
LastErrorTsMs: ss.GetLastErrorTsMs(),
}
}
return out, nil
}
// CreateConsumer creates a new consumer on this stream.
func (s *Stream) CreateConsumer(ctx context.Context, config ConsumerConfig) (*Consumer, error) {
if config.DurableName == "" {
return nil, invalidErr("ConsumerConfig.DurableName is required")
}
c := s.client.streamsClient()
r, err := c.CreateConsumer(ctx, &pb.CreateConsumerRequest{
Stream: s.Name,
Config: config.toPB(),
})
if err != nil {
return nil, rpcErr(err)
}
if !r.GetSuccess() {
return nil, serverErr(r.GetResultCode(), r.GetMessage())
}
return newConsumer(s.client, s.Name, config.DurableName), nil
}
// GetOrCreateConsumer is the idempotent variant.
func (s *Stream) GetOrCreateConsumer(ctx context.Context, config ConsumerConfig) (*Consumer, error) {
if config.DurableName == "" {
return nil, invalidErr("ConsumerConfig.DurableName is required")
}
c, err := s.GetConsumer(ctx, config.DurableName)
if err == nil {
return c, nil
}
if IsServerCode(err, "no_such_consumer") {
return s.CreateConsumer(ctx, config)
}
return nil, err
}
// GetConsumer returns a handle to an existing consumer.
func (s *Stream) GetConsumer(ctx context.Context, name string) (*Consumer, error) {
c := s.client.streamsClient()
r, err := c.GetConsumerInfo(ctx, &pb.GetConsumerInfoRequest{
Stream: s.Name,
Consumer: name,
})
if err != nil {
return nil, rpcErr(err)
}
if !r.GetSuccess() {
return nil, serverErr(r.GetResultCode(), r.GetMessage())
}
return newConsumer(s.client, s.Name, name), nil
}
// DeleteConsumer deletes a consumer by name.
func (s *Stream) DeleteConsumer(ctx context.Context, name string) error {
c := s.client.streamsClient()
r, err := c.DeleteConsumer(ctx, &pb.DeleteConsumerRequest{
Stream: s.Name,
Consumer: name,
})
if err != nil {
return rpcErr(err)
}
if !r.GetSuccess() {
return serverErr(r.GetResultCode(), r.GetMessage())
}
return nil
}
// --- Consumer ---
// DeliverPolicy controls where a consumer starts delivering.
type DeliverPolicy int
const (
DeliverAll DeliverPolicy = iota // from the first message
DeliverNew // from messages published after subscription
DeliverLast // from the last message
DeliverByStartSeq // from a given sequence
DeliverByStartTime // from a given timestamp
)
// ConsumerConfig is the consumer creation config.
type ConsumerConfig struct {
DurableName string
FilterSubject string
DeliverPolicy DeliverPolicy
StartSeq uint64 // for DeliverByStartSeq
StartTimeMs int64 // for DeliverByStartTime
AckWait time.Duration
MaxDeliver uint32
DeliverGroup string
DeadLetterSubject string
}
func (c ConsumerConfig) toPB() *pb.ConsumerConfigPb {
ackWaitMs := uint64(30_000)
if c.AckWait > 0 {
ackWaitMs = uint64(c.AckWait.Milliseconds())
}
maxDeliver := c.MaxDeliver
if maxDeliver == 0 {
maxDeliver = 5
}
var policy *pb.DeliveryPolicyPb
switch c.DeliverPolicy {
case DeliverNew:
now := time.Now().UnixMilli()
policy = &pb.DeliveryPolicyPb{
Type: pb.DeliveryPolicyType_DELIVERY_BY_START_TIME,
StartTimeMs: now,
}
case DeliverLast:
policy = &pb.DeliveryPolicyPb{Type: pb.DeliveryPolicyType_DELIVERY_LAST}
case DeliverByStartSeq:
policy = &pb.DeliveryPolicyPb{
Type: pb.DeliveryPolicyType_DELIVERY_BY_START_SEQ,
StartSeq: c.StartSeq,
}
case DeliverByStartTime:
policy = &pb.DeliveryPolicyPb{
Type: pb.DeliveryPolicyType_DELIVERY_BY_START_TIME,
StartTimeMs: c.StartTimeMs,
}
default: // DeliverAll
policy = &pb.DeliveryPolicyPb{Type: pb.DeliveryPolicyType_DELIVERY_ALL}
}
return &pb.ConsumerConfigPb{
Name: c.DurableName,
FilterSubject: c.FilterSubject,
DeliveryPolicy: policy,
AckWaitMs: ackWaitMs,
MaxDeliver: maxDeliver,
DeliverGroup: c.DeliverGroup,
DeadLetterSubject: c.DeadLetterSubject,
}
}
// Consumer is a consumer handle. Cheap to copy.
type Consumer struct {
client *Client
StreamName string
Name string
}
func newConsumer(c *Client, stream, name string) *Consumer {
return &Consumer{client: c, StreamName: stream, Name: name}
}
// --- Message ---
// Message is a delivered message. Carries enough context to Ack/Nak/Term/
// InProgress through the parent consumer.
type Message struct {
Subject string
Payload []byte
Headers [][2]string
Sequence uint64
TsMs int64
DeliverCount uint32
client *Client
stream string
consumer string
}
func msgFromPB(c *Client, stream, consumer string, m *pb.MessagePb) *Message {
hdrs := make([][2]string, len(m.GetHeaders()))
for i, h := range m.GetHeaders() {
hdrs[i] = [2]string{h.GetKey(), h.GetValue()}
}
return &Message{
Subject: m.GetSubject(),
Payload: m.GetPayload(),
Headers: hdrs,
Sequence: m.GetSeq(),
TsMs: m.GetTsMs(),
DeliverCount: m.GetDeliverCount(),
client: c,
stream: stream,
consumer: consumer,
}
}
// Ack positively acknowledges the message.
func (m *Message) Ack(ctx context.Context) error {
c := m.client.streamsClient()
r, err := c.Ack(ctx, &pb.AckRequest{Stream: m.stream, Consumer: m.consumer, Seq: m.Sequence})
if err != nil {
return rpcErr(err)
}
if !r.GetSuccess() {
return serverErr(r.GetResultCode(), r.GetMessage())
}
return nil
}
// Nak negatively acknowledges — request immediate redelivery.
func (m *Message) Nak(ctx context.Context) error {
return m.NakWithDelay(ctx, 0)
}
// NakWithDelay negatively acknowledges with a redelivery delay.
func (m *Message) NakWithDelay(ctx context.Context, delay time.Duration) error {
c := m.client.streamsClient()
r, err := c.Nak(ctx, &pb.NakRequest{
Stream: m.stream,
Consumer: m.consumer,
Seq: m.Sequence,
DelayMs: uint64(delay.Milliseconds()),
})
if err != nil {
return rpcErr(err)
}
if !r.GetSuccess() {
return serverErr(r.GetResultCode(), r.GetMessage())
}
return nil
}
// Term terminates delivery permanently regardless of max_deliver.
func (m *Message) Term(ctx context.Context) error {
c := m.client.streamsClient()
r, err := c.Term(ctx, &pb.TermRequest{Stream: m.stream, Consumer: m.consumer, Seq: m.Sequence})
if err != nil {
return rpcErr(err)
}
if !r.GetSuccess() {
return serverErr(r.GetResultCode(), r.GetMessage())
}
return nil
}
// InProgress extends the ack_wait window without acking.
func (m *Message) InProgress(ctx context.Context) error {
c := m.client.streamsClient()
r, err := c.InProgress(ctx, &pb.InProgressRequest{Stream: m.stream, Consumer: m.consumer, Seq: m.Sequence})
if err != nil {
return rpcErr(err)
}
if !r.GetSuccess() {
return serverErr(r.GetResultCode(), r.GetMessage())
}
return nil
}
// --- Consumer methods ---
// Messages opens a push-mode subscription. Returns a *Messages iterator.
func (con *Consumer) Messages(ctx context.Context) (*Messages, error) {
return con.MessagesWithBatchSize(ctx, 0)
}
// MessagesWithBatchSize opens a subscription with a custom server-side batch
// hint. 0 = server default.
func (con *Consumer) MessagesWithBatchSize(ctx context.Context, batchSize uint32) (*Messages, error) {
c := con.client.streamsClient()
stream, err := c.Subscribe(ctx, &pb.SubscribeRequest{
Stream: con.StreamName,
Consumer: con.Name,
BatchSize: batchSize,
StopWhenEmpty: false,
})
if err != nil {
return nil, rpcErr(err)
}
return &Messages{
inner: stream,
client: con.client,
stream: con.StreamName,
consumer: con.Name,
}, nil
}
// Fetch pull-style: fetches up to batchSize messages immediately.
func (con *Consumer) Fetch(ctx context.Context, batchSize uint32) ([]*Message, error) {
c := con.client.streamsClient()
r, err := c.Fetch(ctx, &pb.FetchRequest{
Stream: con.StreamName,
Consumer: con.Name,
BatchSize: batchSize,
})
if err != nil {
return nil, rpcErr(err)
}
if !r.GetSuccess() {
return nil, serverErr(r.GetResultCode(), r.GetMessage())
}
msgs := make([]*Message, len(r.GetMessages()))
for i, m := range r.GetMessages() {
msgs[i] = msgFromPB(con.client, con.StreamName, con.Name, m)
}
return msgs, nil
}
// --- Messages iterator ---
// Messages is a push-mode subscription iterator. Call Next to get each
// delivered message. Receiving from Messages is blocking; cancel the ctx
// to stop.
type Messages struct {
inner pb.WaymakerStreamsService_SubscribeClient
client *Client
stream string
consumer string
}
// Next blocks until the next message arrives or an error occurs.
// Returns (nil, nil) on normal end-of-stream.
func (m *Messages) Next() (*Message, error) {
for {
ev, err := m.inner.Recv()
if err != nil {
if err == io.EOF {
return nil, nil
}
return nil, rpcErr(err)
}
switch e := ev.GetEvent().(type) {
case *pb.SubscribeEvent_Message:
return msgFromPB(m.client, m.stream, m.consumer, e.Message), nil
case *pb.SubscribeEvent_Stopped:
if e.Stopped.GetReason() == "" {
return nil, nil
}
return nil, serverErr("subscribe_stopped", e.Stopped.GetReason())
}
}
}
// --- Client entry points for streams ---
// CreateStream creates a new stream.
func (c *Client) CreateStream(ctx context.Context, config StreamConfig) (*Stream, error) {
sc := c.streamsClient()
r, err := sc.CreateStream(ctx, &pb.CreateStreamRequest{Config: config.toPB()})
if err != nil {
return nil, rpcErr(err)
}
if !r.GetSuccess() {
return nil, serverErr(r.GetResultCode(), r.GetMessage())
}
return newStream(c, config.Name), nil
}
// GetStream returns a handle to an existing stream. Confirms existence
// via GetStreamInfo.
func (c *Client) GetStream(ctx context.Context, name string) (*Stream, error) {
sc := c.streamsClient()
r, err := sc.GetStreamInfo(ctx, &pb.GetStreamInfoRequest{Name: name})
if err != nil {
return nil, rpcErr(err)
}
if !r.GetSuccess() {
return nil, serverErr(r.GetResultCode(), r.GetMessage())
}
return newStream(c, name), nil
}
// GetOrCreateStream is the idempotent create-or-get.
func (c *Client) GetOrCreateStream(ctx context.Context, config StreamConfig) (*Stream, error) {
s, err := c.GetStream(ctx, config.Name)
if err == nil {
return s, nil
}
if IsServerCode(err, "no_such_stream") {
return c.CreateStream(ctx, config)
}
return nil, err
}
// UpdateStream applies the mutable subset of a stream's config.
func (c *Client) UpdateStream(ctx context.Context, name string, update StreamUpdate) error {
var maxAgeMs *uint64
if update.MaxAge != nil {
v := uint64(update.MaxAge.Milliseconds())
maxAgeMs = &v
}
sc := c.streamsClient()
r, err := sc.UpdateStream(ctx, &pb.UpdateStreamRequest{
Name: name,
MaxAgeMs: maxAgeMs,
MaxMsgs: update.MaxMessages,
MaxBytes: update.MaxBytes,
MaxMsgBytes: update.MaxMsgBytes,
StrictLimits: update.StrictLimits,
})
if err != nil {
return rpcErr(err)
}
if !r.GetSuccess() {
return serverErr(r.GetResultCode(), r.GetMessage())
}
return nil
}
// DeleteStream deletes a stream and all its consumers.
func (c *Client) DeleteStream(ctx context.Context, name string) error {
sc := c.streamsClient()
r, err := sc.DeleteStream(ctx, &pb.DeleteStreamRequest{Name: name})
if err != nil {
return rpcErr(err)
}
if !r.GetSuccess() {
return serverErr(r.GetResultCode(), r.GetMessage())
}
return nil
}
// GetStreamSources enumerates every (sourcing, source) tail running on
// the node serving the request. Multi-node clusters need to query each
// node.
func (c *Client) GetStreamSources(ctx context.Context) ([]SourceStatus, error) {
sc := c.streamsClient()
r, err := sc.GetStreamSources(ctx, &pb.GetStreamSourcesRequest{})
if err != nil {
return nil, rpcErr(err)
}
if !r.GetSuccess() {
return nil, serverErr(r.GetResultCode(), r.GetMessage())
}
out := make([]SourceStatus, len(r.GetEntries()))
for i, e := range r.GetEntries() {
out[i] = SourceStatus{
SourcingStream: e.GetSourcingStream(),
SourceStream: e.GetSourceStream(),
LastSourcedSeq: e.GetLastSourcedSeq(),
PulledTotal: e.GetPulledTotal(),
LastError: e.GetLastError(),
LastErrorTsMs: e.GetLastErrorTsMs(),
}
}
return out, nil
}

9
go/util.go Normal file
View File

@ -0,0 +1,9 @@
package waymaker
import "crypto/rand"
// cryptoRandRead is a thin wrapper so lock.go can call it without importing
// crypto/rand directly (keeps the import set tidy).
func cryptoRandRead(b []byte) (int, error) {
return rand.Read(b)
}

64
proto/cache.proto Normal file
View File

@ -0,0 +1,64 @@
syntax = "proto3";
package waymaker.cache;
option go_package = "/apis/waymaker_cache";
// ============================================================
// WaymakerCacheService generic TTL / eviction layer over any
// bucket-shaped store (KV, Hash, Set). Methods are deliberately
// minimal at this stage; the design will land alongside the
// first concrete eviction policy (LRU, ARC, expiry-driven).
//
// Phase-5 milestone:
// - This proto pins the crate layout + the wire namespace
// (`waymaker.cache`) so clients don't need to migrate when
// real handlers ship.
// - The server registers the service but every RPC returns
// `Unimplemented` until the policy work lands.
// ============================================================
service WaymakerCacheService {
// Apply a TTL policy to a bucket. `policy_id` selects from
// server-configured policies (initially: `lru`, `expiry`).
rpc AttachPolicy (AttachPolicyRequest) returns (AttachPolicyResponse);
// Detach the policy currently bound to `bucket` (no-op if
// none).
rpc DetachPolicy (DetachPolicyRequest) returns (DetachPolicyResponse);
// Report current cache stats (hit/miss/eviction counters,
// memory footprint) for a bucket.
rpc Stats (StatsRequest) returns (StatsResponse);
}
message AttachPolicyRequest {
string bucket = 1;
string policy_id = 2;
// Policy-specific knobs (e.g. `max_entries`, `default_ttl_ms`)
// interpretation is server-side.
map<string, string> params = 3;
}
message AttachPolicyResponse {
bool success = 1;
string result_code = 2;
string message = 3;
}
message DetachPolicyRequest { string bucket = 1; }
message DetachPolicyResponse {
bool success = 1;
string result_code = 2;
string message = 3;
}
message StatsRequest { string bucket = 1; }
message StatsResponse {
bool success = 1;
string result_code = 2;
string message = 3;
uint64 hit_count = 4;
uint64 miss_count = 5;
uint64 eviction_count = 6;
uint64 size_bytes = 7;
uint64 entry_count = 8;
}

288
proto/collections.proto Normal file
View File

@ -0,0 +1,288 @@
syntax = "proto3";
package waymaker.collections;
option go_package = "/apis/waymaker_collections";
// ============================================================
// WaymakerCollectionsService Hash / Set / Queue data
// structures layered on streams. Each store name is the
// underlying stream name; subjects encode the
// (hash_key|set_key) + (field|member) tuple; member-existence
// uses a "1" marker payload (Set) or per-field values (Hash).
//
// Lives in its own proto + crate so the wire surface is
// composable; handlers continue to share `StreamsController`
// state in the server crate.
// ============================================================
service WaymakerCollectionsService {
// ----- Hash -------------------------------------------------
rpc CreateHashStore (CreateHashStoreRequest) returns (CreateHashStoreResponse);
rpc DeleteHashStore (DeleteHashStoreRequest) returns (DeleteHashStoreResponse);
rpc HashSet (HashSetRequest) returns (HashSetResponse);
rpc HashGet (HashGetRequest) returns (HashGetResponse);
rpc HashExists (HashExistsRequest) returns (HashExistsResponse);
rpc HashDelete (HashDeleteRequest) returns (HashDeleteResponse);
rpc HashGetAll (HashGetAllRequest) returns (HashGetAllResponse);
rpc HashFields (HashFieldsRequest) returns (HashFieldsResponse);
rpc HashLen (HashLenRequest) returns (HashLenResponse);
// ----- Set --------------------------------------------------
rpc CreateSetStore (CreateSetStoreRequest) returns (CreateSetStoreResponse);
rpc DeleteSetStore (DeleteSetStoreRequest) returns (DeleteSetStoreResponse);
rpc SetAdd (SetAddRequest) returns (SetAddResponse);
rpc SetRemove (SetRemoveRequest) returns (SetRemoveResponse);
rpc SetIsMember (SetIsMemberRequest) returns (SetIsMemberResponse);
rpc SetMembers (SetMembersRequest) returns (SetMembersResponse);
rpc SetLen (SetLenRequest) returns (SetLenResponse);
// ----- Queue ------------------------------------------------
rpc CreateQueue (CreateQueueRequest) returns (CreateQueueResponse);
rpc DeleteQueue (DeleteQueueRequest) returns (DeleteQueueResponse);
rpc QueuePush (QueuePushRequest) returns (QueuePushResponse);
rpc QueuePop (QueuePopRequest) returns (QueuePopResponse);
rpc QueueRange (QueueRangeRequest) returns (QueueRangeResponse);
rpc QueueLen (QueueLenRequest) returns (QueueLenResponse);
}
// ===== Hash ==================================================
message CreateHashStoreRequest {
string name = 1;
uint64 max_bytes = 2;
bool ephemeral = 3;
}
message CreateHashStoreResponse {
bool success = 1;
string result_code = 2;
string message = 3;
}
message DeleteHashStoreRequest { string name = 1; }
message DeleteHashStoreResponse {
bool success = 1;
string result_code = 2;
string message = 3;
}
message HashSetRequest {
string bucket = 1;
string hash_key = 2;
string field = 3;
bytes value = 4;
}
message HashSetResponse {
bool success = 1;
string result_code = 2;
string message = 3;
uint64 revision = 4;
}
message HashGetRequest {
string bucket = 1;
string hash_key = 2;
string field = 3;
}
message HashGetResponse {
bool success = 1;
string result_code = 2;
string message = 3;
optional bytes value = 4;
uint64 revision = 5;
}
message HashExistsRequest {
string bucket = 1;
string hash_key = 2;
string field = 3;
}
message HashExistsResponse {
bool success = 1;
string result_code = 2;
string message = 3;
bool exists = 4;
}
message HashDeleteRequest {
string bucket = 1;
string hash_key = 2;
string field = 3;
}
message HashDeleteResponse {
bool success = 1;
string result_code = 2;
string message = 3;
}
message HashGetAllRequest {
string bucket = 1;
string hash_key = 2;
}
message HashGetAllResponse {
bool success = 1;
string result_code = 2;
string message = 3;
repeated HashFieldEntry entries = 4;
}
message HashFieldEntry {
string field = 1;
bytes value = 2;
uint64 revision = 3;
}
message HashFieldsRequest {
string bucket = 1;
string hash_key = 2;
}
message HashFieldsResponse {
bool success = 1;
string result_code = 2;
string message = 3;
repeated string fields = 4;
}
message HashLenRequest {
string bucket = 1;
string hash_key = 2;
}
message HashLenResponse {
bool success = 1;
string result_code = 2;
string message = 3;
uint64 count = 4;
}
// ===== Set ====================================================
message CreateSetStoreRequest {
string name = 1;
uint64 max_bytes = 2;
bool ephemeral = 3;
}
message CreateSetStoreResponse {
bool success = 1;
string result_code = 2;
string message = 3;
}
message DeleteSetStoreRequest { string name = 1; }
message DeleteSetStoreResponse {
bool success = 1;
string result_code = 2;
string message = 3;
}
message SetAddRequest {
string bucket = 1;
string set_key = 2;
string member = 3;
}
message SetAddResponse {
bool success = 1;
string result_code = 2;
string message = 3;
}
message SetRemoveRequest {
string bucket = 1;
string set_key = 2;
string member = 3;
}
message SetRemoveResponse {
bool success = 1;
string result_code = 2;
string message = 3;
}
message SetIsMemberRequest {
string bucket = 1;
string set_key = 2;
string member = 3;
}
message SetIsMemberResponse {
bool success = 1;
string result_code = 2;
string message = 3;
bool is_member = 4;
}
message SetMembersRequest {
string bucket = 1;
string set_key = 2;
}
message SetMembersResponse {
bool success = 1;
string result_code = 2;
string message = 3;
repeated string members = 4;
}
message SetLenRequest {
string bucket = 1;
string set_key = 2;
}
message SetLenResponse {
bool success = 1;
string result_code = 2;
string message = 3;
uint64 count = 4;
}
// ===== Queue =================================================
message CreateQueueRequest {
string name = 1;
uint64 max_bytes = 2;
uint64 max_messages = 3;
bool ephemeral = 4;
}
message CreateQueueResponse {
bool success = 1;
string result_code = 2;
string message = 3;
}
message DeleteQueueRequest { string name = 1; }
message DeleteQueueResponse {
bool success = 1;
string result_code = 2;
string message = 3;
}
message QueuePushRequest {
string bucket = 1;
bytes value = 2;
}
message QueuePushResponse {
bool success = 1;
string result_code = 2;
string message = 3;
uint64 sequence = 4;
}
message QueuePopRequest {
string bucket = 1;
}
message QueuePopResponse {
bool success = 1;
string result_code = 2;
string message = 3;
optional bytes value = 4;
}
message QueueRangeRequest {
string bucket = 1;
uint64 from_sequence = 2;
uint64 limit = 3;
}
message QueueRangeResponse {
bool success = 1;
string result_code = 2;
string message = 3;
repeated bytes values = 4;
}
message QueueLenRequest { string bucket = 1; }
message QueueLenResponse {
bool success = 1;
string result_code = 2;
string message = 3;
uint64 count = 4;
}

207
proto/kv.proto Normal file
View File

@ -0,0 +1,207 @@
syntax = "proto3";
package waymaker.kv;
option go_package = "/apis/waymaker_kv";
// ============================================================
// WaymakerKvService distributed key-value store implemented
// as a thin façade over the streams subsystem. Each KV bucket
// is backed by a stream named `kv:<bucket>`; KV revisions are
// stream sequence numbers; KV deletes are tombstones (per the
// WIRE_SPEC.md tombstone header convention).
//
// The service lives in its own proto so the wire surface is
// composable: an operator could ship a KV-only build, or layer
// KV on top of someone else's streams implementation.
// ============================================================
service WaymakerKvService {
// ----- Bucket lifecycle -------------------------------------
rpc CreateBucket (KvCreateBucketRequest) returns (KvCreateBucketResponse);
rpc DeleteBucket (KvDeleteBucketRequest) returns (KvDeleteBucketResponse);
// ----- Mutations --------------------------------------------
rpc Put (KvPutRequest) returns (KvPutResponse);
// CAS create succeeds only when the key has never been
// written or its current value is a tombstone.
rpc Create (KvCreateRequest) returns (KvPutResponse);
// CAS update succeeds only when `expected_revision`
// matches the server-side revision.
rpc Update (KvUpdateRequest) returns (KvPutResponse);
rpc Delete (KvDeleteRequest) returns (KvDeleteResponse);
// ----- Reads ------------------------------------------------
rpc Get (KvGetRequest) returns (KvGetResponse);
rpc Keys (KvKeysRequest) returns (KvKeysResponse);
rpc History (KvHistoryRequest) returns (KvHistoryResponse);
// ----- TTL refresh ------------------------------------------
rpc Touch (KvTouchRequest) returns (KvPutResponse);
// ----- Watch ------------------------------------------------
// Server-streamed event flow for a single bucket. When `key`
// is empty, every put/delete in the bucket fans out; when
// `key` is set, only events at that key are emitted.
rpc Watch (KvWatchRequest) returns (stream KvWatchEvent);
}
message KvCreateBucketRequest {
string bucket = 1;
uint64 max_bytes = 2; // 0 = unbounded
uint64 max_value_size = 3; // 0 = no per-value cap
// Bucket-level TTL (ms). 0 = no time-based eviction.
// Independent of per-key TTL set via Put.
uint64 max_age_ms = 4;
bool ephemeral = 5;
// Per-key revision cap. 0 (default) = unbounded history depth
// is then bounded only by the bucket's stream-level retention
// (max_age_ms / max_bytes). When N > 0, after each successful
// write to a key, older revisions of *that key* beyond the N
// most recent are dropped via per-message pruning. Useful when
// one bucket hosts many keys with very different write rates
// a fast-churning key won't crowd out older revisions of a
// slow-changing key. NATS JetStream's "MaxRevisions" semantic.
uint64 max_revisions = 6;
}
message KvCreateBucketResponse {
bool success = 1;
string result_code = 2; // "ok" | "already_exists" | "invalid_config" | "internal"
string message = 3;
}
message KvDeleteBucketRequest { string bucket = 1; }
message KvDeleteBucketResponse {
bool success = 1;
string result_code = 2; // "ok" | "no_such_bucket" | "internal"
string message = 3;
}
message KvPutRequest {
string bucket = 1;
string key = 2;
bytes value = 3;
uint64 ttl_ms = 4; // per-key TTL; 0 = no TTL
}
message KvCreateRequest {
string bucket = 1;
string key = 2;
bytes value = 3;
uint64 ttl_ms = 4;
}
message KvUpdateRequest {
string bucket = 1;
string key = 2;
bytes value = 3;
// The revision the caller believes is current. Server returns
// wrong_revision if mismatch.
uint64 expected_revision = 4;
uint64 ttl_ms = 5;
}
message KvPutResponse {
bool success = 1;
// "ok" | "no_such_bucket" | "wrong_revision" | "invalid_key" | "internal"
string result_code = 2;
string message = 3;
uint64 revision = 4;
}
message KvGetRequest {
string bucket = 1;
string key = 2;
}
message KvGetResponse {
bool success = 1;
string result_code = 2;
string message = 3;
optional KvEntry entry = 4;
}
message KvEntry {
bytes value = 1;
uint64 revision = 2;
int64 ts_ms = 3;
}
message KvDeleteRequest {
string bucket = 1;
string key = 2;
}
message KvDeleteResponse {
bool success = 1;
string result_code = 2;
string message = 3;
uint64 revision = 4;
}
message KvKeysRequest { string bucket = 1; }
message KvKeysResponse {
bool success = 1;
string result_code = 2;
string message = 3;
repeated KvKeyEntry entries = 4;
}
message KvKeyEntry {
string key = 1;
uint64 revision = 2;
bool deleted = 3;
}
message KvHistoryRequest {
string bucket = 1;
string key = 2;
uint64 from_revision = 3; // 0 = from beginning
uint64 limit = 4; // 0 = server default
}
message KvHistoryResponse {
bool success = 1;
string result_code = 2;
string message = 3;
repeated KvHistoryEntry entries = 4;
}
message KvHistoryEntry {
bytes value = 1;
uint64 revision = 2;
int64 ts_ms = 3;
bool deleted = 4;
}
message KvTouchRequest {
string bucket = 1;
string key = 2;
uint64 ttl_ms = 3;
}
message KvWatchRequest {
string bucket = 1;
string key = 2; // empty = whole bucket
}
message KvWatchEvent {
oneof event {
KvPutEvent put = 1;
KvDeleteEvent delete = 2;
}
}
message KvPutEvent {
string key = 1;
bytes value = 2;
uint64 revision = 3;
int64 ts_ms = 4;
}
message KvDeleteEvent {
string key = 1;
uint64 revision = 2;
int64 ts_ms = 3;
}

366
proto/sketches.proto Normal file
View File

@ -0,0 +1,366 @@
syntax = "proto3";
package waymaker.sketches;
option go_package = "/apis/waymaker_sketches";
// In-memory probabilistic data structures: Bloom filter,
// HyperLogLog, Count-Min Sketch, Top-K, t-digest.
//
// Storage model:
// - State is opaque binary per filter, owned by the ring's
// hash-owner for the filter's name.
// - Periodic snapshot replication pushes the bytes to N-1
// secondaries via ReplicateProbState. Adoption sweep promotes
// replicas to primary on ring shifts.
// - Disk persistence: every ~60s, owned filters snapshot to
// <data-dir>/probabilistic/<type>/<name>.bin via tempfile +
// rename. Hydrated on controller startup.
// - Ephemeral by default survives node failover + process
// restart, full-cluster restart without prior snapshot loses
// state.
//
// See specs/WIRE_SPEC.md (top-level) for the serialization byte
// layout per type cross-language clients implement against that
// document for the replication state-transfer protocol.
service WaymakerSketchesService {
// ----- Bloom filter -----
rpc BloomReserve (BloomReserveRequest) returns (BloomReserveResponse);
rpc BloomAdd (BloomAddRequest) returns (BloomAddResponse);
rpc BloomMultiAdd (BloomMultiAddRequest) returns (BloomMultiAddResponse);
rpc BloomExists (BloomExistsRequest) returns (BloomExistsResponse);
rpc BloomMultiExists (BloomMultiExistsRequest) returns (BloomMultiExistsResponse);
rpc BloomInfo (BloomInfoRequest) returns (BloomInfoResponse);
rpc BloomDelete (BloomDeleteRequest) returns (BloomDeleteResponse);
// ----- HyperLogLog -----
rpc HllReserve (HllReserveRequest) returns (HllReserveResponse);
rpc HllAdd (HllAddRequest) returns (HllAddResponse);
rpc HllCount (HllCountRequest) returns (HllCountResponse);
rpc HllMerge (HllMergeRequest) returns (HllMergeResponse);
rpc HllDelete (HllDeleteRequest) returns (HllDeleteResponse);
// ----- Count-Min Sketch -----
rpc CmsReserve (CmsReserveRequest) returns (CmsReserveResponse);
rpc CmsIncrBy (CmsIncrByRequest) returns (CmsIncrByResponse);
rpc CmsQuery (CmsQueryRequest) returns (CmsQueryResponse);
rpc CmsDelete (CmsDeleteRequest) returns (CmsDeleteResponse);
// ----- Top-K -----
rpc TopKReserve (TopKReserveRequest) returns (TopKReserveResponse);
rpc TopKAdd (TopKAddRequest) returns (TopKAddResponse);
rpc TopKQuery (TopKQueryRequest) returns (TopKQueryResponse);
rpc TopKList (TopKListRequest) returns (TopKListResponse);
rpc TopKDelete (TopKDeleteRequest) returns (TopKDeleteResponse);
// ----- t-digest -----
rpc TDigestCreate (TDigestCreateRequest) returns (TDigestCreateResponse);
rpc TDigestAdd (TDigestAddRequest) returns (TDigestAddResponse);
rpc TDigestQuantile (TDigestQuantileRequest) returns (TDigestQuantileResponse);
rpc TDigestMinMax (TDigestMinMaxRequest) returns (TDigestMinMaxResponse);
rpc TDigestDelete (TDigestDeleteRequest) returns (TDigestDeleteResponse);
// Internal: snapshot replication. Primary pushes serialized
// filter state to N-1 secondaries periodically. Version counter
// dedupes out-of-order pushes.
rpc ReplicateProbState (ReplicateProbStateRequest) returns (ReplicateProbStateResponse);
}
// ===== Bloom filter ===========================================
message BloomReserveRequest {
string name = 1;
uint64 capacity = 2;
double error_rate = 3;
}
message BloomReserveResponse {
bool success = 1;
string result_code = 2;
string message = 3;
}
message BloomAddRequest {
string name = 1;
bytes item = 2;
}
message BloomAddResponse {
bool success = 1;
string result_code = 2;
string message = 3;
}
message BloomMultiAddRequest {
string name = 1;
repeated bytes items = 2;
}
message BloomMultiAddResponse {
bool success = 1;
string result_code = 2;
string message = 3;
}
message BloomExistsRequest {
string name = 1;
bytes item = 2;
}
message BloomExistsResponse {
bool success = 1;
string result_code = 2;
string message = 3;
bool exists = 4;
}
message BloomMultiExistsRequest {
string name = 1;
repeated bytes items = 2;
}
message BloomMultiExistsResponse {
bool success = 1;
string result_code = 2;
string message = 3;
repeated bool exists = 4;
}
message BloomInfoRequest { string name = 1; }
message BloomInfoResponse {
bool success = 1;
string result_code = 2;
string message = 3;
uint64 capacity = 4;
double error_rate = 5;
uint64 bits_set = 6;
uint64 bit_count = 7;
uint32 hash_count = 8;
uint64 items_added = 9;
}
message BloomDeleteRequest { string name = 1; }
message BloomDeleteResponse {
bool success = 1;
string result_code = 2;
string message = 3;
}
// ===== HyperLogLog ============================================
message HllReserveRequest {
string name = 1;
uint32 precision = 2;
}
message HllReserveResponse {
bool success = 1;
string result_code = 2;
string message = 3;
}
message HllAddRequest {
string name = 1;
repeated bytes items = 2;
}
message HllAddResponse {
bool success = 1;
string result_code = 2;
string message = 3;
}
message HllCountRequest { string name = 1; }
message HllCountResponse {
bool success = 1;
string result_code = 2;
string message = 3;
uint64 estimate = 4;
}
message HllMergeRequest {
string destination = 1;
repeated string sources = 2;
}
message HllMergeResponse {
bool success = 1;
string result_code = 2;
string message = 3;
}
message HllDeleteRequest { string name = 1; }
message HllDeleteResponse {
bool success = 1;
string result_code = 2;
string message = 3;
}
// ===== Count-Min Sketch =======================================
message CmsReserveRequest {
string name = 1;
uint64 width = 2;
uint64 depth = 3;
}
message CmsReserveResponse {
bool success = 1;
string result_code = 2;
string message = 3;
}
message CmsIncrByItem {
bytes item = 1;
uint64 count = 2;
}
message CmsIncrByRequest {
string name = 1;
repeated CmsIncrByItem items = 2;
}
message CmsIncrByResponse {
bool success = 1;
string result_code = 2;
string message = 3;
repeated uint64 counts = 4;
}
message CmsQueryRequest {
string name = 1;
repeated bytes items = 2;
}
message CmsQueryResponse {
bool success = 1;
string result_code = 2;
string message = 3;
repeated uint64 counts = 4;
}
message CmsDeleteRequest { string name = 1; }
message CmsDeleteResponse {
bool success = 1;
string result_code = 2;
string message = 3;
}
// ===== Top-K ==================================================
message TopKReserveRequest {
string name = 1;
uint32 k = 2;
uint64 width = 3;
uint64 depth = 4;
double decay = 5;
}
message TopKReserveResponse {
bool success = 1;
string result_code = 2;
string message = 3;
}
message TopKAddRequest {
string name = 1;
repeated bytes items = 2;
}
message TopKAddResponse {
bool success = 1;
string result_code = 2;
string message = 3;
repeated bytes evicted = 4;
}
message TopKQueryRequest {
string name = 1;
repeated bytes items = 2;
}
message TopKQueryResponse {
bool success = 1;
string result_code = 2;
string message = 3;
repeated bool in_top_k = 4;
}
message TopKListRequest { string name = 1; }
message TopKListResponse {
bool success = 1;
string result_code = 2;
string message = 3;
repeated TopKEntry entries = 4;
}
message TopKEntry {
bytes item = 1;
uint64 count = 2;
}
message TopKDeleteRequest { string name = 1; }
message TopKDeleteResponse {
bool success = 1;
string result_code = 2;
string message = 3;
}
// ===== t-digest ===============================================
message TDigestCreateRequest {
string name = 1;
uint32 compression = 2;
}
message TDigestCreateResponse {
bool success = 1;
string result_code = 2;
string message = 3;
}
message TDigestAddRequest {
string name = 1;
repeated double values = 2;
}
message TDigestAddResponse {
bool success = 1;
string result_code = 2;
string message = 3;
}
message TDigestQuantileRequest {
string name = 1;
repeated double quantiles = 2;
}
message TDigestQuantileResponse {
bool success = 1;
string result_code = 2;
string message = 3;
repeated double values = 4;
}
message TDigestMinMaxRequest { string name = 1; }
message TDigestMinMaxResponse {
bool success = 1;
string result_code = 2;
string message = 3;
double min = 4;
double max = 5;
}
message TDigestDeleteRequest { string name = 1; }
message TDigestDeleteResponse {
bool success = 1;
string result_code = 2;
string message = 3;
}
// ===== Internal: snapshot replication =========================
enum ProbType {
PROB_UNSPECIFIED = 0;
PROB_BLOOM = 1;
PROB_HLL = 2;
PROB_CMS = 3;
PROB_TOPK = 4;
PROB_TDIGEST = 5;
}
message ReplicateProbStateRequest {
ProbType type = 1;
string name = 2;
// Opaque binary snapshot see specs/WIRE_SPEC.md "Probabilistic
// subsystem" for the per-type byte layout.
bytes snapshot = 3;
uint64 version = 4;
}
message ReplicateProbStateResponse {
bool success = 1;
string result_code = 2;
string message = 3;
}

276
proto/waymaker_locks.proto Normal file
View File

@ -0,0 +1,276 @@
syntax = "proto3";
package waymaker;
// Specifies the Go package for the generated code.
option go_package = "/apis/waymaker";
// VERSION COMPATIBILITY POLICY
// ----------------------------
// This proto is the wire contract for the WaymakerService gRPC API.
// To preserve backwards compatibility across rolling upgrades:
//
// 1. Existing field numbers MUST NOT be reused. To remove a field,
// add a `reserved <number>;` line instead.
// 2. Existing field types MUST NOT change.
// 3. New fields MUST be `optional` (proto3 default) and SHOULD have
// sensible zero-value semantics so old clients setting nothing
// and old servers ignoring the field both stay correct.
// 4. New enum values MUST be appended at the end. Never renumber.
// 5. The package name and service name MUST NOT change.
// 6. RPC removal is a major break bump to a new package
// (e.g. `waymaker.v2`) instead.
//
// Reserved field numbers below are tombstones for fields that were
// considered but not yet shipped; do not reuse them.
// WaymakerService defines a gRPC service for managing distributed locks.
service WaymakerService {
// Lock attempts to acquire a lock based on the provided LockRequest.
// The response is a stream of LockEvent messages that provide updates
// on the status of the lock acquisition.
rpc Lock (LockRequest) returns (stream LockEvent) {}
// ReadLock attempts to acquire a read lock, which allows multiple readers
// but no writers to hold the lock simultaneously. The response is a stream
// of LockEvent messages that provide updates on the status of the lock acquisition.
rpc ReadLock (LockRequest) returns (stream LockEvent) {}
// UnLock releases a previously acquired lock based on the provided UnLockRequest.
// The response is an UnLockResponse indicating the success or failure of the operation.
rpc UnLock (UnLockRequest) returns (UnLockResponse) {}
// LeaseStatus retrieves the current status of a lock lease based on the provided
// LeaseStatusRequest. The response is a LeaseStatusResponse containing details
// about the lease.
rpc LeaseStatus (LeaseStatusRequest) returns (LeaseStatusResponse) {}
// ExtendLease extends the lease of an already acquired lock based on the provided
// ExtendLeaseRequest. The response is an ExtendLeaseResponse indicating the success
// or failure of the operation and the updated lease information.
rpc ExtendLease (ExtendLeaseRequest) returns (ExtendLeaseResponse) {}
// MultiLock acquires N locks atomically all keys are granted or none are.
// The server sorts keys lexicographically to guarantee deadlock-free
// ordering between any two MultiLock callers (without this, callers asking
// for (k1,k2) and (k2,k1) could deadlock under contention). `max_wait_period`
// applies to the whole batch as a single deadline, not per key. On any
// failure the server releases every key it already acquired in this batch
// before returning. Returns a unary response no streaming.
rpc MultiLock (MultiLockRequest) returns (MultiLockResponse) {}
// ListAcquiredLocks returns every lock CURRENTLY HELD on the node serving the
// request (the node that owns each key via the consistent-hash ring). It is a
// read-only operator/introspection surface (waymaker-ctl `locks list`); it does
// NOT include waiters. An optional `key_prefix` filters the result server-side.
// In a multi-node cluster, call it on each node to see the full picture, since
// each node only holds the locks for the keys it owns.
rpc ListAcquiredLocks (ListAcquiredLocksRequest) returns (ListAcquiredLocksResponse) {}
}
// LockEventType defines the various types of events that can occur during
// the lock acquisition process.
enum LockEventType {
Unknown = 0; // Default value when the event type is not known.
Waiting = 1; // Indicates that the lock request is waiting to be granted.
Acquired = 2; // Indicates that the lock has been successfully acquired.
Failed = 3; // Indicates that the lock request has failed.
Expired = 4; // Indicates that the lock has expired.
Heartbeat = 5; // Periodic event indicating that the lock is still active.
}
// FenceScope controls ONE thing: how durable the per-key fence_token
// counter (the monotonic uint64) is across failures. Three things it does
// NOT control do not conflate them with the scope:
//
// * Whether a HELD LOCK survives a node loss / rollout. That is
// cluster.replication-factor plus secondary adoption, and it already
// applies to every non-Ephemeral lock regardless of scope. A stronger
// scope does not make a lock survive; a replicated lease does.
// * Client-side transparency across a primary bounce. The lock client
// transparently re-binds its event stream and re-confirms ownership
// after a disconnect, but that is a client-lib behaviour no scope
// value changes it.
// * Mutual exclusion. Holding the lock is NOT, by itself, a guarantee
// that no one else acts. The holder MUST validate fence_token at its
// own side effect (the DB write / object PUT) and reject anything
// carrying a fence below the last one it durably committed. Even
// ScopeQuorum does not let you skip that check an all-at-once
// cluster restart can still drop an in-memory token. See USAGE.md
// "Fence tokens" for the enforcement rule.
//
// Unspecified: server treats as Ephemeral.
// Ephemeral: per-key counter in RAM on the owning node. Resets on
// process restart or hash-ring rebalance. Fast no I/O.
// Right for rate limiting, cache lockout, advisory locks,
// anywhere fence resets across failure are tolerable.
// Local: per-key counter persisted to disk on the owning node.
// Survives process restart on the same node. Still resets
// on hash-ring rebalance (a different node has its own
// disk). One fsync per acquire (~1-5ms on SSD).
// Quorum: Raft-replicated per-key counter cluster-wide monotonic,
// survives any single-node failure (the surviving quorum
// keeps the count). One Raft commit per acquire (~5ms).
// Requires the cluster Raft backend to be wired; a
// single-node or test build returns BadInput for this
// scope. Pick this when an external resource fences on the
// token and two holders must never see fences that fail to
// prove an ordering. (Named Quorum, not Global: the
// guarantee is "a Raft quorum agrees on the count", which
// carries its own limit and makes no geographic claim.)
enum FenceScope {
ScopeUnspecified = 0;
ScopeEphemeral = 1;
ScopeLocal = 2;
// Wire value 3 is unchanged from the former ScopeGlobal old and new
// binaries interoperate mid-rollout; only the symbol name changed.
ScopeQuorum = 3;
}
// LockRequest defines the parameters for requesting a lock.
message LockRequest {
string key = 1; // The unique key representing the lock.
uint32 max_wait_period = 2; // The maximum time (in milliseconds) to wait for the lock to be granted.
uint32 max_lease_period = 3; // The maximum time (in milliseconds) the lock can be held.
uint32 priority = 4; // The priority level of the lock request.
string requester_info = 10; // Additional information about the requester.
string requester_application = 11; // The name of the application making the request.
string request_id = 12; // Idempotency key for retries of the same logical acquire request.
FenceScope fence_scope = 13; // Persistence/durability tier for fence_token. Defaults to Ephemeral.
}
// LockEvent represents an event related to the lock acquisition process.
message LockEvent {
bool success = 1; // Indicates whether the event was successful.
LockEventType event_type = 2; // The type of event that occurred.
string message = 3; // A message providing additional details about the event.
string id = 4; // The unique identifier of the lock.
string key = 5; // The key associated with the lock.
int64 lease_expires_at = 6; // The timestamp (in Unix milliseconds) when the lease expires.
int64 acquired_at = 7; // The timestamp (in Unix milliseconds) when the lock was acquired.
int64 waiting_expires_at = 8; // The timestamp (in Unix milliseconds) when the waiting period expires.
// Monotonic-per-key fence token assigned at acquire. Increments by 1 per
// successful acquisition of `key`. 0 on non-Acquired events.
//
// Held in RAM on the consistent-hash-owning node. Monotonic within that
// node's process lifetime; resets to 0 across node restart, crash, or
// hash-ring rebalance. This is intentional see README "Known
// limitations" and "When to use waymaker" for the use cases this suits
// vs. when to reach for a different tool (etcd / ZooKeeper / Consul).
uint64 fence_token = 9;
}
// UnLockRequest defines the parameters for releasing a lock.
message UnLockRequest {
string key = 1; // The unique key representing the lock.
string id = 2; // The unique identifier of the lock to be released.
}
// UnLockResponse represents the response to an UnLock request.
message UnLockResponse {
bool success = 1; // Indicates whether the unlock operation was successful.
string result_code = 2; // A code indicating the result of the unlock operation.
string message = 3; // A message providing additional details about the unlock operation.
}
// Lease represents the details of a lock lease.
message Lease {
string id = 1; // The unique identifier of the lock.
string key = 2; // The key associated with the lock.
bool acquired = 3; // Indicates whether the lock has been acquired.
// Field 4 was historically unused; do not reuse.
reserved 4;
uint32 priority = 5; // The priority level of the lock.
int64 created_at = 6; // The timestamp (in Unix milliseconds) when the lock was created.
int64 lease_expires_at = 7; // The timestamp (in Unix milliseconds) when the lease expires.
int64 waiting_expires_at = 8;// The timestamp (in Unix milliseconds) when the waiting period expires.
// Fence token assigned at acquire. See LockEvent.fence_token caveats.
uint64 fence_token = 9;
}
// ExtendLeaseRequest defines the parameters for extending the lease of a lock.
message ExtendLeaseRequest {
string key = 1; // The unique key representing the lock.
string id = 2; // The unique identifier of the lock to extend the lease for.
uint32 lease_timeout = 3; // The additional time (in milliseconds) to extend the lease.
// The new lease expiration will be the current time + lease_timeout.
}
// ExtendLeaseResponse represents the response to an ExtendLease request.
message ExtendLeaseResponse {
bool success = 1; // Indicates whether the lease extension was successful.
string result_code = 2; // A code indicating the result of the lease extension.
string message = 3; // A message providing additional details about the lease extension.
Lease lease = 4; // The updated lease details after the extension.
}
// LeaseStatusRequest defines the parameters for retrieving the status of a lock lease.
message LeaseStatusRequest {
string key = 1; // The unique key representing the lock.
string id = 2; // The unique identifier of the lock to check the status of.
}
// LeaseStatusResponse represents the response to a LeaseStatus request.
message LeaseStatusResponse {
bool success = 1; // Indicates whether the lease status retrieval was successful.
string result_code = 2; // A code indicating the result of the lease status retrieval.
string message = 3; // A message providing additional details about the lease status retrieval.
Lease lease = 4; // The current lease details for the lock.
}
// A single (key, lock-kind) entry inside a MultiLockRequest.
message MultiLockKey {
string key = 1; // The unique key representing the lock.
bool write_lock = 2; // true = exclusive (write), false = shared (read).
}
// MultiLockRequest defines the parameters for atomically acquiring N locks.
// See MultiLock RPC docs for ordering and rollback semantics.
message MultiLockRequest {
repeated MultiLockKey keys = 1; // 1..N keys to acquire. Re-ordered server-side.
uint32 max_wait_period = 2; // Total batch deadline (ms). Per-key budget is the remainder.
uint32 max_lease_period = 3; // Per-key lease length (ms).
uint32 priority = 4; // Priority applied to every key.
string requester_info = 10;
string requester_application = 11;
string request_id = 12; // Idempotency key for the batch.
FenceScope fence_scope = 13; // Applies to every key in the batch.
}
// MultiLockResponse unary result of a MultiLock attempt.
message MultiLockResponse {
bool success = 1; // true iff all keys were acquired.
string result_code = 2; // "ok" | "timeout" | "no_keys" | "invalid_scope" | "internal"
string message = 3; // Free-form detail; empty on success.
// Populated only on success, in lexicographic key order (the order the
// server acquired them in). On failure this is empty and any locks
// briefly held during the attempt have already been released.
repeated Lease leases = 4;
}
// ListAcquiredLocksRequest filter for ListAcquiredLocks.
message ListAcquiredLocksRequest {
string key_prefix = 1; // Optional; empty = every held key on this node.
}
// AcquiredLock one lock currently HELD (not waiting) on the serving node.
message AcquiredLock {
string key = 1; // The lock key.
string lock_id = 2; // The holder's unique lock id.
bool write_lock = 3; // true = exclusive (write); false = shared (read).
uint32 priority = 4; // Priority the lock was acquired at.
uint64 fence_token = 5; // Fence token assigned at acquire.
int64 lease_expires_at = 6; // Lease expiry (epoch ms).
int64 acquired_at = 7; // Acquire time (epoch ms).
string request_id = 8; // Idempotency key of the acquire.
string requester_info = 9; // Caller-supplied requester metadata (free-form string).
string requester_application = 10; // Caller-supplied application name.
}
// ListAcquiredLocksResponse held locks on the serving node.
message ListAcquiredLocksResponse {
bool success = 1;
repeated AcquiredLock locks = 2;
}

1864
proto/waymaker_streams.proto Normal file

File diff suppressed because it is too large Load Diff

34
rust/Cargo.toml Normal file
View File

@ -0,0 +1,34 @@
[package]
name = "waymaker-client"
version = "0.1.27"
edition = "2021"
description = "Official Rust client for waymaker — locks, streams, KV, collections, sketches, cache, object store"
repository = "https://git.awesomike.com/pub/waymaker-client"
license = "MIT OR Apache-2.0"
[lib]
path = "src/lib.rs"
[dependencies]
# Async runtime. `sync` for watch/Notify in the Lock hold loop; no `signal`
# (that was for the operator CLI, which stays in the waymaker repo).
tokio = { version = "1", default-features = false, features = [
"rt-multi-thread", "macros", "net", "time", "sync"
] }
tokio-stream = { version = "0.1", default-features = false }
futures = "0.3"
thiserror = "2"
bytes = "1"
# gRPC client. tls-ring lets the client talk to a TLS-protected waymaker
# without dragging in any server-side TLS termination code.
tonic = { version = "0.14", features = ["codegen", "tls-ring"] }
tonic-prost = "0.14"
prost = "0.14"
# request_id idempotency keys.
uuid = { version = "1", features = ["v4", "v7", "fast-rng"] }
[build-dependencies]
# Compiles the vendored protos in ../proto into client stubs at build time.
tonic-prost-build = "0.14"

52
rust/README.md Normal file
View File

@ -0,0 +1,52 @@
# waymaker-client (Rust)
Official Rust client for [waymaker](https://git.awesomike.com/dev/waymaker).
```toml
[dependencies]
waymaker-client = { git = "https://git.awesomike.com/pub/waymaker-client", tag = "v0.1.27" }
# (the crate lives in the rust/ subdir; cargo resolves it automatically)
```
Proto stubs are generated from `../proto` at build time via `build.rs` — no
dependency on the waymaker server workspace.
## Surfaces
- `lock` — read/write locks, leader election, TTL leases. The `Lock` handle
keeps a background task that **transparently re-binds** its event stream
across a primary bounce (reusing the `request_id`) and exposes live state:
`lock.fence_token()`, `lock.watch()` (a `watch::Receiver<LockState>`),
`lock.is_lost()`. Re-read the fence before every fenced side effect.
- `stream` — JetStream-lite publish + pull/push consumers.
- `kv` — Put/Get/Create/Update(CAS)/Delete/Keys/History/Watch.
- `cache` — Redis-shape Hash / Set / Queue (collections).
- `probabilistic` — Bloom / HLL / CMS / TopK / t-digest.
- `object` — chunked object Put/Get.
Raw generated stubs are available under `server` / `streams_server` /
`kv_server` / `collections_server` / `sketches_server` / `cache_server`.
## Leader election
```rust
use std::time::Duration;
use waymaker_client::{Client, lock};
let client = Client::connect("http://127.0.0.1:8818").await?;
let lock = client.acquire_lock("leader:reports", lock::Config {
max_wait: Duration::ZERO, // try-acquire
lease_ttl: Duration::from_secs(30),
scope: lock::Scope::Quorum, // Raft-replicated fence
..Default::default()
}).await?;
let _renewal = lock.spawn_renewal(Duration::from_secs(15));
// Re-read the fence before each fenced write; stop if leadership is lost.
let mut state = lock.watch();
loop {
if lock.is_lost() { break; }
do_fenced_write(lock.fence_token()).await?;
state.changed().await.ok();
}
```

19
rust/build.rs Normal file
View File

@ -0,0 +1,19 @@
// Compile the vendored protos (../proto, synced from the waymaker server repo)
// into client stubs. Client-only: no server traits are generated.
fn main() {
let protos = [
"../proto/waymaker_locks.proto",
"../proto/waymaker_streams.proto",
"../proto/kv.proto",
"../proto/collections.proto",
"../proto/sketches.proto",
"../proto/cache.proto",
];
tonic_prost_build::configure()
.build_server(false)
.compile_protos(&protos, &["../proto"])
.expect("failed to compile waymaker protos");
for p in protos {
println!("cargo:rerun-if-changed={p}");
}
}

210
rust/src/cache/hash.rs vendored Normal file
View File

@ -0,0 +1,210 @@
//! Redis-shape Hash. Thin RPC binding over server's `Hash*` RPCs.
//! All conventions (subject patterns, tombstone marker) live
//! server-side in `waymaker_streams::wire_conventions`.
use crate::client::Client;
use crate::error::{Error, Result};
use crate::collections_server::{
CreateHashStoreRequest, DeleteHashStoreRequest, HashDeleteRequest, HashExistsRequest,
HashFieldsRequest, HashGetAllRequest, HashGetRequest, HashLenRequest, HashSetRequest,
};
use std::collections::HashMap;
use tonic::Request;
#[derive(Debug, Clone, Default)]
pub struct HashStoreConfig {
pub name: String,
pub max_bytes: Option<u64>,
pub ephemeral: bool,
}
#[derive(Clone)]
pub struct HashStore {
client: Client,
name: String,
}
impl HashStore {
pub(crate) fn new(client: Client, name: String) -> Self {
Self { client, name }
}
pub fn name(&self) -> &str {
&self.name
}
pub fn hash(&self, hash_key: impl Into<String>) -> Hash {
Hash {
client: self.client.clone(),
bucket: self.name.clone(),
hash_key: hash_key.into(),
}
}
}
#[derive(Clone)]
pub struct Hash {
client: Client,
bucket: String,
hash_key: String,
}
impl Hash {
pub async fn set(
&self,
field: impl AsRef<str>,
value: impl Into<Vec<u8>>,
) -> Result<u64> {
let mut c = self.client.collections_client();
let r = c
.hash_set(Request::new(HashSetRequest {
bucket: self.bucket.clone(),
hash_key: self.hash_key.clone(),
field: field.as_ref().into(),
value: value.into(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.revision)
}
pub async fn get(&self, field: impl AsRef<str>) -> Result<Option<Vec<u8>>> {
let mut c = self.client.collections_client();
let r = c
.hash_get(Request::new(HashGetRequest {
bucket: self.bucket.clone(),
hash_key: self.hash_key.clone(),
field: field.as_ref().into(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.value)
}
pub async fn exists(&self, field: impl AsRef<str>) -> Result<bool> {
let mut c = self.client.collections_client();
let r = c
.hash_exists(Request::new(HashExistsRequest {
bucket: self.bucket.clone(),
hash_key: self.hash_key.clone(),
field: field.as_ref().into(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.exists)
}
pub async fn delete_field(&self, field: impl AsRef<str>) -> Result<()> {
let mut c = self.client.collections_client();
let r = c
.hash_delete(Request::new(HashDeleteRequest {
bucket: self.bucket.clone(),
hash_key: self.hash_key.clone(),
field: field.as_ref().into(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(())
}
pub async fn fields(&self) -> Result<Vec<String>> {
let mut c = self.client.collections_client();
let r = c
.hash_fields(Request::new(HashFieldsRequest {
bucket: self.bucket.clone(),
hash_key: self.hash_key.clone(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.fields)
}
pub async fn len(&self) -> Result<usize> {
let mut c = self.client.collections_client();
let r = c
.hash_len(Request::new(HashLenRequest {
bucket: self.bucket.clone(),
hash_key: self.hash_key.clone(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.count as usize)
}
pub async fn get_all(&self) -> Result<HashMap<String, Vec<u8>>> {
let mut c = self.client.collections_client();
let r = c
.hash_get_all(Request::new(HashGetAllRequest {
bucket: self.bucket.clone(),
hash_key: self.hash_key.clone(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.entries.into_iter().map(|e| (e.field, e.value)).collect())
}
}
impl Client {
pub async fn create_hash_store(&self, config: HashStoreConfig) -> Result<HashStore> {
let mut c = self.collections_client();
let r = c
.create_hash_store(Request::new(CreateHashStoreRequest {
name: config.name.clone(),
max_bytes: config.max_bytes.unwrap_or(0),
ephemeral: config.ephemeral,
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(HashStore::new(self.clone(), config.name))
}
pub async fn get_or_create_hash_store(&self, config: HashStoreConfig) -> Result<HashStore> {
match self.create_hash_store(config.clone()).await {
Ok(s) => Ok(s),
Err(Error::Server { code, .. }) if code == "already_exists" => {
Ok(HashStore::new(self.clone(), config.name))
}
Err(e) => Err(e),
}
}
pub fn hash_store(&self, name: impl Into<String>) -> HashStore {
HashStore::new(self.clone(), name.into())
}
pub async fn delete_hash_store(&self, name: impl Into<String>) -> Result<()> {
let mut c = self.collections_client();
let r = c
.delete_hash_store(Request::new(DeleteHashStoreRequest {
name: name.into(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(())
}
}

21
rust/src/cache/mod.rs vendored Normal file
View File

@ -0,0 +1,21 @@
//! Redis-shape typed structures backed by waymaker server RPCs.
//!
//! Every operation is a single typed RPC: no client-side knowledge
//! of subject patterns, header names, or marker bytes is needed.
//! The conventions live server-side in
//! `waymaker_streams::wire_conventions` — every language client
//! stays in sync because there's only one place where the wire
//! contract lives.
//!
//! Three Redis-flavored types:
//! - [`Hash`] — HSET/HGET/HDEL/HGETALL.
//! - [`Set`] — SADD/SREM/SMEMBERS/SISMEMBER.
//! - [`Queue`] — RPUSH/LPOP/LRANGE/LLEN (queue-style).
pub mod hash;
pub mod queue;
pub mod set;
pub use hash::{Hash, HashStore, HashStoreConfig};
pub use queue::{Queue, QueueConfig};
pub use set::{Set, SetStore, SetStoreConfig};

138
rust/src/cache/queue.rs vendored Normal file
View File

@ -0,0 +1,138 @@
//! Redis-shape Queue. Thin RPC binding.
use crate::client::Client;
use crate::error::{Error, Result};
use crate::collections_server::{
CreateQueueRequest, DeleteQueueRequest, QueueLenRequest, QueuePopRequest, QueuePushRequest,
QueueRangeRequest,
};
use tonic::Request;
#[derive(Debug, Clone, Default)]
pub struct QueueConfig {
pub name: String,
pub max_bytes: Option<u64>,
pub max_messages: Option<u64>,
pub ephemeral: bool,
}
#[derive(Clone)]
pub struct Queue {
client: Client,
name: String,
}
impl Queue {
pub(crate) fn new(client: Client, name: String) -> Self {
Self { client, name }
}
pub fn name(&self) -> &str {
&self.name
}
pub async fn push(&self, value: impl Into<Vec<u8>>) -> Result<u64> {
let mut c = self.client.collections_client();
let r = c
.queue_push(Request::new(QueuePushRequest {
bucket: self.name.clone(),
value: value.into(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.sequence)
}
pub async fn pop(&self) -> Result<Option<Vec<u8>>> {
let mut c = self.client.collections_client();
let r = c
.queue_pop(Request::new(QueuePopRequest {
bucket: self.name.clone(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.value)
}
pub async fn range(&self, from: u64, limit: u64) -> Result<Vec<Vec<u8>>> {
let mut c = self.client.collections_client();
let r = c
.queue_range(Request::new(QueueRangeRequest {
bucket: self.name.clone(),
from_sequence: from,
limit,
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.values)
}
pub async fn len(&self) -> Result<u64> {
let mut c = self.client.collections_client();
let r = c
.queue_len(Request::new(QueueLenRequest {
bucket: self.name.clone(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.count)
}
}
impl Client {
pub async fn create_queue(&self, config: QueueConfig) -> Result<Queue> {
let mut c = self.collections_client();
let r = c
.create_queue(Request::new(CreateQueueRequest {
name: config.name.clone(),
max_bytes: config.max_bytes.unwrap_or(0),
max_messages: config.max_messages.unwrap_or(0),
ephemeral: config.ephemeral,
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(Queue::new(self.clone(), config.name))
}
pub async fn get_or_create_queue(&self, config: QueueConfig) -> Result<Queue> {
match self.create_queue(config.clone()).await {
Ok(q) => Ok(q),
Err(Error::Server { code, .. }) if code == "already_exists" => {
Ok(Queue::new(self.clone(), config.name))
}
Err(e) => Err(e),
}
}
pub fn queue(&self, name: impl Into<String>) -> Queue {
Queue::new(self.clone(), name.into())
}
pub async fn delete_queue(&self, name: impl Into<String>) -> Result<()> {
let mut c = self.collections_client();
let r = c
.delete_queue(Request::new(DeleteQueueRequest {
name: name.into(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(())
}
}

171
rust/src/cache/set.rs vendored Normal file
View File

@ -0,0 +1,171 @@
//! Redis-shape Set. Thin RPC binding.
use crate::client::Client;
use crate::error::{Error, Result};
use crate::collections_server::{
CreateSetStoreRequest, DeleteSetStoreRequest, SetAddRequest, SetIsMemberRequest,
SetLenRequest, SetMembersRequest, SetRemoveRequest,
};
use tonic::Request;
#[derive(Debug, Clone, Default)]
pub struct SetStoreConfig {
pub name: String,
pub max_bytes: Option<u64>,
pub ephemeral: bool,
}
#[derive(Clone)]
pub struct SetStore {
client: Client,
name: String,
}
impl SetStore {
pub(crate) fn new(client: Client, name: String) -> Self {
Self { client, name }
}
pub fn name(&self) -> &str {
&self.name
}
pub fn set(&self, set_key: impl Into<String>) -> Set {
Set {
client: self.client.clone(),
bucket: self.name.clone(),
set_key: set_key.into(),
}
}
}
#[derive(Clone)]
pub struct Set {
client: Client,
bucket: String,
set_key: String,
}
impl Set {
pub async fn add(&self, member: impl AsRef<str>) -> Result<()> {
let mut c = self.client.collections_client();
let r = c
.set_add(Request::new(SetAddRequest {
bucket: self.bucket.clone(),
set_key: self.set_key.clone(),
member: member.as_ref().into(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(())
}
pub async fn remove(&self, member: impl AsRef<str>) -> Result<()> {
let mut c = self.client.collections_client();
let r = c
.set_remove(Request::new(SetRemoveRequest {
bucket: self.bucket.clone(),
set_key: self.set_key.clone(),
member: member.as_ref().into(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(())
}
pub async fn is_member(&self, member: impl AsRef<str>) -> Result<bool> {
let mut c = self.client.collections_client();
let r = c
.set_is_member(Request::new(SetIsMemberRequest {
bucket: self.bucket.clone(),
set_key: self.set_key.clone(),
member: member.as_ref().into(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.is_member)
}
pub async fn members(&self) -> Result<Vec<String>> {
let mut c = self.client.collections_client();
let r = c
.set_members(Request::new(SetMembersRequest {
bucket: self.bucket.clone(),
set_key: self.set_key.clone(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.members)
}
pub async fn len(&self) -> Result<usize> {
let mut c = self.client.collections_client();
let r = c
.set_len(Request::new(SetLenRequest {
bucket: self.bucket.clone(),
set_key: self.set_key.clone(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.count as usize)
}
}
impl Client {
pub async fn create_set_store(&self, config: SetStoreConfig) -> Result<SetStore> {
let mut c = self.collections_client();
let r = c
.create_set_store(Request::new(CreateSetStoreRequest {
name: config.name.clone(),
max_bytes: config.max_bytes.unwrap_or(0),
ephemeral: config.ephemeral,
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(SetStore::new(self.clone(), config.name))
}
pub async fn get_or_create_set_store(&self, config: SetStoreConfig) -> Result<SetStore> {
match self.create_set_store(config.clone()).await {
Ok(s) => Ok(s),
Err(Error::Server { code, .. }) if code == "already_exists" => {
Ok(SetStore::new(self.clone(), config.name))
}
Err(e) => Err(e),
}
}
pub fn set_store(&self, name: impl Into<String>) -> SetStore {
SetStore::new(self.clone(), name.into())
}
pub async fn delete_set_store(&self, name: impl Into<String>) -> Result<()> {
let mut c = self.collections_client();
let r = c
.delete_set_store(Request::new(DeleteSetStoreRequest {
name: name.into(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(())
}
}

243
rust/src/client.rs Normal file
View File

@ -0,0 +1,243 @@
//! Core connection handle.
//!
//! Owns the gRPC channel and exposes top-level entry points for
//! every subsystem (streams, locks, KV, object store). Per-subsystem
//! operations live on the handle returned by those entry points
//! (e.g. `Stream::publish`, `Lock::extend`).
//!
//! `Client` is cheap to clone — the underlying tonic `Channel` is
//! Arc-based, so passing a `Client` to spawned tasks does not
//! duplicate the connection.
//!
//! ## HA / multi-host
//!
//! `Client::connect` accepts a single URL; `Client::connect_multi`
//! accepts a list. With multiple endpoints, tonic's
//! `Channel::balance_list` round-robins requests across them and
//! automatically reroutes when one endpoint goes down. This is
//! the right shape for a clustered waymaker deployment where any
//! node can serve any RPC (with internal proxy hops when the local
//! node is not the primary for a given key).
use crate::error::{Error, Result};
use crate::stream::{Stream, StreamConfig, StreamUpdate};
use crate::streams_server::waymaker_streams_service_client::WaymakerStreamsServiceClient;
use crate::streams_server::{
CreateStreamRequest, DeleteStreamRequest, GetStreamInfoRequest, UpdateStreamRequest,
};
use crate::server::waymaker_service_client::WaymakerServiceClient;
use tonic::transport::{Channel, Endpoint};
/// A connected waymaker client. Holds a tonic `Channel` and exposes
/// both gRPC service clients (locks + streams) lazily.
#[derive(Clone)]
pub struct Client {
pub(crate) channel: Channel,
}
impl Client {
/// Connect to a single waymaker server. Accepts the same
/// `http://host:port` (or `https://...`) URL shape as
/// `tonic::transport::Channel::from_static`.
pub async fn connect(url: impl Into<String>) -> Result<Self> {
let url: String = url.into();
let endpoint = Channel::from_shared(url)
.map_err(|e| Error::Invalid(format!("invalid url: {e}")))?;
let channel = endpoint.connect().await?;
Ok(Self { channel })
}
/// Connect to multiple waymaker servers with load-balanced
/// failover. The returned `Client` round-robins requests across
/// endpoints and automatically reroutes when one is down — the
/// right shape for a clustered deployment.
///
/// Requires at least one URL; errors if the list is empty or any
/// URL fails to parse.
pub fn connect_multi<I, S>(urls: I) -> Result<Self>
where
I: IntoIterator<Item = S>,
S: Into<String>,
{
let endpoints: Vec<Endpoint> = urls
.into_iter()
.map(|u| {
let u = u.into();
Channel::from_shared(u.clone())
.map_err(|e| Error::Invalid(format!("invalid url {u:?}: {e}")))
})
.collect::<Result<_>>()?;
if endpoints.is_empty() {
return Err(Error::Invalid("connect_multi requires at least one URL".into()));
}
// `balance_list` returns a Channel that connects lazily and
// load-balances across all endpoints; it does not need an
// initial successful handshake.
let channel = Channel::balance_list(endpoints.into_iter());
Ok(Self { channel })
}
/// Streams gRPC client over this connection. Constructed
/// per-call so each RPC sees the channel's current routing
/// state (round-robin / failover decisions).
pub fn streams_client(&self) -> WaymakerStreamsServiceClient<Channel> {
WaymakerStreamsServiceClient::new(self.channel.clone())
}
/// Locks gRPC client over this connection.
pub fn locks_client(&self) -> WaymakerServiceClient<Channel> {
WaymakerServiceClient::new(self.channel.clone())
}
/// Sketches gRPC client over this connection. The sketches
/// service is its own surface (`WaymakerSketchesService`);
/// per-call construction matches the streams pattern.
pub(crate) fn sketches_client(
&self,
) -> crate::sketches_server::waymaker_sketches_service_client::WaymakerSketchesServiceClient<
Channel,
> {
crate::sketches_server::waymaker_sketches_service_client::WaymakerSketchesServiceClient::new(
self.channel.clone(),
)
}
/// KV gRPC client over this connection. The KV service
/// (`WaymakerKvService`) is its own surface; per-call
/// construction matches the streams pattern.
pub(crate) fn kv_client(
&self,
) -> crate::kv_server::waymaker_kv_service_client::WaymakerKvServiceClient<Channel> {
crate::kv_server::waymaker_kv_service_client::WaymakerKvServiceClient::new(
self.channel.clone(),
)
}
/// Collections (Hash / Set / Queue) gRPC client.
pub(crate) fn collections_client(
&self,
) -> crate::collections_server::waymaker_collections_service_client::WaymakerCollectionsServiceClient<Channel>
{
crate::collections_server::waymaker_collections_service_client::WaymakerCollectionsServiceClient::new(
self.channel.clone(),
)
}
/// Cache (TTL / eviction policy) gRPC client.
pub fn cache_client(
&self,
) -> crate::cache_server::waymaker_cache_service_client::WaymakerCacheServiceClient<Channel>
{
crate::cache_server::waymaker_cache_service_client::WaymakerCacheServiceClient::new(
self.channel.clone(),
)
}
// ----- Streams subsystem entry points -----
/// Create a new stream. Errors if the stream already exists.
pub async fn create_stream(&self, config: StreamConfig) -> Result<Stream> {
let name = config.name.clone();
let mut c = self.streams_client();
let r = c
.create_stream(tonic::Request::new(CreateStreamRequest {
config: Some(config.into_pb()),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(Stream::new(self.clone(), name))
}
/// Return a handle to an existing stream. Confirms the stream
/// exists via `GetStreamInfo`.
pub async fn get_stream(&self, name: impl Into<String>) -> Result<Stream> {
let name: String = name.into();
let mut c = self.streams_client();
let r = c
.get_stream_info(tonic::Request::new(GetStreamInfoRequest { name: name.clone() }))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(Stream::new(self.clone(), name))
}
/// Create the stream if it doesn't exist; otherwise return a
/// handle to the existing one.
pub async fn get_or_create_stream(&self, config: StreamConfig) -> Result<Stream> {
match self.get_stream(config.name.clone()).await {
Ok(s) => Ok(s),
Err(Error::Server { code, .. }) if code == "no_such_stream" => {
self.create_stream(config).await
}
Err(e) => Err(e),
}
}
/// Apply the mutable subset of a stream's config.
pub async fn update_stream(
&self,
name: impl Into<String>,
update: StreamUpdate,
) -> Result<()> {
let mut c = self.streams_client();
let r = c
.update_stream(tonic::Request::new(UpdateStreamRequest {
name: name.into(),
max_age_ms: update.max_age_ms.map(|d| d.as_millis() as u64),
max_msgs: update.max_msgs,
max_bytes: update.max_bytes,
max_msg_bytes: update.max_msg_bytes,
strict_limits: update.strict_limits,
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(())
}
/// Delete a stream and all its consumers.
pub async fn delete_stream(&self, name: impl Into<String>) -> Result<()> {
let mut c = self.streams_client();
let r = c
.delete_stream(tonic::Request::new(DeleteStreamRequest { name: name.into() }))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(())
}
/// Slice 3 admin: enumerate every (sourcing, source) tail
/// running on the node serving the request. Multi-node clusters
/// need to query each node — this RPC is local-scope by design.
pub async fn get_stream_sources(&self) -> Result<Vec<crate::stream::SourceStatus>> {
use crate::streams_server::GetStreamSourcesRequest;
let mut c = self.streams_client();
let r = c
.get_stream_sources(tonic::Request::new(GetStreamSourcesRequest {}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.entries
.into_iter()
.map(|e| crate::stream::SourceStatus {
sourcing_stream: e.sourcing_stream,
source_stream: e.source_stream,
last_sourced_seq: e.last_sourced_seq,
pulled_total: e.pulled_total,
last_error: e.last_error,
last_error_ts_ms: e.last_error_ts_ms,
})
.collect())
}
}

39
rust/src/error.rs Normal file
View File

@ -0,0 +1,39 @@
//! Typed error surface for the high-level wrapper.
//!
//! Mirrors the shape of `async_nats::Error` — most callers in the
//! awesomike codebase treat errors as `Box<dyn Error>` so a simple
//! enum wrapping the underlying causes is enough.
use thiserror::Error;
#[derive(Debug, Error)]
pub enum Error {
#[error("connect: {0}")]
Connect(#[from] tonic::transport::Error),
/// RPC-layer transport failure (typically network).
#[error("rpc: {0}")]
Rpc(#[from] tonic::Status),
/// Server returned `success=false`. The string is the
/// `result_code` from the response (`no_such_stream`,
/// `no_such_consumer`, `internal`, etc).
#[error("server returned {code}: {message}")]
Server { code: String, message: String },
/// The caller passed a value the wrapper couldn't translate to
/// a wire field (e.g. an unknown retention policy).
#[error("invalid argument: {0}")]
Invalid(String),
}
impl Error {
pub(crate) fn server(code: impl Into<String>, message: impl Into<String>) -> Self {
Self::Server {
code: code.into(),
message: message.into(),
}
}
}
pub type Result<T> = std::result::Result<T, Error>;

371
rust/src/kv/mod.rs Normal file
View File

@ -0,0 +1,371 @@
//! KV subsystem — thin RPC binding over the server's `Kv*` RPCs.
//!
//! All conventions (subject patterns, tombstone marker, TTL
//! header) live **server-side** in
//! `waymaker_streams::wire_conventions`. The client just calls
//! the typed RPCs — every language client stays in sync without
//! having to mirror the wire details.
use crate::client::Client;
use crate::error::{Error, Result};
use crate::kv_server::{
self as pb, KvCreateBucketRequest, KvCreateRequest, KvDeleteBucketRequest, KvDeleteRequest,
KvGetRequest, KvHistoryRequest, KvKeysRequest, KvPutRequest, KvTouchRequest,
KvUpdateRequest, KvWatchRequest,
};
use std::pin::Pin;
use std::task::{Context, Poll};
use std::time::Duration;
use tonic::Request;
/// Bucket creation config.
#[derive(Debug, Clone, Default)]
pub struct Config {
pub name: String,
/// Cap on total bucket bytes. `None` = unbounded.
pub max_bytes: Option<u64>,
/// Cap on per-value bytes. `None` = no cap.
pub max_value_size: Option<u64>,
/// Bucket-level TTL (independent of per-key TTL).
pub max_age: Option<Duration>,
/// Memory-only.
pub ephemeral: bool,
/// Per-key revision cap. `0` (default) = unbounded — history
/// is bounded only by the bucket's stream-level retention
/// (`max_bytes`/`max_age`). When N > 0, after each write the
/// older revisions of *that key* beyond the N most recent are
/// dropped. NATS JetStream's `MaxRevisions` semantic. Useful
/// when one bucket hosts many keys with very different write
/// rates — a fast-churning key won't crowd out a slow one.
pub max_revisions: u64,
}
/// A reference to a bucket. Cheap to clone.
#[derive(Clone)]
pub struct Bucket {
client: Client,
name: String,
}
impl Bucket {
pub(crate) fn new(client: Client, name: String) -> Self {
Self { client, name }
}
pub fn name(&self) -> &str {
&self.name
}
/// Put `value` under `key`. Latest-write-wins. Returns the
/// new revision.
pub async fn put(&self, key: impl AsRef<str>, value: impl Into<Vec<u8>>) -> Result<u64> {
let mut c = self.client.kv_client();
let r = c
.put(Request::new(KvPutRequest {
bucket: self.name.clone(),
key: key.as_ref().to_string(),
value: value.into(),
ttl_ms: 0,
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.revision)
}
/// Put with per-key TTL. Server auto-expires the value after
/// `ttl` elapses (best-effort, on the next sweep tick).
pub async fn put_with_ttl(
&self,
key: impl AsRef<str>,
value: impl Into<Vec<u8>>,
ttl: Duration,
) -> Result<u64> {
let mut c = self.client.kv_client();
let r = c
.put(Request::new(KvPutRequest {
bucket: self.name.clone(),
key: key.as_ref().to_string(),
value: value.into(),
ttl_ms: ttl.as_millis() as u64,
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.revision)
}
/// Atomic create — fails with `Error::Server { code:
/// "wrong_revision", .. }` if the key already has any value.
pub async fn create(&self, key: impl AsRef<str>, value: impl Into<Vec<u8>>) -> Result<u64> {
let mut c = self.client.kv_client();
let r = c
.create(Request::new(KvCreateRequest {
bucket: self.name.clone(),
key: key.as_ref().to_string(),
value: value.into(),
ttl_ms: 0,
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.revision)
}
/// CAS update — succeeds only if current revision matches
/// `expected_revision`.
pub async fn update(
&self,
key: impl AsRef<str>,
value: impl Into<Vec<u8>>,
expected_revision: u64,
) -> Result<u64> {
let mut c = self.client.kv_client();
let r = c
.update(Request::new(KvUpdateRequest {
bucket: self.name.clone(),
key: key.as_ref().to_string(),
value: value.into(),
expected_revision,
ttl_ms: 0,
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.revision)
}
/// Get the latest value. `None` when absent or tombstoned.
pub async fn get(&self, key: impl AsRef<str>) -> Result<Option<Vec<u8>>> {
Ok(self.get_with_revision(key).await?.map(|(v, _)| v))
}
/// Get value + revision (for chaining CAS).
pub async fn get_with_revision(
&self,
key: impl AsRef<str>,
) -> Result<Option<(Vec<u8>, u64)>> {
let mut c = self.client.kv_client();
let r = c
.get(Request::new(KvGetRequest {
bucket: self.name.clone(),
key: key.as_ref().to_string(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.entry.map(|e| (e.value, e.revision)))
}
/// Tombstone `key`.
pub async fn delete(&self, key: impl AsRef<str>) -> Result<()> {
let mut c = self.client.kv_client();
let r = c
.delete(Request::new(KvDeleteRequest {
bucket: self.name.clone(),
key: key.as_ref().to_string(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(())
}
/// Extend TTL on `key` without changing its value.
pub async fn touch(&self, key: impl AsRef<str>, ttl: Duration) -> Result<u64> {
let mut c = self.client.kv_client();
let r = c
.touch(Request::new(KvTouchRequest {
bucket: self.name.clone(),
key: key.as_ref().to_string(),
ttl_ms: ttl.as_millis() as u64,
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.revision)
}
/// List every key in the bucket. Tombstoned entries are
/// excluded by default.
pub async fn keys(&self) -> Result<Vec<String>> {
let mut c = self.client.kv_client();
let r = c
.keys(Request::new(KvKeysRequest {
bucket: self.name.clone(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.entries
.into_iter()
.filter(|e| !e.deleted)
.map(|e| e.key)
.collect())
}
/// Historical values at `key` in publish order.
pub async fn history(&self, key: impl AsRef<str>) -> Result<Vec<HistoryEntry>> {
let mut c = self.client.kv_client();
let r = c
.history(Request::new(KvHistoryRequest {
bucket: self.name.clone(),
key: key.as_ref().to_string(),
from_revision: 0,
limit: 0,
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.entries
.into_iter()
.map(|e| HistoryEntry {
value: e.value,
revision: e.revision,
ts_ms: e.ts_ms,
tombstone: e.deleted,
})
.collect())
}
/// Watch live changes at `key`.
pub async fn watch(&self, key: impl AsRef<str>) -> Result<Watch> {
self.watch_inner(Some(key.as_ref().to_string())).await
}
/// Watch every key in the bucket.
pub async fn watch_all(&self) -> Result<Watch> {
self.watch_inner(None).await
}
async fn watch_inner(&self, key: Option<String>) -> Result<Watch> {
let mut c = self.client.kv_client();
let stream = c
.watch(Request::new(KvWatchRequest {
bucket: self.name.clone(),
key: key.unwrap_or_default(),
}))
.await?
.into_inner();
Ok(Watch { inner: stream })
}
}
#[derive(Debug, Clone)]
pub struct HistoryEntry {
pub value: Vec<u8>,
pub revision: u64,
pub ts_ms: i64,
pub tombstone: bool,
}
#[derive(Debug, Clone)]
pub enum Event {
Put { key: String, value: Vec<u8>, revision: u64 },
Delete { key: String, revision: u64 },
}
pub struct Watch {
inner: tonic::Streaming<pb::KvWatchEvent>,
}
impl tokio_stream::Stream for Watch {
type Item = Result<Event>;
fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
use pb::kv_watch_event::Event as Ev;
loop {
match Pin::new(&mut self.inner).poll_next(cx) {
Poll::Pending => return Poll::Pending,
Poll::Ready(None) => return Poll::Ready(None),
Poll::Ready(Some(Err(s))) => return Poll::Ready(Some(Err(Error::Rpc(s)))),
Poll::Ready(Some(Ok(ev))) => match ev.event {
Some(Ev::Put(p)) => {
return Poll::Ready(Some(Ok(Event::Put {
key: p.key,
value: p.value,
revision: p.revision,
})));
}
Some(Ev::Delete(d)) => {
return Poll::Ready(Some(Ok(Event::Delete {
key: d.key,
revision: d.revision,
})));
}
None => continue,
},
}
}
}
}
impl Client {
/// Create a new KV bucket.
pub async fn create_kv(&self, config: Config) -> Result<Bucket> {
let mut c = self.kv_client();
let r = c
.create_bucket(Request::new(KvCreateBucketRequest {
bucket: config.name.clone(),
max_bytes: config.max_bytes.unwrap_or(0),
max_value_size: config.max_value_size.unwrap_or(0),
max_age_ms: config.max_age.map(|d| d.as_millis() as u64).unwrap_or(0),
ephemeral: config.ephemeral,
max_revisions: config.max_revisions,
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(Bucket::new(self.clone(), config.name))
}
/// Idempotent create-or-get.
pub async fn get_or_create_kv(&self, config: Config) -> Result<Bucket> {
match self.create_kv(config.clone()).await {
Ok(b) => Ok(b),
Err(Error::Server { code, .. }) if code == "already_exists" => {
Ok(Bucket::new(self.clone(), config.name))
}
Err(e) => Err(e),
}
}
/// Return a handle without verifying existence.
pub fn kv(&self, name: impl Into<String>) -> Bucket {
Bucket::new(self.clone(), name.into())
}
/// Delete the bucket.
pub async fn delete_kv(&self, name: impl Into<String>) -> Result<()> {
let mut c = self.kv_client();
let r = c
.delete_bucket(Request::new(KvDeleteBucketRequest {
bucket: name.into(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(())
}
}

72
rust/src/lib.rs Normal file
View File

@ -0,0 +1,72 @@
//! Official Rust client for waymaker.
//!
//! The proto stubs are generated at build time from the vendored
//! `.proto` files in this repo's `proto/` directory (the waymaker
//! server repo is the source of truth — see `scripts/sync-protos.sh`).
//! Each subsystem's stubs live under a `<sub>_server` module so a
//! consumer can reach the raw wire types when needed; the ergonomic
//! wrappers (`lock`, `stream`, `kv`, …) are layered on top.
//!
//! Module names are kept as `server` / `streams_server` / … for
//! source compatibility with the pre-extraction crate, where these
//! were re-exports of the per-subsystem `proto` modules.
/// Lock proto stubs (`WaymakerService` — the external client surface).
/// Generated from `proto/waymaker_locks.proto` (package `waymaker`).
/// The server-internal `ProxyService` (node-to-node proxy + lease
/// replication) is deliberately NOT vendored here — it is a server
/// concern, not part of the client.
pub mod server {
tonic::include_proto!("waymaker");
}
/// Streams proto stubs (`WaymakerStreamsService` + internal
/// replication RPCs). Generated from `proto/waymaker_streams.proto`.
pub mod streams_server {
tonic::include_proto!("waymaker.streams");
}
/// Sketches proto stubs (`WaymakerSketchesService`). Generated from
/// `proto/sketches.proto`.
pub mod sketches_server {
tonic::include_proto!("waymaker.sketches");
}
/// KV proto stubs (`WaymakerKvService`). Generated from
/// `proto/kv.proto`.
pub mod kv_server {
tonic::include_proto!("waymaker.kv");
}
/// Collections proto stubs (`WaymakerCollectionsService` —
/// Hash / Set / Queue). Generated from `proto/collections.proto`.
pub mod collections_server {
tonic::include_proto!("waymaker.collections");
}
/// Cache proto stubs (`WaymakerCacheService`). Generated from
/// `proto/cache.proto`.
pub mod cache_server {
tonic::include_proto!("waymaker.cache");
}
// High-level wrapper. The proto-generated types in `server` /
// `streams_server` stay available for callers that need the raw
// wire surface; the per-subsystem modules below are the
// ergonomic ones.
pub mod cache;
pub mod client;
pub mod error;
pub mod kv;
pub mod lock;
pub mod object;
pub mod probabilistic;
pub mod stream;
pub use client::Client;
pub use error::{Error, Result};
// Re-exported so callers can call `.next()` on `consumer.messages()`
// without a separate `use tokio_stream::StreamExt`.
pub use tokio_stream::StreamExt;

836
rust/src/lock/mod.rs Normal file
View File

@ -0,0 +1,836 @@
//! Locks subsystem — wraps the rwlock RPCs (Lock / ReadLock /
//! UnLock / LeaseStatus / ExtendLease / MultiLock).
//!
//! Entry points live on `Client`:
//! - `client.acquire_lock("key", lock::Config { ... })` — write lock
//! - `client.acquire_read_lock("key", lock::Config { ... })` — read lock
//! - `client.lease_status(&lock_id)`
//! - `client.multi_lock(...)`
//!
//! The returned [`Lock`] keeps a background task that holds the
//! server event stream open and, if that stream drops (e.g. the
//! key's primary bounces), transparently re-binds it and re-confirms
//! ownership — reusing the original `request_id` so a still-held
//! lease is recovered rather than re-contended. The lock's live
//! state (fence token, lease expiry, and a `lost` flag) is published
//! through [`Lock::watch`]; re-read [`Lock::fence_token`] before every
//! fenced side effect, because a lost-then-re-won lock carries a new,
//! higher token.
//!
//! ## Leader-election pattern
//!
//! Match the operator workflow used in graylin etc.:
//!
//! ```ignore
//! let lock = client.acquire_lock("leader:foo", lock::Config {
//! max_wait: Duration::ZERO, // try-and-fail
//! lease_ttl: Duration::from_secs(60),
//! scope: lock::Scope::Local,
//! ..Default::default()
//! }).await?;
//! let _renewal = lock.spawn_renewal(Duration::from_secs(30));
//! // ... do work; drop _renewal + drop lock when done.
//! ```
use crate::client::Client;
use crate::error::{Error, Result};
use crate::server::{
waymaker_service_client::WaymakerServiceClient, ExtendLeaseRequest, LeaseStatusRequest,
LockEvent, LockEventType, LockRequest, MultiLockKey, MultiLockRequest, UnLockRequest,
};
use std::sync::Arc;
use std::time::Duration;
use tokio::sync::{watch, Notify};
use tonic::transport::Channel;
use tonic::Request;
use uuid::Uuid;
/// Fence-scope tier — controls **one thing only**: how durable the
/// per-key fence-token counter (the monotonic `u64`) is across
/// failures. Mirrors the proto `FenceScope`.
///
/// What the scope does **not** control:
/// - **Whether a held lock survives a node loss / rollout.** That is
/// `cluster.replication-factor` plus secondary adoption, which
/// already applies to every non-`Ephemeral` lock regardless of
/// scope. A stronger scope does not make a lock survive; a
/// replicated lease does.
/// - **Client-side transparency.** [`Lock`] transparently re-binds
/// its event stream after a disconnect, but that is a client-lib
/// behaviour — no scope value changes it.
/// - **Mutual exclusion.** Holding the lock is not, by itself, a
/// guarantee that no one else acts. You MUST validate
/// [`Lock::fence_token`] at your side effect (the DB write / object
/// PUT) and reject anything carrying a fence below the last you
/// durably committed. Even [`Scope::Quorum`] does not let you skip
/// that check — an all-at-once cluster restart can still drop an
/// in-memory token.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Scope {
/// Counter in RAM on the owning node; resets on that node's
/// restart or a hash-ring rebalance. Fastest (no I/O). Right for
/// advisory locks / rate limiting where a fence reset across
/// failure is tolerable.
Ephemeral,
/// Counter persisted to disk on the owning node. Survives a
/// process restart on the same node; still resets if the ring
/// rebalances the key to a different node. One fsync per acquire.
Local,
/// Raft-replicated per-key counter — cluster-wide monotonic,
/// survives any single-node failure (the surviving quorum keeps
/// the count). One Raft commit per acquire. Requires the cluster
/// backend to be wired; a single-node / test build returns
/// `BadInput` for this scope. Pick this when an external resource
/// fences on the token and two holders must never see fences that
/// fail to prove an ordering.
///
/// Named `Quorum` (not `Global`): the guarantee is "a Raft quorum
/// agrees on the count", which carries its own limit in the word
/// and makes no geographic claim. The wire value is unchanged, so
/// old and new binaries interoperate mid-rollout.
Quorum,
}
impl Default for Scope {
fn default() -> Self {
Self::Ephemeral
}
}
impl Scope {
fn to_pb(self) -> i32 {
use crate::server::FenceScope as F;
let s = match self {
Self::Ephemeral => F::ScopeEphemeral,
Self::Local => F::ScopeLocal,
Self::Quorum => F::ScopeQuorum,
};
s as i32
}
}
/// Lock acquisition config. `Default` is `max_wait=infinite,
/// lease_ttl=30s, priority=0, scope=Ephemeral`. Callers typically
/// override `max_wait` (to `ZERO` for try-acquire) and `lease_ttl`
/// (to whatever the renewal cadence supports).
#[derive(Debug, Clone)]
pub struct Config {
/// How long the server will block waiting for the lock to
/// become available. `Duration::ZERO` = try-acquire (fail
/// immediately if contended). `Duration::MAX` = block
/// indefinitely.
pub max_wait: Duration,
/// How long the lease lives once acquired (before
/// auto-expiration). Renewal extends this.
pub lease_ttl: Duration,
/// Priority class — higher priorities jump the wait queue.
/// Default 0.
pub priority: u32,
/// Fence-token persistence tier.
pub scope: Scope,
/// Free-form requester metadata for server-side audit. Empty
/// strings ok.
pub requester_info: String,
/// Application name (sent verbatim — useful for operator
/// dashboards). Default `"waymaker-client"`.
pub requester_application: String,
/// Idempotency key for retries of the same logical acquire.
/// Auto-filled with a UUID if left empty.
pub request_id: String,
}
impl Default for Config {
fn default() -> Self {
Self {
max_wait: Duration::from_secs(60 * 60),
lease_ttl: Duration::from_secs(30),
priority: 0,
scope: Scope::Ephemeral,
requester_info: String::new(),
requester_application: "waymaker-client".into(),
request_id: String::new(),
}
}
}
impl Config {
fn into_request(self, key: String) -> LockRequest {
let request_id = if self.request_id.is_empty() {
Uuid::now_v7().to_string()
} else {
self.request_id
};
let clamp_ms = |d: Duration| -> u32 {
if d == Duration::MAX {
u32::MAX
} else {
d.as_millis().min(u32::MAX as u128) as u32
}
};
LockRequest {
key,
max_wait_period: clamp_ms(self.max_wait),
max_lease_period: clamp_ms(self.lease_ttl),
priority: self.priority,
requester_info: self.requester_info,
requester_application: self.requester_application,
request_id,
fence_scope: self.scope.to_pb(),
}
}
}
/// Lease details returned with `Acquired` / `Heartbeat` / status
/// queries.
#[derive(Debug, Clone)]
pub struct Lease {
pub id: String,
pub key: String,
pub acquired_at_ms: i64,
pub lease_expires_at_ms: i64,
pub fence_token: u64,
pub priority: u32,
}
/// A live snapshot of a held lock's state, delivered through
/// [`Lock::watch`]. `fence_token` / `id` change only if the lock was
/// lost and transparently re-won after a primary failure; `lost`
/// flips to `true` once the client gives up re-establishing it.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct LockState {
/// Current lease id. Stable across a transparent re-bind; changes
/// only if the lock was lost and freshly re-acquired.
pub id: String,
/// Current fence token. Re-read before every fenced side effect —
/// a lost-then-re-won lock carries a higher token, and acting on a
/// stale copy defeats fencing.
pub fence_token: u64,
/// Lease expiry (epoch ms), refreshed from heartbeats / re-acquire.
pub lease_expires_at_ms: i64,
/// `true` once the client could no longer prove it holds the lock
/// (the lease was taken by someone else, or expired and could not
/// be re-won). A `lost` holder MUST stop acting as the holder.
pub lost: bool,
}
/// An acquired lock. Dropping the handle **does not** auto-release
/// the lock — call [`Lock::unlock`] explicitly (or let the lease
/// expire). This matches the underlying RPC semantics; auto-release
/// on drop would silently swallow errors and surprise callers
/// who expect the server-side state to outlive the local handle.
///
/// A background task keeps the server event stream open. If that
/// stream drops — typically because the key's primary bounced — the
/// task transparently re-binds it, reusing the original `request_id`
/// so a still-held lease is recovered rather than re-contended. The
/// lease itself is kept alive by the server's TTL plus
/// [`Lock::spawn_renewal`], independent of the stream: dropping the
/// stream loses event visibility, not the lock. Live state is exposed
/// through the accessors below and [`Lock::watch`].
pub struct Lock {
client: Client,
pub key: String,
/// Live lock state. `borrow()` yields the current snapshot.
state: watch::Receiver<LockState>,
/// Signals the background hold task to stop. Set by `unlock` and
/// `Drop` so the task never re-acquires a deliberately-released
/// lock.
stop: Arc<Notify>,
_hold: tokio::task::JoinHandle<()>,
}
impl Lock {
/// Current lease id (live).
pub fn id(&self) -> String {
self.state.borrow().id.clone()
}
/// Current fence token (live). Re-read this before every fenced
/// side effect — see [`LockState::fence_token`].
pub fn fence_token(&self) -> u64 {
self.state.borrow().fence_token
}
/// Current lease expiry, epoch ms (live).
pub fn lease_expires_at_ms(&self) -> i64 {
self.state.borrow().lease_expires_at_ms
}
/// `true` once the client has lost the lock and could not re-win
/// it. A lost holder must stop acting as the holder.
pub fn is_lost(&self) -> bool {
self.state.borrow().lost
}
/// Subscribe to live state changes. Await `.changed()`, then
/// re-read [`Lock::fence_token`] / [`Lock::is_lost`] before your
/// next side effect. Multiple receivers are fine.
pub fn watch(&self) -> watch::Receiver<LockState> {
self.state.clone()
}
/// Extend the lease by `additional`.
pub async fn extend(&self, additional: Duration) -> Result<Lease> {
let mut c: WaymakerServiceClient<Channel> = self.client.locks_client();
let r = c
.extend_lease(Request::new(ExtendLeaseRequest {
key: self.key.clone(),
id: self.id(),
lease_timeout: additional.as_millis().min(u32::MAX as u128) as u32,
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
let lease = r.lease.ok_or_else(|| Error::server("internal", "missing lease in response"))?;
Ok(Lease {
id: lease.id,
key: lease.key,
acquired_at_ms: lease.created_at,
lease_expires_at_ms: lease.lease_expires_at,
fence_token: lease.fence_token,
priority: lease.priority,
})
}
/// Release the lock. After this returns, the server-side state
/// is gone and subsequent operations on this `Lock` will fail.
pub async fn unlock(self) -> Result<()> {
// Stop the hold task FIRST so it cannot re-acquire the lock we
// are about to release — a re-acquire racing the UnLock would
// resurrect a lock the caller believes is gone.
self.stop.notify_one();
let id = self.id();
let mut c = self.client.locks_client();
let r = c
.un_lock(Request::new(UnLockRequest {
key: self.key.clone(),
id,
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(())
}
/// Spawn a background task that periodically extends the
/// lease. Returns a `RenewalHandle` — drop it (or call
/// `.stop()`) to halt renewal. Matches the leader-election
/// pattern: the renewal runs independently of the work loop so
/// the work can legitimately outlive any single lease window.
pub fn spawn_renewal(&self, every: Duration) -> RenewalHandle {
let client = self.client.clone();
let key = self.key.clone();
// Clone the live-state receiver so each tick renews the *current*
// lease id — a transparent re-acquire after a primary failure can
// mint a new id, and renewing the stale one would silently let the
// real lease expire.
let state = self.state.clone();
let ttl_ms = (every.as_millis() * 2).min(u32::MAX as u128) as u32;
let stop = Arc::new(Notify::new());
let stop_task = stop.clone();
let handle = tokio::spawn(async move {
let mut ticker = tokio::time::interval(every);
// Skip the immediate first tick — interval fires
// straight away by default and the caller has the
// freshly-acquired lease.
ticker.tick().await;
loop {
tokio::select! {
// `biased`: honor stop FIRST, even mid-interval, so a
// completed (or dropped) holder can't leave a renewal
// task quietly extending the lease — the bug that wedged
// a held lock until the holding session ended. The old
// code only checked the stop flag AFTER the next tick,
// so stop lagged a full interval and a stray extend could
// still fire after stop was requested.
biased;
_ = stop_task.notified() => break,
_ = ticker.tick() => {
// Read the live id; skip if the lock is already
// lost (extending a dead lease just errors).
let (id, lost) = {
let s = state.borrow();
(s.id.clone(), s.lost)
};
if lost {
continue;
}
let mut c = client.locks_client();
// Bound the RPC so a hung/wedged server can't freeze
// the renewal task (and therefore `stop().await`)
// forever — without this the holder could never
// release and every other waiter would skip until a
// process restart.
let _ = tokio::time::timeout(
every,
c.extend_lease(Request::new(ExtendLeaseRequest {
key: key.clone(),
id,
lease_timeout: ttl_ms,
})),
)
.await;
}
}
}
});
RenewalHandle { stop, handle: Some(handle) }
}
}
impl Drop for Lock {
fn drop(&mut self) {
// Stop the background hold task so a dropped (but not explicitly
// unlocked) handle does not keep re-acquiring in the background.
// `notify_one` leaves a permit if the task is mid-RPC; `abort`
// is the backstop.
self.stop.notify_one();
self._hold.abort();
}
}
// ---- Background hold / re-bind / loss-detection loop -----------------
//
// The lease lives on the server's TTL + renewal, NOT on this event
// stream (granting a lock sets its `sender_stream` to None). So a
// dropped stream means "lost event visibility", not "lost lock". This
// task re-binds the stream when it can, and only declares the lock
// `lost` when it can no longer prove ownership via lease_status.
/// Outcome of draining the current event stream.
enum Drained {
/// `unlock` / drop asked us to stop.
Stopped,
/// The stream ended or the server reported the lease gone; try to
/// re-establish ownership.
Disconnected,
}
/// Outcome of one idempotent re-acquire attempt.
enum Rebound {
/// Got a live stream back (transparent re-bind, or a clean re-win
/// after a real loss). Carries the fresh `(stream, id, fence, exp)`.
Bound(tonic::Streaming<LockEvent>, String, u64, i64),
/// The acquire did not grant (key held — possibly by our own
/// still-adopted lease on a new primary, possibly by someone else)
/// or the RPC failed. Disambiguate via lease_status.
NotBound,
}
/// Whether we still demonstrably own the lease.
enum Ownership {
Held(i64),
Lost,
Unknown,
}
const HOLD_BASE_BACKOFF: Duration = Duration::from_millis(200);
const HOLD_MAX_BACKOFF: Duration = Duration::from_secs(10);
const HOLD_RPC_TIMEOUT: Duration = Duration::from_secs(10);
/// Publish `next` if it differs from the local mirror `cur`, notifying
/// watchers only on a real change.
fn publish(tx: &watch::Sender<LockState>, cur: &mut LockState, next: LockState) {
if *cur != next {
*cur = next.clone();
let _ = tx.send_replace(next);
}
}
/// Background task body: hold the event stream, re-bind it on drop, and
/// surface fence-token changes / loss through `state_tx`.
async fn hold_loop(
client: Client,
key: String,
read: bool,
reacquire_req: LockRequest,
mut stream: tonic::Streaming<LockEvent>,
state_tx: watch::Sender<LockState>,
init: LockState,
stop: Arc<Notify>,
) {
// Local mirror of the published state — lets us read the current id
// and publish only on change without depending on `Sender::borrow`.
let mut cur = init;
loop {
match drain_stream(&mut stream, &state_tx, &mut cur, &stop).await {
Drained::Stopped => return,
Drained::Disconnected => {}
}
// Re-establish: re-bind the stream; if we cannot, decide whether
// we still own the lease or have truly lost it.
let mut backoff = HOLD_BASE_BACKOFF;
loop {
// Race every step against `stop` so unlock/drop halts us
// promptly — and, crucially, before any re-acquire could
// resurrect a released lock.
let rebound = match race_stop(reacquire(&client, read, &reacquire_req), &stop).await {
Some(r) => r,
None => return,
};
match rebound {
Rebound::Bound(s, id, fence, exp) => {
publish(
&state_tx,
&mut cur,
LockState { id, fence_token: fence, lease_expires_at_ms: exp, lost: false },
);
stream = s;
break; // resume draining the fresh stream
}
Rebound::NotBound => {
let cur_id = cur.id.clone();
let owned =
match race_stop(confirm_ownership(&client, &key, &cur_id), &stop).await {
Some(o) => o,
None => return,
};
match owned {
Ownership::Held(exp) => {
// Still ours (renewal keeps the lease alive);
// we just could not get an event stream — keep
// monitoring with backoff.
let next = LockState { lease_expires_at_ms: exp, ..cur.clone() };
publish(&state_tx, &mut cur, next);
}
Ownership::Unknown => { /* transient — back off, retry */ }
Ownership::Lost => {
let next = LockState { lost: true, ..cur.clone() };
publish(&state_tx, &mut cur, next);
return;
}
}
if race_stop(tokio::time::sleep(backoff), &stop).await.is_none() {
return;
}
backoff = (backoff * 2).min(HOLD_MAX_BACKOFF);
}
}
}
}
}
/// Drain events from the current stream until it ends, the lease is
/// reported gone, or `stop` fires. Heartbeat / Acquired updates flow
/// into `state_tx`.
async fn drain_stream(
stream: &mut tonic::Streaming<LockEvent>,
state_tx: &watch::Sender<LockState>,
cur: &mut LockState,
stop: &Notify,
) -> Drained {
loop {
tokio::select! {
biased;
_ = stop.notified() => return Drained::Stopped,
msg = stream.message() => match msg {
Ok(Some(ev)) => {
let et = ev.event_type;
if et == LockEventType::Heartbeat as i32 {
let next = LockState { lease_expires_at_ms: ev.lease_expires_at, ..cur.clone() };
publish(state_tx, cur, next);
} else if et == LockEventType::Acquired as i32 {
// Re-emitted after an idempotent re-bind.
let next = LockState {
id: ev.id,
fence_token: ev.fence_token,
lease_expires_at_ms: ev.lease_expires_at,
lost: false,
};
publish(state_tx, cur, next);
} else if et == LockEventType::Expired as i32
|| et == LockEventType::Failed as i32
{
// Server says the lease ended — try to recover.
return Drained::Disconnected;
}
// Waiting / Unknown: ignore, keep reading.
}
Ok(None) | Err(_) => return Drained::Disconnected,
},
}
}
}
/// One idempotent re-acquire attempt (reusing the original
/// `request_id`, `max_wait = 0`). On the original primary with the
/// lease still held this re-binds the stream and re-emits the held
/// entry; if the lease was lost and the key is free it re-wins with a
/// fresh, higher token.
async fn reacquire(client: &Client, read: bool, req: &LockRequest) -> Rebound {
let req = req.clone();
let mut c = client.locks_client();
let attempt = async move {
let mut stream = if read {
c.read_lock(Request::new(req)).await?.into_inner()
} else {
c.lock(Request::new(req)).await?.into_inner()
};
loop {
match stream.message().await? {
Some(ev) if ev.event_type == LockEventType::Acquired as i32 => {
return Ok::<
Option<(tonic::Streaming<LockEvent>, String, u64, i64)>,
tonic::Status,
>(Some((
stream,
ev.id,
ev.fence_token,
ev.lease_expires_at,
)));
}
Some(ev)
if ev.event_type == LockEventType::Failed as i32
|| ev.event_type == LockEventType::Expired as i32 =>
{
return Ok(None); // not granted (contended)
}
Some(_) => continue, // Waiting / Heartbeat
None => return Ok(None), // closed before grant
}
}
};
match tokio::time::timeout(HOLD_RPC_TIMEOUT, attempt).await {
Ok(Ok(Some((s, id, fence, exp)))) => Rebound::Bound(s, id, fence, exp),
Ok(Ok(None)) => Rebound::NotBound,
Ok(Err(_)) | Err(_) => Rebound::NotBound,
}
}
/// Query whether `id` still holds `key`. `success = false` (or a
/// missing lease) means we lost it; a transport / timeout error is
/// inconclusive (`Unknown`) so we never falsely declare loss.
async fn confirm_ownership(client: &Client, key: &str, id: &str) -> Ownership {
let mut c = client.locks_client();
let call = c.lease_status(Request::new(LeaseStatusRequest {
key: key.to_string(),
id: id.to_string(),
}));
match tokio::time::timeout(HOLD_RPC_TIMEOUT, call).await {
Ok(Ok(resp)) => {
let r = resp.into_inner();
match (r.success, r.lease) {
(true, Some(lease)) => Ownership::Held(lease.lease_expires_at),
_ => Ownership::Lost,
}
}
Ok(Err(_)) | Err(_) => Ownership::Unknown,
}
}
/// Run `fut`, returning `None` if `stop` fires first.
async fn race_stop<F: std::future::Future>(fut: F, stop: &Notify) -> Option<F::Output> {
tokio::select! {
biased;
_ = stop.notified() => None,
v = fut => Some(v),
}
}
/// Handle that keeps a background renewal task alive. Drop or
/// `.stop()` to halt renewal cleanly.
pub struct RenewalHandle {
stop: Arc<Notify>,
handle: Option<tokio::task::JoinHandle<()>>,
}
impl RenewalHandle {
/// Halt the renewal task. After this returns, no further
/// `ExtendLease` RPCs will be sent. The task selects on this
/// `Notify` so it stops promptly (within one bounded `extend_lease`
/// at worst), rather than lagging a full renewal interval.
pub async fn stop(mut self) {
self.stop.notify_one();
if let Some(h) = self.handle.take() {
let _ = h.await;
}
}
}
impl Drop for RenewalHandle {
fn drop(&mut self) {
self.stop.notify_one();
if let Some(h) = self.handle.take() {
h.abort();
}
}
}
impl Client {
/// Acquire an exclusive (write) lock. Blocks for at most
/// `config.max_wait`; if the lock isn't granted in time the server emits a
/// `LockEventType::Expired` (the contended wait-TTL elapsed) which this client
/// surfaces as `Error::Server { code: "expired", .. }`. (There is no
/// `"timeout"` code — callers that loop-until-granted must match `"expired"`.)
pub async fn acquire_lock(
&self,
key: impl Into<String>,
config: Config,
) -> Result<Lock> {
self.acquire_lock_inner(key.into(), config, /* read = */ false).await
}
/// Acquire a shared (read) lock.
pub async fn acquire_read_lock(
&self,
key: impl Into<String>,
config: Config,
) -> Result<Lock> {
self.acquire_lock_inner(key.into(), config, /* read = */ true).await
}
async fn acquire_lock_inner(&self, key: String, config: Config, read: bool) -> Result<Lock> {
let req = config.into_request(key.clone());
// Template for transparent re-acquire after a stream drop: same
// request_id (so a still-held lease is recovered idempotently
// rather than re-contended) but never block.
let mut reacquire_req = req.clone();
reacquire_req.max_wait_period = 0;
let mut c = self.locks_client();
let mut stream = if read {
c.read_lock(Request::new(req)).await?.into_inner()
} else {
c.lock(Request::new(req)).await?.into_inner()
};
// Consume events until we see Acquired / Failed / Expired. The
// stream stays open after Acquired — the hold task keeps it bound
// (and re-binds it across primary bounces).
loop {
let event = stream.message().await?;
let event = match event {
Some(e) => e,
None => {
return Err(Error::server(
"stream_closed",
"lock stream closed before acquire",
));
}
};
match event.event_type {
t if t == LockEventType::Acquired as i32 => {
let init = LockState {
id: event.id,
fence_token: event.fence_token,
lease_expires_at_ms: event.lease_expires_at,
lost: false,
};
let (state_tx, state_rx) = watch::channel(init.clone());
let stop = Arc::new(Notify::new());
// Hand the open stream to the hold task, which parks
// it, re-binds on drop, and publishes fence/loss via
// `state_tx`.
let hold = tokio::spawn(hold_loop(
self.clone(),
key.clone(),
read,
reacquire_req,
stream,
state_tx,
init,
stop.clone(),
));
return Ok(Lock {
client: self.clone(),
key,
state: state_rx,
stop,
_hold: hold,
});
}
t if t == LockEventType::Failed as i32 => {
return Err(Error::server("failed", event.message));
}
t if t == LockEventType::Expired as i32 => {
return Err(Error::server("expired", event.message));
}
_ => continue, // Waiting / Heartbeat / Unknown — keep reading.
}
}
}
/// Query the current state of a lock by id.
pub async fn lease_status(
&self,
key: impl Into<String>,
id: impl Into<String>,
) -> Result<Lease> {
let mut c = self.locks_client();
let r = c
.lease_status(Request::new(LeaseStatusRequest {
key: key.into(),
id: id.into(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
let lease = r.lease.ok_or_else(|| Error::server("internal", "missing lease in response"))?;
Ok(Lease {
id: lease.id,
key: lease.key,
acquired_at_ms: lease.created_at,
lease_expires_at_ms: lease.lease_expires_at,
fence_token: lease.fence_token,
priority: lease.priority,
})
}
/// Acquire N locks atomically. Server sorts keys to guarantee
/// deadlock-free ordering. On any failure, every partial lock
/// is released before the call returns.
///
/// Returns the leases in the server's acquisition order
/// (lexicographic by key).
pub async fn multi_lock(
&self,
keys: impl IntoIterator<Item = (String, /* write_lock */ bool)>,
config: Config,
) -> Result<Vec<Lease>> {
let request_id = if config.request_id.is_empty() {
Uuid::now_v7().to_string()
} else {
config.request_id.clone()
};
let keys: Vec<MultiLockKey> = keys
.into_iter()
.map(|(key, write_lock)| MultiLockKey { key, write_lock })
.collect();
let mut c = self.locks_client();
let r = c
.multi_lock(Request::new(MultiLockRequest {
keys,
max_wait_period: config.max_wait.as_millis().min(u32::MAX as u128) as u32,
max_lease_period: config.lease_ttl.as_millis().min(u32::MAX as u128) as u32,
priority: config.priority,
requester_info: config.requester_info,
requester_application: config.requester_application,
request_id,
fence_scope: config.scope.to_pb(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.leases
.into_iter()
.map(|lease| Lease {
id: lease.id,
key: lease.key,
acquired_at_ms: lease.created_at,
lease_expires_at_ms: lease.lease_expires_at,
fence_token: lease.fence_token,
priority: lease.priority,
})
.collect())
}
}

368
rust/src/object/mod.rs Normal file
View File

@ -0,0 +1,368 @@
//! Object store subsystem — wraps the PutObject / GetObject /
//! DeleteObject / GetObjectInfo / ListObjects / GetObjectRange /
//! ListObjectRevisions RPCs.
//!
//! An object-store bucket maps to a stream with subject filter
//! `objm.>` + `objc.>` (metadata vs chunks). The wrapper hides
//! that wire convention behind a per-bucket [`Store`] handle.
//!
//! Entry points on `Client`:
//! - `client.create_object_store(object::Config { ... })`
//! - `client.get_or_create_object_store(cfg)` — idempotent
//! - `client.object_store("name")` — handle without creation
//!
//! v1 covers the unary RPCs. The streaming PutObject /
//! GetObject variants for very large objects can drop in
//! alongside without breaking the surface.
use crate::client::Client;
use crate::error::{Error, Result};
use crate::streams_server::{
self as pb, CreateStreamRequest, DeleteObjectRequest, GetObjectInfoRequest,
GetObjectRangeRequest, GetObjectRequest, ListObjectRevisionsRequest, ListObjectsRequest,
MessageHeader, PutObjectRequest,
};
use crate::stream::{RetentionPolicy, StreamConfig};
use tonic::Request;
const META_SUBJECT_FILTER: &str = "objm.>";
const CHUNK_SUBJECT_FILTER: &str = "objc.>";
/// Server-side default chunk size — 1 MiB. Surfaced here so
/// callers can pre-allocate buffers; passing 0 to `Store::put`
/// uses this implicitly.
pub const DEFAULT_CHUNK_SIZE: u64 = 1024 * 1024;
/// Object-store bucket creation config.
#[derive(Debug, Clone, Default)]
pub struct Config {
pub name: String,
/// Cap on total bucket bytes. `None` = unbounded.
pub max_bytes: Option<u64>,
/// Memory-only bucket (underlying stream is ephemeral).
pub ephemeral: bool,
}
impl Config {
fn into_stream_config(self) -> StreamConfig {
StreamConfig {
name: self.name,
subjects: vec![META_SUBJECT_FILTER.into(), CHUNK_SUBJECT_FILTER.into()],
retention: RetentionPolicy::Limits,
max_bytes: self.max_bytes,
ephemeral: self.ephemeral,
max_msgs_per_subject: 0,
..Default::default()
}
}
}
/// Object metadata. Mirrors the proto `ObjectInfo`.
#[derive(Debug, Clone)]
pub struct ObjectInfo {
pub name: String,
pub total_bytes: u64,
pub chunk_count: u64,
pub chunk_size: u64,
pub sha256: String,
pub ts_ms: i64,
pub headers: Vec<(String, String)>,
pub revision: u64,
pub deduped: bool,
}
impl From<pb::ObjectInfo> for ObjectInfo {
fn from(i: pb::ObjectInfo) -> Self {
Self {
name: i.name,
total_bytes: i.total_bytes,
chunk_count: i.chunk_count,
chunk_size: i.chunk_size,
sha256: i.sha256,
ts_ms: i.ts_ms,
headers: i
.headers
.into_iter()
.map(|MessageHeader { key, value }| (key, value))
.collect(),
revision: i.metadata_seq,
deduped: i.deduped,
}
}
}
/// One entry returned by [`Store::list`].
#[derive(Debug, Clone)]
pub struct ObjectEntry {
pub name: String,
pub total_bytes: u64,
pub deleted: bool,
}
/// Options for [`Store::put_with`].
#[derive(Debug, Clone, Default)]
pub struct PutOptions {
/// Bytes per chunk. 0 = server default (`DEFAULT_CHUNK_SIZE`).
pub chunk_size: u64,
/// Optional headers to attach to the object's metadata.
pub headers: Vec<(String, String)>,
/// Optional pre-computed SHA-256 hex. The server verifies
/// against the running hash. Empty = server computes its own.
pub sha256: String,
/// Cross-object dedupe: chunks are content-addressed and
/// shared across objects with identical content.
pub dedupe: bool,
}
/// A reference to an object-store bucket. Cheap to clone.
#[derive(Clone)]
pub struct Store {
client: Client,
name: String,
}
impl Store {
pub(crate) fn new(client: Client, name: String) -> Self {
Self { client, name }
}
/// The bucket name (also the underlying stream name).
pub fn name(&self) -> &str {
&self.name
}
/// Put a small-to-medium object. The whole payload is sent in
/// one RPC; the server chunks server-side. For very large
/// objects use the streaming Put variant (not yet wired into
/// the wrapper — drop down to `streams_server::*`).
pub async fn put(
&self,
name: impl Into<String>,
payload: impl Into<Vec<u8>>,
) -> Result<ObjectInfo> {
self.put_with(name, payload, PutOptions::default()).await
}
/// Put with explicit options (chunk size, headers, sha256,
/// dedupe).
pub async fn put_with(
&self,
name: impl Into<String>,
payload: impl Into<Vec<u8>>,
options: PutOptions,
) -> Result<ObjectInfo> {
let mut c = self.client.streams_client();
let r = c
.put_object(Request::new(PutObjectRequest {
bucket: self.name.clone(),
name: name.into(),
payload: payload.into(),
chunk_size: options.chunk_size,
headers: options
.headers
.into_iter()
.map(|(k, v)| MessageHeader { key: k, value: v })
.collect(),
sha256: options.sha256,
dedupe: options.dedupe,
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
let info = r
.info
.ok_or_else(|| Error::server("internal", "missing info in PutObject response"))?;
Ok(info.into())
}
/// Get an object's whole payload + metadata.
pub async fn get(&self, name: impl Into<String>) -> Result<(ObjectInfo, Vec<u8>)> {
let mut c = self.client.streams_client();
let r = c
.get_object(Request::new(GetObjectRequest {
bucket: self.name.clone(),
name: name.into(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
let info = r
.info
.ok_or_else(|| Error::server("internal", "missing info in GetObject response"))?;
Ok((info.into(), r.payload))
}
/// Read a byte range of an object's payload. `len = 0` reads
/// the whole tail from `offset`.
pub async fn get_range(
&self,
name: impl Into<String>,
offset: u64,
len: u64,
) -> Result<(ObjectInfo, u64, Vec<u8>)> {
let mut c = self.client.streams_client();
let r = c
.get_object_range(Request::new(GetObjectRangeRequest {
bucket: self.name.clone(),
name: name.into(),
offset,
len,
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
let info = r
.info
.ok_or_else(|| Error::server("internal", "missing info in GetObjectRange response"))?;
Ok((info.into(), r.actual_offset, r.payload))
}
/// Get metadata without the payload bytes. Returns `None` if
/// the object has been tombstoned (deleted).
pub async fn info(&self, name: impl Into<String>) -> Result<Option<ObjectInfo>> {
let mut c = self.client.streams_client();
let r = c
.get_object_info(Request::new(GetObjectInfoRequest {
bucket: self.name.clone(),
name: name.into(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
if r.deleted {
return Ok(None);
}
Ok(r.info.map(Into::into))
}
/// Tombstone an object.
pub async fn delete(&self, name: impl Into<String>) -> Result<u64> {
let mut c = self.client.streams_client();
let r = c
.delete_object(Request::new(DeleteObjectRequest {
bucket: self.name.clone(),
name: name.into(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.tombstone_seq)
}
/// List objects in the bucket. Tombstoned entries excluded by
/// default — set `include_deleted` to include them.
pub async fn list(&self, name_prefix: impl Into<String>) -> Result<Vec<ObjectEntry>> {
self.list_inner(name_prefix.into(), /* include_deleted = */ false).await
}
/// Like [`Self::list`] but includes tombstoned entries.
pub async fn list_with_deleted(
&self,
name_prefix: impl Into<String>,
) -> Result<Vec<ObjectEntry>> {
self.list_inner(name_prefix.into(), true).await
}
async fn list_inner(&self, name_prefix: String, include_deleted: bool) -> Result<Vec<ObjectEntry>> {
let mut c = self.client.streams_client();
let r = c
.list_objects(Request::new(ListObjectsRequest {
bucket: self.name.clone(),
name_prefix,
include_deleted,
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.entries
.into_iter()
.map(|e| ObjectEntry {
name: e.name,
total_bytes: e.total_bytes,
deleted: e.deleted,
})
.collect())
}
/// List every metadata revision of `name` in seq order.
/// Useful for audit / debugging.
pub async fn revisions(&self, name: impl Into<String>) -> Result<Vec<ObjectRevision>> {
let mut c = self.client.streams_client();
let r = c
.list_object_revisions(Request::new(ListObjectRevisionsRequest {
bucket: self.name.clone(),
name: name.into(),
from_seq: 0,
limit: 0,
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.revisions
.into_iter()
.map(|rev| ObjectRevision {
metadata_seq: rev.metadata_seq,
total_bytes: rev.total_bytes,
sha256: rev.sha256,
ts_ms: rev.ts_ms,
deleted: rev.deleted,
})
.collect())
}
}
/// One revision of an object's metadata.
#[derive(Debug, Clone)]
pub struct ObjectRevision {
pub metadata_seq: u64,
pub total_bytes: u64,
pub sha256: String,
pub ts_ms: i64,
pub deleted: bool,
}
impl Client {
/// Create a new object-store bucket. Errors if the underlying
/// stream already exists.
pub async fn create_object_store(&self, config: Config) -> Result<Store> {
let name = config.name.clone();
let mut c = self.streams_client();
let r = c
.create_stream(Request::new(CreateStreamRequest {
config: Some(config.into_stream_config().into_pb()),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(Store::new(self.clone(), name))
}
/// Idempotent — create the bucket if it doesn't exist,
/// otherwise return a handle to the existing one.
pub async fn get_or_create_object_store(&self, config: Config) -> Result<Store> {
let name = config.name.clone();
self.get_or_create_stream(config.into_stream_config()).await?;
Ok(Store::new(self.clone(), name))
}
/// Return a handle to an existing object-store bucket without
/// verifying it exists server-side.
pub fn object_store(&self, name: impl Into<String>) -> Store {
Store::new(self.clone(), name.into())
}
}

View File

@ -0,0 +1,175 @@
//! Bloom filter — thin RPC binding.
use crate::client::Client;
use crate::error::{Error, Result};
use crate::sketches_server::{
BloomAddRequest, BloomDeleteRequest, BloomExistsRequest, BloomInfoRequest,
BloomMultiAddRequest, BloomMultiExistsRequest, BloomReserveRequest,
};
use tonic::Request;
#[derive(Debug, Clone, Default)]
pub struct BloomConfig {
pub name: String,
pub capacity: u64,
/// Target false-positive rate (e.g. 0.01 for 1%). 0.0 = server
/// default (0.01).
pub error_rate: f64,
}
#[derive(Debug, Clone)]
pub struct BloomInfo {
pub capacity: u64,
pub error_rate: f64,
pub bits_set: u64,
pub bit_count: u64,
pub hash_count: u32,
pub items_added: u64,
}
#[derive(Clone)]
pub struct Bloom {
client: Client,
name: String,
}
impl Bloom {
pub(crate) fn new(client: Client, name: String) -> Self {
Self { client, name }
}
pub fn name(&self) -> &str {
&self.name
}
pub async fn add(&self, item: impl Into<Vec<u8>>) -> Result<()> {
let mut c = self.client.sketches_client();
let r = c
.bloom_add(Request::new(BloomAddRequest {
name: self.name.clone(),
item: item.into(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(())
}
pub async fn add_many<I, T>(&self, items: I) -> Result<()>
where
I: IntoIterator<Item = T>,
T: Into<Vec<u8>>,
{
let mut c = self.client.sketches_client();
let r = c
.bloom_multi_add(Request::new(BloomMultiAddRequest {
name: self.name.clone(),
items: items.into_iter().map(Into::into).collect(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(())
}
/// Probabilistic membership test. `true` = probably present
/// (may be a false positive); `false` = definitely absent.
pub async fn exists(&self, item: impl Into<Vec<u8>>) -> Result<bool> {
let mut c = self.client.sketches_client();
let r = c
.bloom_exists(Request::new(BloomExistsRequest {
name: self.name.clone(),
item: item.into(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.exists)
}
pub async fn exists_many<I, T>(&self, items: I) -> Result<Vec<bool>>
where
I: IntoIterator<Item = T>,
T: Into<Vec<u8>>,
{
let mut c = self.client.sketches_client();
let r = c
.bloom_multi_exists(Request::new(BloomMultiExistsRequest {
name: self.name.clone(),
items: items.into_iter().map(Into::into).collect(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.exists)
}
pub async fn info(&self) -> Result<BloomInfo> {
let mut c = self.client.sketches_client();
let r = c
.bloom_info(Request::new(BloomInfoRequest {
name: self.name.clone(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(BloomInfo {
capacity: r.capacity,
error_rate: r.error_rate,
bits_set: r.bits_set,
bit_count: r.bit_count,
hash_count: r.hash_count,
items_added: r.items_added,
})
}
}
impl Client {
/// Reserve a new Bloom filter.
pub async fn create_bloom(&self, config: BloomConfig) -> Result<Bloom> {
let mut c = self.sketches_client();
let r = c
.bloom_reserve(Request::new(BloomReserveRequest {
name: config.name.clone(),
capacity: config.capacity,
error_rate: config.error_rate,
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(Bloom::new(self.clone(), config.name))
}
/// Return a handle to an existing Bloom filter without
/// verifying. First operation on a missing filter fails with
/// `no_such_filter`.
pub fn bloom(&self, name: impl Into<String>) -> Bloom {
Bloom::new(self.clone(), name.into())
}
/// Delete a Bloom filter and free its memory.
pub async fn delete_bloom(&self, name: impl Into<String>) -> Result<()> {
let mut c = self.sketches_client();
let r = c
.bloom_delete(Request::new(BloomDeleteRequest {
name: name.into(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(())
}
}

View File

@ -0,0 +1,109 @@
//! Count-Min Sketch — typed bindings.
//!
//! Server-side implementation is a follow-up slice; every call
//! currently surfaces `Error::Server { code: "unimplemented", .. }`.
use crate::client::Client;
use crate::error::{Error, Result};
use crate::sketches_server::{
CmsDeleteRequest, CmsIncrByItem, CmsIncrByRequest, CmsQueryRequest, CmsReserveRequest,
};
use tonic::Request;
#[derive(Debug, Clone, Default)]
pub struct CmsConfig {
pub name: String,
/// Sketch width (columns). 0 = server default.
pub width: u64,
/// Sketch depth (rows / hash functions). 0 = server default.
pub depth: u64,
}
#[derive(Clone)]
pub struct Cms {
client: Client,
name: String,
}
impl Cms {
pub(crate) fn new(client: Client, name: String) -> Self {
Self { client, name }
}
pub fn name(&self) -> &str { &self.name }
pub async fn incr<I>(&self, items: I) -> Result<Vec<u64>>
where
I: IntoIterator<Item = (Vec<u8>, u64)>,
{
let mut c = self.client.sketches_client();
let r = c
.cms_incr_by(Request::new(CmsIncrByRequest {
name: self.name.clone(),
items: items
.into_iter()
.map(|(item, count)| CmsIncrByItem { item, count })
.collect(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.counts)
}
pub async fn query<I, T>(&self, items: I) -> Result<Vec<u64>>
where
I: IntoIterator<Item = T>,
T: Into<Vec<u8>>,
{
let mut c = self.client.sketches_client();
let r = c
.cms_query(Request::new(CmsQueryRequest {
name: self.name.clone(),
items: items.into_iter().map(Into::into).collect(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.counts)
}
}
impl Client {
pub async fn create_cms(&self, config: CmsConfig) -> Result<Cms> {
let mut c = self.sketches_client();
let r = c
.cms_reserve(Request::new(CmsReserveRequest {
name: config.name.clone(),
width: config.width,
depth: config.depth,
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(Cms::new(self.clone(), config.name))
}
pub fn cms(&self, name: impl Into<String>) -> Cms {
Cms::new(self.clone(), name.into())
}
pub async fn delete_cms(&self, name: impl Into<String>) -> Result<()> {
let mut c = self.sketches_client();
let r = c
.cms_delete(Request::new(CmsDeleteRequest {
name: name.into(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(())
}
}

View File

@ -0,0 +1,120 @@
//! HyperLogLog cardinality estimator — thin RPC binding.
use crate::client::Client;
use crate::error::{Error, Result};
use crate::sketches_server::{
HllAddRequest, HllCountRequest, HllDeleteRequest, HllMergeRequest, HllReserveRequest,
};
use tonic::Request;
#[derive(Debug, Clone, Default)]
pub struct HllConfig {
pub name: String,
/// 2^precision = register count. Valid range 4..=18. 0 =
/// server default (14, ~16 KB, ~1% error).
pub precision: u32,
}
#[derive(Clone)]
pub struct Hll {
client: Client,
name: String,
}
impl Hll {
pub(crate) fn new(client: Client, name: String) -> Self {
Self { client, name }
}
pub fn name(&self) -> &str {
&self.name
}
pub async fn add<I, T>(&self, items: I) -> Result<()>
where
I: IntoIterator<Item = T>,
T: Into<Vec<u8>>,
{
let mut c = self.client.sketches_client();
let r = c
.hll_add(Request::new(HllAddRequest {
name: self.name.clone(),
items: items.into_iter().map(Into::into).collect(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(())
}
/// Estimated cardinality.
pub async fn count(&self) -> Result<u64> {
let mut c = self.client.sketches_client();
let r = c
.hll_count(Request::new(HllCountRequest {
name: self.name.clone(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.estimate)
}
/// Merge `sources` into this HLL (union of their registers).
pub async fn merge_from<I, S>(&self, sources: I) -> Result<()>
where
I: IntoIterator<Item = S>,
S: Into<String>,
{
let mut c = self.client.sketches_client();
let r = c
.hll_merge(Request::new(HllMergeRequest {
destination: self.name.clone(),
sources: sources.into_iter().map(Into::into).collect(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(())
}
}
impl Client {
pub async fn create_hll(&self, config: HllConfig) -> Result<Hll> {
let mut c = self.sketches_client();
let r = c
.hll_reserve(Request::new(HllReserveRequest {
name: config.name.clone(),
precision: config.precision,
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(Hll::new(self.clone(), config.name))
}
pub fn hll(&self, name: impl Into<String>) -> Hll {
Hll::new(self.clone(), name.into())
}
pub async fn delete_hll(&self, name: impl Into<String>) -> Result<()> {
let mut c = self.sketches_client();
let r = c
.hll_delete(Request::new(HllDeleteRequest {
name: name.into(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(())
}
}

View File

@ -0,0 +1,22 @@
//! Probabilistic data structures — thin RPC bindings.
//!
//! - [`bloom`] — Bloom filter (functional)
//! - [`hll`] — HyperLogLog cardinality estimator (functional)
//! - [`cms`] — Count-Min Sketch (proto+API present; server returns
//! `unimplemented` — implementation is a follow-up slice)
//! - [`topk`] — Top-K (proto+API present; server returns
//! `unimplemented`)
//! - [`tdigest`] — t-digest quantile sketch (proto+API present;
//! server returns `unimplemented`)
pub mod bloom;
pub mod cms;
pub mod hll;
pub mod tdigest;
pub mod topk;
pub use bloom::{Bloom, BloomConfig, BloomInfo};
pub use cms::{Cms, CmsConfig};
pub use hll::{Hll, HllConfig};
pub use tdigest::{TDigest, TDigestConfig};
pub use topk::{TopK, TopKConfig, TopKListEntry};

View File

@ -0,0 +1,116 @@
//! t-digest quantile sketch — typed bindings. Server-side
//! implementation deferred.
use crate::client::Client;
use crate::error::{Error, Result};
use crate::sketches_server::{
TDigestAddRequest, TDigestCreateRequest, TDigestDeleteRequest, TDigestMinMaxRequest,
TDigestQuantileRequest,
};
use tonic::Request;
#[derive(Debug, Clone, Default)]
pub struct TDigestConfig {
pub name: String,
/// Compression. Higher = better tail accuracy at cost of
/// memory. 0 = server default.
pub compression: u32,
}
#[derive(Clone)]
pub struct TDigest {
client: Client,
name: String,
}
impl TDigest {
pub(crate) fn new(client: Client, name: String) -> Self {
Self { client, name }
}
pub fn name(&self) -> &str { &self.name }
pub async fn add<I>(&self, values: I) -> Result<()>
where
I: IntoIterator<Item = f64>,
{
let mut c = self.client.sketches_client();
let r = c
.t_digest_add(Request::new(TDigestAddRequest {
name: self.name.clone(),
values: values.into_iter().collect(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(())
}
pub async fn quantile<I>(&self, quantiles: I) -> Result<Vec<f64>>
where
I: IntoIterator<Item = f64>,
{
let mut c = self.client.sketches_client();
let r = c
.t_digest_quantile(Request::new(TDigestQuantileRequest {
name: self.name.clone(),
quantiles: quantiles.into_iter().collect(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.values)
}
pub async fn min_max(&self) -> Result<(f64, f64)> {
let mut c = self.client.sketches_client();
let r = c
.t_digest_min_max(Request::new(TDigestMinMaxRequest {
name: self.name.clone(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok((r.min, r.max))
}
}
impl Client {
pub async fn create_tdigest(&self, config: TDigestConfig) -> Result<TDigest> {
let mut c = self.sketches_client();
let r = c
.t_digest_create(Request::new(TDigestCreateRequest {
name: config.name.clone(),
compression: config.compression,
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(TDigest::new(self.clone(), config.name))
}
pub fn tdigest(&self, name: impl Into<String>) -> TDigest {
TDigest::new(self.clone(), name.into())
}
pub async fn delete_tdigest(&self, name: impl Into<String>) -> Result<()> {
let mut c = self.sketches_client();
let r = c
.t_digest_delete(Request::new(TDigestDeleteRequest {
name: name.into(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(())
}
}

View File

@ -0,0 +1,134 @@
//! Top-K — typed bindings. Server-side implementation deferred.
use crate::client::Client;
use crate::error::{Error, Result};
use crate::sketches_server::{
TopKAddRequest, TopKDeleteRequest, TopKListRequest, TopKQueryRequest, TopKReserveRequest,
};
use tonic::Request;
#[derive(Debug, Clone, Default)]
pub struct TopKConfig {
pub name: String,
pub k: u32,
/// Underlying sketch width. 0 = server default.
pub width: u64,
/// Underlying sketch depth. 0 = server default.
pub depth: u64,
/// Probability-decay factor (0.0..=1.0). 0 = server default.
pub decay: f64,
}
#[derive(Debug, Clone)]
pub struct TopKListEntry {
pub item: Vec<u8>,
pub count: u64,
}
#[derive(Clone)]
pub struct TopK {
client: Client,
name: String,
}
impl TopK {
pub(crate) fn new(client: Client, name: String) -> Self {
Self { client, name }
}
pub fn name(&self) -> &str { &self.name }
/// Add items. Returns one entry per request item:
/// the evicted item (if any) for that slot, or empty bytes.
pub async fn add<I, T>(&self, items: I) -> Result<Vec<Vec<u8>>>
where
I: IntoIterator<Item = T>,
T: Into<Vec<u8>>,
{
let mut c = self.client.sketches_client();
let r = c
.top_k_add(Request::new(TopKAddRequest {
name: self.name.clone(),
items: items.into_iter().map(Into::into).collect(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.evicted)
}
pub async fn query<I, T>(&self, items: I) -> Result<Vec<bool>>
where
I: IntoIterator<Item = T>,
T: Into<Vec<u8>>,
{
let mut c = self.client.sketches_client();
let r = c
.top_k_query(Request::new(TopKQueryRequest {
name: self.name.clone(),
items: items.into_iter().map(Into::into).collect(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.in_top_k)
}
pub async fn list(&self) -> Result<Vec<TopKListEntry>> {
let mut c = self.client.sketches_client();
let r = c
.top_k_list(Request::new(TopKListRequest {
name: self.name.clone(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.entries
.into_iter()
.map(|e| TopKListEntry { item: e.item, count: e.count })
.collect())
}
}
impl Client {
pub async fn create_topk(&self, config: TopKConfig) -> Result<TopK> {
let mut c = self.sketches_client();
let r = c
.top_k_reserve(Request::new(TopKReserveRequest {
name: config.name.clone(),
k: config.k,
width: config.width,
depth: config.depth,
decay: config.decay,
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(TopK::new(self.clone(), config.name))
}
pub fn topk(&self, name: impl Into<String>) -> TopK {
TopK::new(self.clone(), name.into())
}
pub async fn delete_topk(&self, name: impl Into<String>) -> Result<()> {
let mut c = self.sketches_client();
let r = c
.top_k_delete(Request::new(TopKDeleteRequest {
name: name.into(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(())
}
}

371
rust/src/stream/consumer.rs Normal file
View File

@ -0,0 +1,371 @@
//! Consumer handle + delivered messages.
//!
//! - `Consumer::messages()` returns a `Stream<Item = Result<Message>>`
//! that yields delivered messages one at a time, via the server's
//! `Subscribe` server-streaming RPC.
//! - `Consumer::fetch(batch)` returns a one-shot batch via the
//! `Fetch` unary RPC — for pull-style polling consumers.
//! - Each `Message` carries a back-reference to the client so
//! `message.ack()` / `nak()` / `term()` / `in_progress()` can
//! round-trip to the server.
use crate::client::Client;
use crate::error::{Error, Result};
use crate::streams_server::{
AckRequest, FetchRequest, InProgressRequest, MessageHeader, NakRequest, SubscribeRequest,
TermRequest,
};
use std::pin::Pin;
use std::task::{Context as TaskContext, Poll};
use tonic::Request;
/// A consumer handle. Returned by `Stream::create_consumer` /
/// `get_consumer`.
#[derive(Clone)]
pub struct Consumer {
pub(crate) client: Client,
pub(crate) stream: String,
pub(crate) name: String,
}
impl Consumer {
pub(crate) fn new(client: Client, stream: String, name: String) -> Self {
Self { client, stream, name }
}
/// The consumer's durable name.
pub fn name(&self) -> &str {
&self.name
}
/// The parent stream's name.
pub fn stream(&self) -> &str {
&self.stream
}
/// Open a push-mode subscription. Returns a `Stream` of
/// delivered messages — equivalent to async-nats's
/// `consumer.messages().await?`.
///
/// The server holds the subscription open until the client
/// drops the returned stream, the consumer is deleted, or an
/// RPC error occurs.
pub async fn messages(&self) -> Result<Messages> {
self.messages_with_batch_size(0).await
}
/// Like `messages()` but with a custom server-side batch
/// hint. 0 = server default (currently 16).
pub async fn messages_with_batch_size(&self, batch_size: u32) -> Result<Messages> {
let mut c = self.client.streams_client();
let stream = c
.subscribe(Request::new(SubscribeRequest {
stream: self.stream.clone(),
consumer: self.name.clone(),
batch_size,
stop_when_empty: false,
}))
.await?
.into_inner();
Ok(Messages {
inner: stream,
client: self.client.clone(),
stream_name: self.stream.clone(),
consumer_name: self.name.clone(),
})
}
/// Pull-style: fetch up to `batch_size` messages right now.
/// Returns the batch immediately (possibly empty) without
/// blocking for new arrivals.
pub async fn fetch(&self, batch_size: u32) -> Result<Vec<Message>> {
let mut c = self.client.streams_client();
let r = c
.fetch(Request::new(FetchRequest {
stream: self.stream.clone(),
consumer: self.name.clone(),
batch_size,
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.messages
.into_iter()
.map(|m| Message::from_pb(self.client.clone(), self.stream.clone(), self.name.clone(), m))
.collect())
}
}
/// A delivered message. Carries enough context to ack/nak/term/
/// in-progress through the parent consumer.
pub struct Message {
pub subject: String,
pub payload: Vec<u8>,
pub headers: Vec<(String, String)>,
pub sequence: u64,
pub ts_ms: i64,
/// How many times this seq has been delivered to this consumer
/// (1 on first delivery; climbs on redelivery).
pub deliver_count: u32,
client: Client,
stream: String,
consumer: String,
}
impl Message {
fn from_pb(
client: Client,
stream: String,
consumer: String,
m: crate::streams_server::MessagePb,
) -> Self {
Self {
subject: m.subject,
payload: m.payload,
headers: m
.headers
.into_iter()
.map(|MessageHeader { key, value }| (key, value))
.collect(),
sequence: m.seq,
ts_ms: m.ts_ms,
deliver_count: m.deliver_count,
client,
stream,
consumer,
}
}
/// Positive acknowledgment — the message is consumed.
pub async fn ack(&self) -> Result<()> {
let mut c = self.client.streams_client();
let r = c
.ack(Request::new(AckRequest {
stream: self.stream.clone(),
consumer: self.consumer.clone(),
seq: self.sequence,
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(())
}
/// Negative acknowledgment — request immediate redelivery.
/// `deliver_count` keeps climbing toward `max_deliver`.
pub async fn nak(&self) -> Result<()> {
self.nak_with_delay(std::time::Duration::ZERO).await
}
/// Negative acknowledgment with delay — server defers
/// redelivery by `delay`.
pub async fn nak_with_delay(&self, delay: std::time::Duration) -> Result<()> {
let mut c = self.client.streams_client();
let r = c
.nak(Request::new(NakRequest {
stream: self.stream.clone(),
consumer: self.consumer.clone(),
seq: self.sequence,
delay_ms: delay.as_millis() as u64,
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(())
}
/// Terminal acknowledgment — drop the message permanently,
/// regardless of `max_deliver`. The message is NOT removed
/// from WorkQueue retention (other consumers can still
/// observe it).
pub async fn term(&self) -> Result<()> {
let mut c = self.client.streams_client();
let r = c
.term(Request::new(TermRequest {
stream: self.stream.clone(),
consumer: self.consumer.clone(),
seq: self.sequence,
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(())
}
/// Heartbeat — extend the `ack_wait` window without acking.
/// `deliver_count` is unchanged.
pub async fn in_progress(&self) -> Result<()> {
let mut c = self.client.streams_client();
let r = c
.in_progress(Request::new(InProgressRequest {
stream: self.stream.clone(),
consumer: self.consumer.clone(),
seq: self.sequence,
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(())
}
}
/// A `Stream<Item = Result<Message>>` returned by
/// `Consumer::messages()`. Yields delivered messages until the
/// underlying server-streaming RPC terminates (server-side close,
/// network error, or client drop).
///
/// Implementations of the `futures::Stream` trait are provided via
/// `tokio_stream::Stream` (re-exported below) so callers can use
/// `while let Some(msg) = stream.next().await { … }` as they do
/// with async-nats.
pub struct Messages {
inner: tonic::Streaming<crate::streams_server::SubscribeEvent>,
client: Client,
stream_name: String,
consumer_name: String,
}
impl tokio_stream::Stream for Messages {
type Item = Result<Message>;
fn poll_next(mut self: Pin<&mut Self>, cx: &mut TaskContext<'_>) -> Poll<Option<Self::Item>> {
// Loop past stream-event frames that don't carry a message
// payload (control frames: subscribe-stopped, etc).
loop {
match Pin::new(&mut self.inner).poll_next(cx) {
Poll::Pending => return Poll::Pending,
Poll::Ready(None) => return Poll::Ready(None),
Poll::Ready(Some(Err(s))) => return Poll::Ready(Some(Err(Error::Rpc(s)))),
Poll::Ready(Some(Ok(event))) => {
use crate::streams_server::subscribe_event::Event as Ev;
match event.event {
Some(Ev::Message(m)) => {
let msg = Message::from_pb(
self.client.clone(),
self.stream_name.clone(),
self.consumer_name.clone(),
m,
);
return Poll::Ready(Some(Ok(msg)));
}
Some(Ev::Stopped(stopped)) => {
// Server-side close. Carry the reason
// out via Err so callers can distinguish
// a normal EOS (None) from "server told
// us to stop because the consumer was
// deleted" etc.
if stopped.reason.is_empty() {
return Poll::Ready(None);
}
return Poll::Ready(Some(Err(Error::server(
"subscribe_stopped",
stopped.reason,
))));
}
None => continue,
}
}
}
}
}
}
/// Consumer delivery start policy.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum DeliverPolicy {
/// Start from the first message in the stream.
All,
/// Start from messages published after subscription (emulated
/// via `ByStartTime(now)` at consumer creation since waymaker
/// has no native `deliver-new`).
New,
/// Start at the last message published.
Last,
/// Start at the given sequence.
ByStartSequence(u64),
/// Start at messages published at or after `start_time_ms`.
ByStartTime(i64),
}
impl Default for DeliverPolicy {
fn default() -> Self {
Self::All
}
}
impl DeliverPolicy {
fn into_pb(self) -> crate::streams_server::DeliveryPolicyPb {
use crate::streams_server::DeliveryPolicyType as T;
let (ty, seq, t_ms) = match self {
DeliverPolicy::All => (T::DeliveryAll, 0, 0),
DeliverPolicy::New => {
let now_ms = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_millis() as i64)
.unwrap_or(0);
(T::DeliveryByStartTime, 0, now_ms)
}
DeliverPolicy::Last => (T::DeliveryLast, 0, 0),
DeliverPolicy::ByStartSequence(s) => (T::DeliveryByStartSeq, s, 0),
DeliverPolicy::ByStartTime(t) => (T::DeliveryByStartTime, 0, t),
};
crate::streams_server::DeliveryPolicyPb {
r#type: ty as i32,
start_seq: seq,
start_time_ms: t_ms,
}
}
}
/// Consumer configuration.
///
/// Waymaker is `ack_policy=Explicit` + `replay_policy=Instant` only,
/// so those async-nats fields aren't exposed.
#[derive(Debug, Clone, Default)]
pub struct ConsumerConfig {
pub durable_name: Option<String>,
pub filter_subject: Option<String>,
pub deliver_policy: DeliverPolicy,
/// How long the server waits for an Ack before considering a
/// delivery redeliverable. Defaults to 30s.
pub ack_wait: std::time::Duration,
/// Cap on delivery attempts. Defaults to 5.
pub max_deliver: u32,
/// Queue-group label. Multiple workers in the same group share
/// the message stream round-robin.
pub deliver_group: Option<String>,
/// Optional dead-letter subject. Messages dropped past
/// `max_deliver` are re-published here within the same stream.
pub dead_letter_subject: Option<String>,
}
impl ConsumerConfig {
pub(crate) fn into_pb(self, fallback_name: &str) -> crate::streams_server::ConsumerConfigPb {
let name = self.durable_name.unwrap_or_else(|| fallback_name.into());
let ack_wait_ms = if self.ack_wait.is_zero() {
30_000
} else {
self.ack_wait.as_millis() as u64
};
let max_deliver = if self.max_deliver == 0 { 5 } else { self.max_deliver };
crate::streams_server::ConsumerConfigPb {
name,
filter_subject: self.filter_subject.unwrap_or_default(),
delivery_policy: Some(self.deliver_policy.into_pb()),
ack_wait_ms,
max_deliver,
deliver_group: self.deliver_group.unwrap_or_default(),
dead_letter_subject: self.dead_letter_subject.unwrap_or_default(),
}
}
}

363
rust/src/stream/mod.rs Normal file
View File

@ -0,0 +1,363 @@
//! Streams subsystem. Owns the stream handle and all stream-
//! specific types (configs, retention, consumer types).
//!
//! Entry points live on `Client`:
//! - `client.create_stream(stream::StreamConfig { ... })`
//! - `client.get_stream("name")`
//! - `client.get_or_create_stream(cfg)`
//!
//! Returned `Stream` carries the per-stream operations:
//! - `stream.publish(subject, payload)`
//! - `stream.create_consumer(stream::ConsumerConfig { ... })`
pub mod consumer;
pub use consumer::{Consumer, ConsumerConfig, DeliverPolicy, Message, Messages};
use crate::client::Client;
use crate::error::{Error, Result};
use crate::streams_server::{
self as pb, CreateConsumerRequest, DeleteConsumerRequest, GetConsumerInfoRequest,
GetStreamInfoRequest, PublishRequest,
};
use std::time::Duration;
use tonic::Request;
/// One row of per-(sourcing, source) status returned by
/// `Stream::sources_status` and `Client::get_stream_sources`.
#[derive(Debug, Clone)]
pub struct SourceStatus {
pub sourcing_stream: String,
pub source_stream: String,
pub last_sourced_seq: u64,
pub pulled_total: u64,
pub last_error: String,
pub last_error_ts_ms: i64,
}
/// A reference to a stream on the server. Cheap to clone.
#[derive(Clone)]
pub struct Stream {
pub(crate) client: Client,
pub(crate) name: String,
}
impl Stream {
pub(crate) fn new(client: Client, name: String) -> Self {
Self { client, name }
}
/// The stream's name.
pub fn name(&self) -> &str {
&self.name
}
/// Publish a message into this stream.
pub async fn publish(
&self,
subject: impl Into<String>,
payload: impl Into<Vec<u8>>,
) -> Result<PublishAck> {
self.publish_with_headers(subject, std::iter::empty::<(String, String)>(), payload)
.await
}
/// Publish with explicit headers.
pub async fn publish_with_headers(
&self,
subject: impl Into<String>,
headers: impl IntoIterator<Item = (String, String)>,
payload: impl Into<Vec<u8>>,
) -> Result<PublishAck> {
let mut c = self.client.streams_client();
let req = PublishRequest {
stream: self.name.clone(),
subject: subject.into(),
payload: payload.into(),
headers: headers
.into_iter()
.map(|(k, v)| pb::MessageHeader { key: k, value: v })
.collect(),
ts_ms: 0,
expected_last_seq: None,
};
let r = c.publish(Request::new(req)).await?.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(PublishAck { sequence: r.seq })
}
/// Slice 3: per-source tail status for this stream. Returns
/// one entry per source feeding this stream. Empty when the
/// stream has no sources OR the tail tasks are running on a
/// different node (this RPC returns the local primary's view).
pub async fn sources_status(&self) -> Result<Vec<SourceStatus>> {
let mut c = self.client.streams_client();
let r = c
.get_stream_info(Request::new(GetStreamInfoRequest {
name: self.name.clone(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(r.sources_status
.into_iter()
.map(|s| SourceStatus {
sourcing_stream: self.name.clone(),
source_stream: s.source_stream,
last_sourced_seq: s.last_sourced_seq,
pulled_total: s.pulled_total,
last_error: s.last_error,
last_error_ts_ms: s.last_error_ts_ms,
})
.collect())
}
/// Create a new consumer on this stream. Errors if a consumer
/// with the same `durable_name` already exists.
pub async fn create_consumer(&self, config: ConsumerConfig) -> Result<Consumer> {
let name = config
.durable_name
.clone()
.ok_or_else(|| Error::Invalid("ConsumerConfig.durable_name is required".into()))?;
let cfg_pb = config.into_pb(&name);
let mut c = self.client.streams_client();
let r = c
.create_consumer(Request::new(CreateConsumerRequest {
stream: self.name.clone(),
config: Some(cfg_pb),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(Consumer::new(self.client.clone(), self.name.clone(), name))
}
/// Idempotent variant — returns an existing consumer if one
/// with the same `durable_name` already exists, otherwise
/// creates it.
pub async fn get_or_create_consumer(&self, config: ConsumerConfig) -> Result<Consumer> {
let name = config
.durable_name
.clone()
.ok_or_else(|| Error::Invalid("ConsumerConfig.durable_name is required".into()))?;
match self.get_consumer(&name).await {
Ok(c) => Ok(c),
Err(Error::Server { code, .. }) if code == "no_such_consumer" => {
self.create_consumer(config).await
}
Err(e) => Err(e),
}
}
/// Return a handle to an existing consumer.
pub async fn get_consumer(&self, name: impl Into<String>) -> Result<Consumer> {
let name: String = name.into();
let mut c = self.client.streams_client();
let r = c
.get_consumer_info(Request::new(GetConsumerInfoRequest {
stream: self.name.clone(),
consumer: name.clone(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(Consumer::new(self.client.clone(), self.name.clone(), name))
}
/// Delete a consumer by name.
pub async fn delete_consumer(&self, name: impl Into<String>) -> Result<()> {
let mut c = self.client.streams_client();
let r = c
.delete_consumer(Request::new(DeleteConsumerRequest {
stream: self.name.clone(),
consumer: name.into(),
}))
.await?
.into_inner();
if !r.success {
return Err(Error::server(r.result_code, r.message));
}
Ok(())
}
}
/// Returned by `Stream::publish` — just the assigned stream
/// sequence; new fields can be added without breaking callers.
#[derive(Debug, Clone, Copy)]
pub struct PublishAck {
pub sequence: u64,
}
/// Retention policy.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum RetentionPolicy {
Limits,
WorkQueue,
Interest,
}
impl Default for RetentionPolicy {
fn default() -> Self {
Self::Limits
}
}
/// Stream configuration. `Default` produces an unbounded Limits
/// stream with no subject filter, suitable for
/// `StreamConfig { name: ..., ..Default::default() }`.
#[derive(Debug, Clone, Default)]
pub struct StreamConfig {
pub name: String,
pub subjects: Vec<String>,
pub retention: RetentionPolicy,
/// Maximum age before messages are pruned. `None` = unbounded.
pub max_age: Option<Duration>,
/// Maximum number of messages. `None` = unbounded.
pub max_messages: Option<u64>,
/// Maximum total stored bytes. `None` = unbounded.
pub max_bytes: Option<u64>,
/// Max bytes per individual message. `None` = no cap.
pub max_message_size: Option<u64>,
/// Block size override. 0 = server default (100_000).
pub block_size: u64,
/// Reject publishes that would push over `max_bytes` /
/// `max_messages` instead of silently dropping the oldest.
pub strict_limits: bool,
/// Memory-only stream. Survives node failover via replication
/// but a full-cluster restart loses it.
pub ephemeral: bool,
/// Per-subject revision cap. `0` (default) = unbounded.
/// When N > 0, after each publish, older messages at that
/// subject beyond the N most recent are dropped. Mirrors
/// NATS JetStream's `MaxMsgsPerSubject` (and backs KV's
/// `max_revisions` knob).
pub max_msgs_per_subject: u64,
/// Cross-stream sources. Slice 1: at most one entry, name-only.
/// See `crates/streams/specs/SOURCES_DESIGN.md`.
pub sources: Vec<StreamSource>,
}
/// Identifies a stream to tail from.
#[derive(Debug, Clone, Default)]
pub struct StreamSource {
pub source_stream: String,
/// Optional NATS pattern. Only source messages whose subject
/// matches are pulled. Empty = pull all.
pub filter_subject: String,
/// First source-seq to pull. `0` (default) = pull from
/// beginning of source (subject to `max_initial_backfill`).
/// Mutually exclusive with `start_time_ms`.
pub start_seq: u64,
/// Slice 3: start position by wall-clock timestamp (ms since
/// epoch). When set, the tail resolves to the first message
/// with `ts_ms >= start_time_ms` and seeds there. Future-of-
/// stream resolves to "seed at current end". Mutually exclusive
/// with `start_seq`.
pub start_time_ms: i64,
/// Slice 2F: cap on the initial backfill window. `0` = no cap.
/// When set, the tail seeds at `max(0, source.last_seq -
/// max_initial_backfill)`. Ignored once durable state exists.
pub max_initial_backfill: u64,
/// Slice 3: NATS-style subject rewrite. When set, the source's
/// subject is matched against `subject_transform.source_pattern`
/// and the captured wildcards substituted into
/// `subject_transform.destination` (1-indexed `{{wildcard(N)}}`
/// placeholders). The sourcing stream sees the rewritten
/// subject; the original is preserved on the
/// `waymaker-source-subject` header.
pub subject_transform: Option<SubjectTransform>,
/// Slice 3: what to do when source retention drops messages
/// past our last_sourced_seq. Default = Halt.
pub on_drop: OnDropPolicy,
/// Slice 3: dead-letter stream name. When non-empty, tail
/// errors (transform mismatch, halt-on-drop firing, etc.) are
/// published as JSON records to this stream so operators can
/// triage without scraping logs.
pub dlq_stream: String,
}
#[derive(Debug, Clone, Default)]
pub struct SubjectTransform {
pub source_pattern: String,
pub destination: String,
}
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub enum OnDropPolicy {
/// Tail records a persistent error + stops advancing.
#[default]
Halt,
/// Tail jumps watermark to source's first available seq.
SkipToFirstAvailable,
}
impl StreamConfig {
pub(crate) fn into_pb(self) -> pb::StreamConfigPb {
let retention = match self.retention {
RetentionPolicy::Limits => pb::Retention {
policy: Some(pb::retention::Policy::Limits(pb::LimitsRetention {
max_age_ms: self.max_age.map(|d| d.as_millis() as u64),
max_msgs: self.max_messages,
max_bytes: self.max_bytes,
strict_limits: self.strict_limits,
})),
},
RetentionPolicy::WorkQueue => pb::Retention {
policy: Some(pb::retention::Policy::WorkQueue(pb::WorkQueueRetention {})),
},
RetentionPolicy::Interest => pb::Retention {
policy: Some(pb::retention::Policy::Interest(pb::InterestRetention {})),
},
};
pb::StreamConfigPb {
name: self.name,
subjects_filter: self.subjects,
retention: Some(retention),
block_size: self.block_size,
max_msg_bytes: self.max_message_size.unwrap_or(0),
ephemeral: self.ephemeral,
max_msgs_per_subject: self.max_msgs_per_subject,
sources: self
.sources
.into_iter()
.map(|s| pb::StreamSourceConfigPb {
source_stream: s.source_stream,
filter_subject: s.filter_subject,
start_seq: s.start_seq,
start_time_ms: s.start_time_ms,
subject_transform: s.subject_transform.map(|t| {
pb::SubjectTransformPb {
source_pattern: t.source_pattern,
destination: t.destination,
}
}),
max_initial_backfill: s.max_initial_backfill,
on_drop: match s.on_drop {
OnDropPolicy::Halt => 0,
OnDropPolicy::SkipToFirstAvailable => 1,
},
dlq_stream: s.dlq_stream,
})
.collect(),
}
}
}
/// Partial-update payload for `Client::update_stream`. Any field
/// left at `None` is not touched server-side.
#[derive(Debug, Clone, Default)]
pub struct StreamUpdate {
pub max_age_ms: Option<Duration>,
pub max_msgs: Option<u64>,
pub max_bytes: Option<u64>,
pub max_msg_bytes: Option<u64>,
pub strict_limits: Option<bool>,
}

37
scripts/gen-go.sh Executable file
View File

@ -0,0 +1,37 @@
#!/usr/bin/env bash
# Generate Go gRPC stubs from the vendored protos into go/genpb/<subsystem>.
# The protos carry no `option go_package`, so each is mapped explicitly here.
# internal_proxy.proto is server-internal and intentionally skipped.
set -euo pipefail
here="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "$here/go"
export PATH="$(go env GOPATH)/bin:$PATH"
MOD=git.awesomike.com/pub/waymaker-client/go
# proto filename (relative to ../proto) -> Go package dir under the module
protos=(
waymaker_locks.proto:genpb/locks
waymaker_streams.proto:genpb/streams
kv.proto:genpb/kv
collections.proto:genpb/collections
sketches.proto:genpb/sketches
cache.proto:genpb/cache
)
files=()
mflags=()
for entry in "${protos[@]}"; do
f="${entry%%:*}"; pkg="${entry##*:}"
files+=("$f")
mflags+=("--go_opt=M$f=$MOD/$pkg" "--go-grpc_opt=M$f=$MOD/$pkg")
done
rm -rf genpb
protoc -I ../proto \
--go_out=. --go_opt=module="$MOD" \
--go-grpc_out=. --go-grpc_opt=module="$MOD" \
"${mflags[@]}" \
"${files[@]}"
echo "Generated Go stubs under go/genpb/"

30
scripts/gen-ts.sh Executable file
View File

@ -0,0 +1,30 @@
#!/usr/bin/env bash
# Generate TypeScript gRPC stubs from the vendored protos into ts/src/genpb,
# using ts-proto with @grpc/grpc-js service output. internal_proxy.proto is
# server-internal and intentionally skipped.
set -euo pipefail
here="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "$here/ts"
plugin="node_modules/.bin/protoc-gen-ts_proto"
if [[ ! -x "$plugin" ]]; then
echo "ts-proto not installed; run 'npm install' in ts/ first" >&2
exit 1
fi
out="src/genpb"
rm -rf "$out"; mkdir -p "$out"
protoc -I ../proto \
--plugin=protoc-gen-ts_proto="$plugin" \
--ts_proto_out="$out" \
--ts_proto_opt=outputServices=grpc-js,esModuleInterop=true,env=node,useExactTypes=false,unrecognizedEnum=false \
waymaker_locks.proto \
waymaker_streams.proto \
kv.proto \
collections.proto \
sketches.proto \
cache.proto
echo "Generated TypeScript stubs under ts/src/genpb/"

56
scripts/sync-protos.sh Executable file
View File

@ -0,0 +1,56 @@
#!/usr/bin/env bash
#
# Sync the canonical .proto files from the waymaker server repo into this
# client repo's proto/ directory.
#
# The waymaker server repo is the SOURCE OF TRUTH for the wire contract.
# This repo vendors copies so the Go / TS / Rust clients can each generate
# stubs without depending on the server's Cargo workspace. Run this whenever
# the server's protos change, then re-run codegen and tag at the SAME version
# as the waymaker release (see VERSION).
#
# Usage:
# WAYMAKER_REPO=/path/to/waymaker ./scripts/sync-protos.sh
# Defaults to ../waymaker relative to this repo.
set -euo pipefail
here="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
WAYMAKER_REPO="${WAYMAKER_REPO:-$here/../waymaker}"
if [[ ! -d "$WAYMAKER_REPO/crates" ]]; then
echo "ERROR: waymaker repo not found at '$WAYMAKER_REPO'." >&2
echo " Set WAYMAKER_REPO=/path/to/waymaker and re-run." >&2
exit 1
fi
# canonical path in waymaker -> filename in proto/
# NOTE: crates/locks/proto/internal_proxy.proto (the server-internal
# node-to-node ProxyService + lease replication) is intentionally NOT
# vendored — it is a server concern, not part of any client.
protos=(
"crates/locks/proto/waymaker_locks.proto"
"crates/streams/proto/waymaker_streams.proto"
"crates/kv/proto/kv.proto"
"crates/collections/proto/collections.proto"
"crates/sketches/proto/sketches.proto"
"crates/cache/proto/cache.proto"
)
dest="$here/proto"
mkdir -p "$dest"
for rel in "${protos[@]}"; do
src="$WAYMAKER_REPO/$rel"
[[ -f "$src" ]] || { echo "ERROR: missing $src" >&2; exit 1; }
cp "$src" "$dest/$(basename "$rel")"
echo "synced $(basename "$rel")"
done
# Keep VERSION in lockstep with the waymaker workspace version.
wm_version="$(grep -m1 '^version' "$WAYMAKER_REPO/Cargo.toml" | sed -E 's/.*"([^"]+)".*/\1/')"
if [[ -n "$wm_version" ]]; then
echo "$wm_version" > "$here/VERSION"
echo "VERSION -> $wm_version (matched waymaker workspace)"
fi
echo "Done. Re-run codegen (scripts/gen-*.sh) and tag at v$(cat "$here/VERSION")."

261
ts/README.md Normal file
View File

@ -0,0 +1,261 @@
# @waymaker/client — TypeScript
Official TypeScript client for [waymaker](https://git.awesomike.com/dev/waymaker).
Full-parity with the Rust client across all subsystems.
## Installation
```
npm install @waymaker/client
```
Requires Node 18+ (for `crypto.randomUUID`, `Buffer`, async iterators).
## Quick start
```ts
import { WaymakerClient } from "@waymaker/client";
const client = WaymakerClient.connect("localhost:8818");
// ---- locks ----
import { Scope } from "@waymaker/client";
const lock = await client.acquireLock("leader:myjob", {
maxWaitMs: 0, // fail immediately if contended
leaseTtlMs: 60_000,
scope: Scope.Local,
});
const renewal = lock.spawnRenewal(30_000);
lock.on("change", (state) => {
if (state.lost) console.error("lock lost — fence token was", state.fenceToken);
});
try {
// Work can outlive a single lease window; renewal keeps it alive.
await doWork(lock.fenceToken());
} finally {
renewal.stop();
await lock.unlock();
}
client.close();
```
## Multi-node (cluster)
```ts
const client = WaymakerClient.connectMulti([
"node1:8818",
"node2:8828",
"node3:8838",
]);
```
grpc-js round-robins requests across the list and reroutes automatically
when an endpoint is unreachable.
## Subsystems
### Locks
```ts
import { Scope } from "@waymaker/client";
// Exclusive lock
const lock = await client.acquireLock("my-resource");
await lock.unlock();
// Shared lock
const rlock = await client.acquireReadLock("my-resource");
await rlock.unlock();
// Atomic multi-lock (deadlock-free: server sorts keys)
const leases = await client.multiLock([
{ key: "a", writeLock: true },
{ key: "b", writeLock: false },
]);
// Operator introspection
const held = await client.listAcquiredLocks("leader:");
```
**Leader-election pattern** (mirrors the waymaker-ctl renew loop):
```ts
const lock = await client.acquireLock("leader:batch", {
maxWaitMs: 0,
leaseTtlMs: 60_000,
scope: Scope.Local,
});
// Renewal runs independently of the work loop.
const renewal = lock.spawnRenewal(20_000);
lock.on("change", (s) => {
if (s.lost) process.exit(1); // give up leadership
});
try {
await runBatch();
} finally {
renewal.stop();
await lock.unlock();
}
```
### Streams
```ts
import { RetentionPolicy, DeliverPolicy } from "@waymaker/client";
const stream = await client.getOrCreateStream({
name: "events",
subjects: ["events.>"],
retention: { policy: RetentionPolicy.Limits, maxAgeMs: 7 * 86_400_000 },
replicationFactor: 3,
});
// Publish
const seq = await stream.publish("events.user.123", Buffer.from("hello"));
// Pull consumer
const consumer = await stream.getOrCreateConsumer({
name: "processor",
deliverPolicy: DeliverPolicy.All,
ackPolicy: "explicit",
});
const msgs = await consumer.fetch(10);
for (const msg of msgs) {
console.log(msg.subject, msg.data.toString());
await msg.ack();
}
// Push consumer (async iterator)
for await (const msg of consumer.messages()) {
await msg.ack();
}
```
### KV
```ts
const bucket = await client.getOrCreateKv({ name: "config", maxRevisions: 5 });
await bucket.put("db.host", "localhost");
const val = await bucket.get("db.host");
// CAS
const rev = await bucket.create("lock", Buffer.from("1"));
await bucket.update("lock", Buffer.from("2"), rev);
// Watch
for await (const ev of await bucket.watch("db.host")) {
if (ev.kind === "put") console.log("new value:", ev.value.toString());
}
```
### Collections
```ts
// Hash
const hs = await client.createHashStore({ name: "users" });
const h = hs.hash("user:42");
await h.set("email", "alice@example.com");
console.log(await h.get("email"));
// Set
const ss = await client.createSetStore({ name: "tags" });
const s = ss.set("article:1");
await s.add(Buffer.from("typescript"));
console.log(await s.members());
// Queue
const q = await client.createQueue({ name: "jobs" });
await q.push(Buffer.from(JSON.stringify({ type: "email" })));
const item = await q.pop();
```
### Sketches
```ts
// Bloom filter
const bloom = await client.bloomReserve("seen-ids", 1_000_000, 0.001);
await bloom.add(Buffer.from("msg-123"));
console.log(await bloom.exists(Buffer.from("msg-123"))); // true
// HyperLogLog
const hll = await client.hllReserve("unique-visitors");
await hll.add(Buffer.from("user-abc"));
console.log(await hll.count()); // approx cardinality
// Count-Min Sketch
const cms = await client.cmsReserve("event-counts", 0.001, 0.999);
await cms.add(Buffer.from("click"), 5);
console.log(await cms.count(Buffer.from("click")));
// Top-K
const topk = await client.topKReserve("hot-keys", 10);
await topk.add(Buffer.from("key-1"), 100);
console.log(await topk.list());
// t-digest
const td = await client.tdigestCreate("latency-p99", 200);
await td.add(42.5);
console.log(await td.quantile(0.99));
```
### Object store
```ts
const store = await client.getOrCreateObjectStore({
name: "artifacts",
maxBytes: 10 * 1024 * 1024 * 1024, // 10 GiB
});
const info = await store.put("report.pdf", pdfBytes);
console.log(info.sha256, info.totalBytes);
const { payload } = await store.get("report.pdf");
const entries = await store.list();
```
### Cache (stub)
The cache subsystem is wired but returns `unimplemented` until the
first eviction policy ships server-side. The client surface is
complete so callers can compile against it today:
```ts
await client.cacheAttachPolicy("my-bucket", "lru", { max_entries: "1000" });
await client.cacheDetachPolicy("my-bucket");
```
## TypeScript notes
- All `bytes` proto fields are typed as `Buffer`.
- `uint64` / `int64` proto fields are typed as `number`. Values beyond
`Number.MAX_SAFE_INTEGER` require `BigInt` — use `Long` from the
`long` package if you need exact 64-bit arithmetic.
- Streaming RPCs (`watch`, push consumers) return async iterables;
iterate with `for await`.
- The `Lock` class extends `EventEmitter`; subscribe with
`lock.on("change", handler)`.
- Strict TypeScript (`noImplicitAny`, `strictNullChecks`). No `any` leaks
through the public API surface.
## Regenerating proto stubs
The generated files in `src/genpb/` must not be hand-edited. Regenerate via:
```
bash ../scripts/gen-ts.sh
```
## Building from source
```
npm install
npm run build # emits to dist/
```

447
ts/package-lock.json generated Normal file
View File

@ -0,0 +1,447 @@
{
"name": "@waymaker/client",
"version": "0.1.27",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "@waymaker/client",
"version": "0.1.27",
"license": "MIT OR Apache-2.0",
"dependencies": {
"@grpc/grpc-js": "^1.12.0",
"long": "^5.2.3"
},
"devDependencies": {
"@types/node": "^22.0.0",
"ts-proto": "^2.6.0",
"typescript": "^5.6.0"
}
},
"node_modules/@bufbuild/protobuf": {
"version": "2.12.0",
"resolved": "https://registry.npmjs.org/@bufbuild/protobuf/-/protobuf-2.12.0.tgz",
"integrity": "sha512-B/XlCaFIP8LOwzo+bz5uFzATYokcwCKQcghqnlfwSmM5eX/qTkvDBnDPs+gXtX/RyjxJ4DRikECcPJbyALA8FA==",
"dev": true,
"license": "(Apache-2.0 AND BSD-3-Clause)"
},
"node_modules/@grpc/grpc-js": {
"version": "1.14.4",
"resolved": "https://registry.npmjs.org/@grpc/grpc-js/-/grpc-js-1.14.4.tgz",
"integrity": "sha512-k9Dj3DV/itK9D06Y8f190Qgop7/Ui+D0njFV3LHMPwPT75DpXLQohE9Wmz0QElrJnzsjB7KPWiKJbOl7IPDArQ==",
"license": "Apache-2.0",
"dependencies": {
"@grpc/proto-loader": "^0.8.0",
"@js-sdsl/ordered-map": "^4.4.2"
},
"engines": {
"node": ">=12.10.0"
}
},
"node_modules/@grpc/proto-loader": {
"version": "0.8.1",
"resolved": "https://registry.npmjs.org/@grpc/proto-loader/-/proto-loader-0.8.1.tgz",
"integrity": "sha512-wtF6h+DY6M3YaDBPAmvuuA6jV8Sif9MjtOI5euKFWRgCDl5PeDpPsHR9u2l6St5ceY8AZgoNDww5+HvEsXFsGg==",
"license": "Apache-2.0",
"dependencies": {
"lodash.camelcase": "^4.3.0",
"long": "^5.0.0",
"protobufjs": "^7.5.5",
"yargs": "^17.7.2"
},
"bin": {
"proto-loader-gen-types": "build/bin/proto-loader-gen-types.js"
},
"engines": {
"node": ">=6"
}
},
"node_modules/@js-sdsl/ordered-map": {
"version": "4.4.2",
"resolved": "https://registry.npmjs.org/@js-sdsl/ordered-map/-/ordered-map-4.4.2.tgz",
"integrity": "sha512-iUKgm52T8HOE/makSxjqoWhe95ZJA1/G1sYsGev2JDKUSS14KAgg1LHb+Ba+IPow0xflbnSkOsZcO08C7w1gYw==",
"license": "MIT",
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/js-sdsl"
}
},
"node_modules/@protobufjs/aspromise": {
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/@protobufjs/aspromise/-/aspromise-1.1.2.tgz",
"integrity": "sha512-j+gKExEuLmKwvz3OgROXtrJ2UG2x8Ch2YZUxahh+s1F2HZ+wAceUNLkvy6zKCPVRkU++ZWQrdxsUeQXmcg4uoQ==",
"license": "BSD-3-Clause"
},
"node_modules/@protobufjs/base64": {
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/@protobufjs/base64/-/base64-1.1.2.tgz",
"integrity": "sha512-AZkcAA5vnN/v4PDqKyMR5lx7hZttPDgClv83E//FMNhR2TMcLUhfRUBHCmSl0oi9zMgDDqRUJkSxO3wm85+XLg==",
"license": "BSD-3-Clause"
},
"node_modules/@protobufjs/codegen": {
"version": "2.0.5",
"resolved": "https://registry.npmjs.org/@protobufjs/codegen/-/codegen-2.0.5.tgz",
"integrity": "sha512-zgXFLzW3Ap33e6d0Wlj4MGIm6Ce8O89n/apUaGNB/jx+hw+ruWEp7EwGUshdLKVRCxZW12fp9r40E1mQrf/34g==",
"license": "BSD-3-Clause"
},
"node_modules/@protobufjs/eventemitter": {
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/@protobufjs/eventemitter/-/eventemitter-1.1.1.tgz",
"integrity": "sha512-vW1GmwMZNnL+gMRaovlh9yZX74kc+TTU3FObkkurpMaRtBfLP3ldjS9KQWlwZgraRE0+dheEEoAxdzcJQ8eXZg==",
"license": "BSD-3-Clause"
},
"node_modules/@protobufjs/fetch": {
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/@protobufjs/fetch/-/fetch-1.1.1.tgz",
"integrity": "sha512-GpptLrs57adMSuHi3VNj0mAF8dwh36LMaYF6XyJ6JMWlVsc+t42tm1HSEDmOs3A8fC9yyeisgLhsTVQokOZ0zw==",
"license": "BSD-3-Clause",
"dependencies": {
"@protobufjs/aspromise": "^1.1.1"
}
},
"node_modules/@protobufjs/float": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/@protobufjs/float/-/float-1.0.2.tgz",
"integrity": "sha512-Ddb+kVXlXst9d+R9PfTIxh1EdNkgoRe5tOX6t01f1lYWOvJnSPDBlG241QLzcyPdoNTsblLUdujGSE4RzrTZGQ==",
"license": "BSD-3-Clause"
},
"node_modules/@protobufjs/inquire": {
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/@protobufjs/inquire/-/inquire-1.1.2.tgz",
"integrity": "sha512-pa0vFRuws4wkvaXKK1uXZMAwAX4/t8ANaJo45iw/oQHNQ9q5xUzwgFmVJGXiga2BeN+zpX7Vf9vmsiIa2J+MUw==",
"license": "BSD-3-Clause"
},
"node_modules/@protobufjs/path": {
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/@protobufjs/path/-/path-1.1.2.tgz",
"integrity": "sha512-6JOcJ5Tm08dOHAbdR3GrvP+yUUfkjG5ePsHYczMFLq3ZmMkAD98cDgcT2iA1lJ9NVwFd4tH/iSSoe44YWkltEA==",
"license": "BSD-3-Clause"
},
"node_modules/@protobufjs/pool": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/@protobufjs/pool/-/pool-1.1.0.tgz",
"integrity": "sha512-0kELaGSIDBKvcgS4zkjz1PeddatrjYcmMWOlAuAPwAeccUrPHdUqo/J6LiymHHEiJT5NrF1UVwxY14f+fy4WQw==",
"license": "BSD-3-Clause"
},
"node_modules/@protobufjs/utf8": {
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/@protobufjs/utf8/-/utf8-1.1.1.tgz",
"integrity": "sha512-oOAWABowe8EAbMyWKM0tYDKi8Yaox52D+HWZhAIJqQXbqe0xI/GV7FhLWqlEKreMkfDjshR5FKgi3mnle0h6Eg==",
"license": "BSD-3-Clause"
},
"node_modules/@types/node": {
"version": "22.19.20",
"resolved": "https://registry.npmjs.org/@types/node/-/node-22.19.20.tgz",
"integrity": "sha512-6tELRwSDYWW9EdZhbeZmYGZ1/7Djkt+Ah3/ScEYT9cDord7UJzasR/4D3VONg9tQI5CDp+/CZC1AXj2pCFOvpw==",
"license": "MIT",
"dependencies": {
"undici-types": "~6.21.0"
}
},
"node_modules/ansi-regex": {
"version": "5.0.1",
"resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-5.0.1.tgz",
"integrity": "sha512-quJQXlTSUGL2LH9SUXo8VwsY4soanhgo6LNSm84E1LBcE8s3O0wpdiRzyR9z/ZZJMlMWv37qOOb9pdJlMUEKFQ==",
"license": "MIT",
"engines": {
"node": ">=8"
}
},
"node_modules/ansi-styles": {
"version": "4.3.0",
"resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-4.3.0.tgz",
"integrity": "sha512-zbB9rCJAT1rbjiVDb2hqKFHNYLxgtk8NURxZ3IZwD3F6NtxbXZQCnnSi1Lkx+IDohdPlFp222wVALIheZJQSEg==",
"license": "MIT",
"dependencies": {
"color-convert": "^2.0.1"
},
"engines": {
"node": ">=8"
},
"funding": {
"url": "https://github.com/chalk/ansi-styles?sponsor=1"
}
},
"node_modules/case-anything": {
"version": "2.1.13",
"resolved": "https://registry.npmjs.org/case-anything/-/case-anything-2.1.13.tgz",
"integrity": "sha512-zlOQ80VrQ2Ue+ymH5OuM/DlDq64mEm+B9UTdHULv5osUMD6HalNTblf2b1u/m6QecjsnOkBpqVZ+XPwIVsy7Ng==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=12.13"
},
"funding": {
"url": "https://github.com/sponsors/mesqueeb"
}
},
"node_modules/cliui": {
"version": "8.0.1",
"resolved": "https://registry.npmjs.org/cliui/-/cliui-8.0.1.tgz",
"integrity": "sha512-BSeNnyus75C4//NQ9gQt1/csTXyo/8Sb+afLAkzAptFuMsod9HFokGNudZpi/oQV73hnVK+sR+5PVRMd+Dr7YQ==",
"license": "ISC",
"dependencies": {
"string-width": "^4.2.0",
"strip-ansi": "^6.0.1",
"wrap-ansi": "^7.0.0"
},
"engines": {
"node": ">=12"
}
},
"node_modules/color-convert": {
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/color-convert/-/color-convert-2.0.1.tgz",
"integrity": "sha512-RRECPsj7iu/xb5oKYcsFHSppFNnsj/52OVTRKb4zP5onXwVF3zVmmToNcOfGC+CRDpfK/U584fMg38ZHCaElKQ==",
"license": "MIT",
"dependencies": {
"color-name": "~1.1.4"
},
"engines": {
"node": ">=7.0.0"
}
},
"node_modules/color-name": {
"version": "1.1.4",
"resolved": "https://registry.npmjs.org/color-name/-/color-name-1.1.4.tgz",
"integrity": "sha512-dOy+3AuW3a2wNbZHIuMZpTcgjGuLU/uBL/ubcZF9OXbDo8ff4O8yVp5Bf0efS8uEoYo5q4Fx7dY9OgQGXgAsQA==",
"license": "MIT"
},
"node_modules/detect-libc": {
"version": "1.0.3",
"resolved": "https://registry.npmjs.org/detect-libc/-/detect-libc-1.0.3.tgz",
"integrity": "sha512-pGjwhsmsp4kL2RTz08wcOlGN83otlqHeD/Z5T8GXZB+/YcpQ/dgo+lbU8ZsGxV0HIvqqxo9l7mqYwyYMD9bKDg==",
"dev": true,
"license": "Apache-2.0",
"bin": {
"detect-libc": "bin/detect-libc.js"
},
"engines": {
"node": ">=0.10"
}
},
"node_modules/dprint-node": {
"version": "1.0.8",
"resolved": "https://registry.npmjs.org/dprint-node/-/dprint-node-1.0.8.tgz",
"integrity": "sha512-iVKnUtYfGrYcW1ZAlfR/F59cUVL8QIhWoBJoSjkkdua/dkWIgjZfiLMeTjiB06X0ZLkQ0M2C1VbUj/CxkIf1zg==",
"dev": true,
"license": "MIT",
"dependencies": {
"detect-libc": "^1.0.3"
}
},
"node_modules/emoji-regex": {
"version": "8.0.0",
"resolved": "https://registry.npmjs.org/emoji-regex/-/emoji-regex-8.0.0.tgz",
"integrity": "sha512-MSjYzcWNOA0ewAHpz0MxpYFvwg6yjy1NG3xteoqz644VCo/RPgnr1/GGt+ic3iJTzQ8Eu3TdM14SawnVUmGE6A==",
"license": "MIT"
},
"node_modules/escalade": {
"version": "3.2.0",
"resolved": "https://registry.npmjs.org/escalade/-/escalade-3.2.0.tgz",
"integrity": "sha512-WUj2qlxaQtO4g6Pq5c29GTcWGDyd8itL8zTlipgECz3JesAiiOKotd8JU6otB3PACgG6xkJUyVhboMS+bje/jA==",
"license": "MIT",
"engines": {
"node": ">=6"
}
},
"node_modules/get-caller-file": {
"version": "2.0.5",
"resolved": "https://registry.npmjs.org/get-caller-file/-/get-caller-file-2.0.5.tgz",
"integrity": "sha512-DyFP3BM/3YHTQOCUL/w0OZHR0lpKeGrxotcHWcqNEdnltqFwXVfhEBQ94eIo34AfQpo0rGki4cyIiftY06h2Fg==",
"license": "ISC",
"engines": {
"node": "6.* || 8.* || >= 10.*"
}
},
"node_modules/is-fullwidth-code-point": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/is-fullwidth-code-point/-/is-fullwidth-code-point-3.0.0.tgz",
"integrity": "sha512-zymm5+u+sCsSWyD9qNaejV3DFvhCKclKdizYaJUuHA83RLjb7nSuGnddCHGv0hk+KY7BMAlsWeK4Ueg6EV6XQg==",
"license": "MIT",
"engines": {
"node": ">=8"
}
},
"node_modules/lodash.camelcase": {
"version": "4.3.0",
"resolved": "https://registry.npmjs.org/lodash.camelcase/-/lodash.camelcase-4.3.0.tgz",
"integrity": "sha512-TwuEnCnxbc3rAvhf/LbG7tJUDzhqXyFnv3dtzLOPgCG/hODL7WFnsbwktkD7yUV0RrreP/l1PALq/YSg6VvjlA==",
"license": "MIT"
},
"node_modules/long": {
"version": "5.3.2",
"resolved": "https://registry.npmjs.org/long/-/long-5.3.2.tgz",
"integrity": "sha512-mNAgZ1GmyNhD7AuqnTG3/VQ26o760+ZYBPKjPvugO8+nLbYfX6TVpJPseBvopbdY+qpZ/lKUnmEc1LeZYS3QAA==",
"license": "Apache-2.0"
},
"node_modules/protobufjs": {
"version": "7.6.2",
"resolved": "https://registry.npmjs.org/protobufjs/-/protobufjs-7.6.2.tgz",
"integrity": "sha512-N9EiLovGEQOJSPF26Ij7qUGvahfEnq0eeYZ02aigIedkmz1qZSwjnP9SBITHJuF/6MYbIW4HDN8zdYjsjqJKXQ==",
"hasInstallScript": true,
"license": "BSD-3-Clause",
"dependencies": {
"@protobufjs/aspromise": "^1.1.2",
"@protobufjs/base64": "^1.1.2",
"@protobufjs/codegen": "^2.0.5",
"@protobufjs/eventemitter": "^1.1.1",
"@protobufjs/fetch": "^1.1.1",
"@protobufjs/float": "^1.0.2",
"@protobufjs/inquire": "^1.1.2",
"@protobufjs/path": "^1.1.2",
"@protobufjs/pool": "^1.1.0",
"@protobufjs/utf8": "^1.1.1",
"@types/node": ">=13.7.0",
"long": "^5.3.2"
},
"engines": {
"node": ">=12.0.0"
}
},
"node_modules/require-directory": {
"version": "2.1.1",
"resolved": "https://registry.npmjs.org/require-directory/-/require-directory-2.1.1.tgz",
"integrity": "sha512-fGxEI7+wsG9xrvdjsrlmL22OMTTiHRwAMroiEeMgq8gzoLC/PQr7RsRDSTLUg/bZAZtF+TVIkHc6/4RIKrui+Q==",
"license": "MIT",
"engines": {
"node": ">=0.10.0"
}
},
"node_modules/string-width": {
"version": "4.2.3",
"resolved": "https://registry.npmjs.org/string-width/-/string-width-4.2.3.tgz",
"integrity": "sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g==",
"license": "MIT",
"dependencies": {
"emoji-regex": "^8.0.0",
"is-fullwidth-code-point": "^3.0.0",
"strip-ansi": "^6.0.1"
},
"engines": {
"node": ">=8"
}
},
"node_modules/strip-ansi": {
"version": "6.0.1",
"resolved": "https://registry.npmjs.org/strip-ansi/-/strip-ansi-6.0.1.tgz",
"integrity": "sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A==",
"license": "MIT",
"dependencies": {
"ansi-regex": "^5.0.1"
},
"engines": {
"node": ">=8"
}
},
"node_modules/ts-poet": {
"version": "6.12.0",
"resolved": "https://registry.npmjs.org/ts-poet/-/ts-poet-6.12.0.tgz",
"integrity": "sha512-xo+iRNMWqyvXpFTaOAvLPA5QAWO6TZrSUs5s4Odaya3epqofBu/fMLHEWl8jPmjhA0s9sgj9sNvF1BmaQlmQkA==",
"dev": true,
"license": "Apache-2.0",
"dependencies": {
"dprint-node": "^1.0.8"
}
},
"node_modules/ts-proto": {
"version": "2.11.8",
"resolved": "https://registry.npmjs.org/ts-proto/-/ts-proto-2.11.8.tgz",
"integrity": "sha512-+5hzECnyVB33jxjG1BIdzAHcRBm7hjnm8womdJVp2A7xJWihP0drHHVsXYTr9i/LpWNGfh80I+AVVNzFM5AwJw==",
"dev": true,
"license": "ISC",
"dependencies": {
"@bufbuild/protobuf": "^2.10.2",
"case-anything": "^2.1.13",
"ts-poet": "^6.12.0",
"ts-proto-descriptors": "2.1.0"
},
"bin": {
"protoc-gen-ts_proto": "protoc-gen-ts_proto"
}
},
"node_modules/ts-proto-descriptors": {
"version": "2.1.0",
"resolved": "https://registry.npmjs.org/ts-proto-descriptors/-/ts-proto-descriptors-2.1.0.tgz",
"integrity": "sha512-S5EZYEQ6L9KLFfjSRpZWDIXDV/W7tAj8uW7pLsihIxyr62EAVSiKuVPwE8iWnr849Bqa53enex1jhDUcpgquzA==",
"dev": true,
"license": "ISC",
"dependencies": {
"@bufbuild/protobuf": "^2.0.0"
}
},
"node_modules/typescript": {
"version": "5.9.3",
"resolved": "https://registry.npmjs.org/typescript/-/typescript-5.9.3.tgz",
"integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==",
"dev": true,
"license": "Apache-2.0",
"bin": {
"tsc": "bin/tsc",
"tsserver": "bin/tsserver"
},
"engines": {
"node": ">=14.17"
}
},
"node_modules/undici-types": {
"version": "6.21.0",
"resolved": "https://registry.npmjs.org/undici-types/-/undici-types-6.21.0.tgz",
"integrity": "sha512-iwDZqg0QAGrg9Rav5H4n0M64c3mkR59cJ6wQp+7C4nI0gsmExaedaYLNO44eT4AtBBwjbTiGPMlt2Md0T9H9JQ==",
"license": "MIT"
},
"node_modules/wrap-ansi": {
"version": "7.0.0",
"resolved": "https://registry.npmjs.org/wrap-ansi/-/wrap-ansi-7.0.0.tgz",
"integrity": "sha512-YVGIj2kamLSTxw6NsZjoBxfSwsn0ycdesmc4p+Q21c5zPuZ1pl+NfxVdxPtdHvmNVOQ6XSYG4AUtyt/Fi7D16Q==",
"license": "MIT",
"dependencies": {
"ansi-styles": "^4.0.0",
"string-width": "^4.1.0",
"strip-ansi": "^6.0.0"
},
"engines": {
"node": ">=10"
},
"funding": {
"url": "https://github.com/chalk/wrap-ansi?sponsor=1"
}
},
"node_modules/y18n": {
"version": "5.0.8",
"resolved": "https://registry.npmjs.org/y18n/-/y18n-5.0.8.tgz",
"integrity": "sha512-0pfFzegeDWJHJIAmTLRP2DwHjdF5s7jo9tuztdQxAhINCdvS+3nGINqPd00AphqJR/0LhANUS6/+7SCb98YOfA==",
"license": "ISC",
"engines": {
"node": ">=10"
}
},
"node_modules/yargs": {
"version": "17.7.2",
"resolved": "https://registry.npmjs.org/yargs/-/yargs-17.7.2.tgz",
"integrity": "sha512-7dSzzRQ++CKnNI/krKnYRV7JKKPUXMEh61soaHKg9mrWEhzFWhFnxPxGl+69cD1Ou63C13NUPCnmIcrvqCuM6w==",
"license": "MIT",
"dependencies": {
"cliui": "^8.0.1",
"escalade": "^3.1.1",
"get-caller-file": "^2.0.5",
"require-directory": "^2.1.1",
"string-width": "^4.2.3",
"y18n": "^5.0.5",
"yargs-parser": "^21.1.1"
},
"engines": {
"node": ">=12"
}
},
"node_modules/yargs-parser": {
"version": "21.1.1",
"resolved": "https://registry.npmjs.org/yargs-parser/-/yargs-parser-21.1.1.tgz",
"integrity": "sha512-tVpsJW7DdjecAiFpbIB1e3qxIQsE6NoPc5/eTdrbbIC4h0LVsWhnoa3g+m2HclBIujHzsxZ4VJVA+GUuc2/LBw==",
"license": "ISC",
"engines": {
"node": ">=12"
}
}
}
}

25
ts/package.json Normal file
View File

@ -0,0 +1,25 @@
{
"name": "@waymaker/client",
"version": "0.1.27",
"description": "Official TypeScript client for waymaker — locks, streams, KV, collections, sketches, cache, object store",
"repository": "https://git.awesomike.com/pub/waymaker-client",
"license": "MIT OR Apache-2.0",
"type": "commonjs",
"main": "dist/index.js",
"types": "dist/index.d.ts",
"files": ["dist", "src"],
"scripts": {
"gen": "../scripts/gen-ts.sh",
"build": "tsc -p tsconfig.json",
"prepublishOnly": "npm run build"
},
"dependencies": {
"@grpc/grpc-js": "^1.12.0",
"long": "^5.2.3"
},
"devDependencies": {
"ts-proto": "^2.6.0",
"typescript": "^5.6.0",
"@types/node": "^22.0.0"
}
}

72
ts/src/cache.ts Normal file
View File

@ -0,0 +1,72 @@
/**
* Cache subsystem TTL/eviction policy service.
*
* The server-side implementation is a stub; all calls return
* `unimplemented`. The client surface is complete so callers can
* import and compile against it and get real responses once the
* server ships the implementation.
*
* Entry points on `WaymakerClient`:
* - `client.cacheAttachPolicy(streamName, policyType, params)`
* - `client.cacheDetachPolicy(streamName)`
*/
import { WaymakerClient, callUnary } from "./client";
import { serverError } from "./error";
declare module "./client" {
interface WaymakerClient {
/**
* Attach a TTL/eviction policy to a bucket.
* Server returns `unimplemented` until the first concrete policy lands.
* `policyId` is the policy kind (e.g. `"lru"`); `params` are
* policy-specific knobs (e.g. `{ max_entries: "1000" }`).
*/
cacheAttachPolicy(
bucket: string,
policyId: string,
params?: Record<string, string>
): Promise<void>;
/**
* Detach a TTL/eviction policy from a bucket.
* Server returns `unimplemented` until the first concrete policy lands.
*/
cacheDetachPolicy(bucket: string): Promise<void>;
}
}
WaymakerClient.prototype.cacheAttachPolicy = async function (
this: WaymakerClient,
bucket: string,
policyId: string,
params: Record<string, string> = {}
): Promise<void> {
const c = this._cacheClient();
try {
const res = await callUnary(c, (req, cb) => c.attachPolicy(req, cb), {
bucket,
policyId,
params,
});
// res may be unknown if server is unimplemented — cast defensively.
const r = res as { success?: boolean; resultCode?: string; message?: string };
if (r.success === false) throw serverError(r.resultCode ?? "error", r.message ?? "");
} finally {
c.close();
}
};
WaymakerClient.prototype.cacheDetachPolicy = async function (
this: WaymakerClient,
bucket: string
): Promise<void> {
const c = this._cacheClient();
try {
const res = await callUnary(c, (req, cb) => c.detachPolicy(req, cb), { bucket });
const r = res as { success?: boolean; resultCode?: string; message?: string };
if (r.success === false) throw serverError(r.resultCode ?? "error", r.message ?? "");
} finally {
c.close();
}
};

226
ts/src/client.ts Normal file
View File

@ -0,0 +1,226 @@
/**
* Core connection handle.
*
* `WaymakerClient` holds the gRPC channel and exposes per-subsystem
* service clients. It is safe to share across async tasks the
* underlying grpc-js `Channel` is reference-counted.
*
* ## Single-host
* ```ts
* const client = await WaymakerClient.connect("localhost:8818");
* ```
*
* ## Multi-host (HA, round-robin)
* ```ts
* const client = WaymakerClient.connectMulti([
* "localhost:8818",
* "localhost:8828",
* "localhost:8838",
* ]);
* ```
*/
import * as grpc from "@grpc/grpc-js";
import { WaymakerServiceClient } from "./genpb/waymaker_locks";
import { WaymakerStreamsServiceClient } from "./genpb/waymaker_streams";
import { WaymakerKvServiceClient } from "./genpb/kv";
import { WaymakerCollectionsServiceClient } from "./genpb/collections";
import { WaymakerSketchesServiceClient } from "./genpb/sketches";
import { WaymakerCacheServiceClient } from "./genpb/cache";
import { invalidError, rpcError, serverError } from "./error";
export type { WaymakerError } from "./error";
export { isServerError, isWaymakerError } from "./error";
/** Options for `WaymakerClient.connect`. */
export interface ConnectOptions {
/** Channel credentials. Defaults to `grpc.credentials.createInsecure()`. */
credentials?: grpc.ChannelCredentials;
/** Extra gRPC channel options. */
channelOptions?: grpc.ChannelOptions;
}
/**
* A connected waymaker client. Cheap to pass around the underlying
* channel is shared, not copied.
*/
export class WaymakerClient {
readonly channel: grpc.Channel;
private constructor(channel: grpc.Channel) {
this.channel = channel;
}
/**
* Connect to a single waymaker server.
*
* `address` must be `"host:port"` (no scheme).
* Returns immediately; the channel connects lazily on the first RPC.
*/
static connect(address: string, opts: ConnectOptions = {}): WaymakerClient {
const creds = opts.credentials ?? grpc.credentials.createInsecure();
const ch = new grpc.Channel(address, creds, opts.channelOptions ?? {});
return new WaymakerClient(ch);
}
/**
* Connect to multiple waymaker servers. grpc-js round-robins
* requests across the list and automatically reroutes when one
* endpoint is unreachable the right shape for a clustered
* deployment.
*
* Each entry in `addresses` must be `"host:port"`.
*/
static connectMulti(addresses: string[], opts: ConnectOptions = {}): WaymakerClient {
if (addresses.length === 0) {
throw invalidError("connectMulti requires at least one address");
}
const creds = opts.credentials ?? grpc.credentials.createInsecure();
// grpc-js supports a comma-joined list in a single Channel address
// string with the round_robin load-balancing policy.
const joined = addresses.map((a) => (a.startsWith("dns://") ? a : `dns:///${a}`)).join(",");
const ch = new grpc.Channel(joined, creds, {
"grpc.lb_policy_name": "round_robin",
...(opts.channelOptions ?? {}),
});
return new WaymakerClient(ch);
}
/** Close the underlying channel and free resources. */
close(): void {
this.channel.close();
}
// ------------------------------------------------------------------
// Per-subsystem gRPC client constructors. Each is constructed
// per-call so individual RPCs pick up the channel's current routing
// state (equivalent to the Rust "lazy per-call client" pattern).
// ------------------------------------------------------------------
/** @internal */
_locksClient(): WaymakerServiceClient {
return new WaymakerServiceClient("", grpc.credentials.createInsecure(), {
channelOverride: this.channel,
});
}
/** @internal */
_streamsClient(): WaymakerStreamsServiceClient {
return new WaymakerStreamsServiceClient("", grpc.credentials.createInsecure(), {
channelOverride: this.channel,
});
}
/** @internal */
_kvClient(): WaymakerKvServiceClient {
return new WaymakerKvServiceClient("", grpc.credentials.createInsecure(), {
channelOverride: this.channel,
});
}
/** @internal */
_collectionsClient(): WaymakerCollectionsServiceClient {
return new WaymakerCollectionsServiceClient("", grpc.credentials.createInsecure(), {
channelOverride: this.channel,
});
}
/** @internal */
_sketchesClient(): WaymakerSketchesServiceClient {
return new WaymakerSketchesServiceClient("", grpc.credentials.createInsecure(), {
channelOverride: this.channel,
});
}
/** @internal */
_cacheClient(): WaymakerCacheServiceClient {
return new WaymakerCacheServiceClient("", grpc.credentials.createInsecure(), {
channelOverride: this.channel,
});
}
}
// ------------------------------------------------------------------
// Helpers used throughout the client codebase.
// ------------------------------------------------------------------
/**
* Wrap a grpc-js unary callback-style call in a Promise.
*
* Pass the method as a bound arrow so that grpc-js receives `this`
* correctly:
* `callUnary(c, (req, cb) => c.someRpc(req, cb), { ...requestFields })`
*
* The generic `Res` parameter is used to type the resolved value.
* Because grpc-js stubs define every RPC with three overloads (no meta /
* with meta / with meta+options), TypeScript cannot infer `Res` from
* contextual typing of the method lambda. Callers must therefore supply
* the response type explicitly:
* `callUnary<HashSetResponse>(c, (req, cb) => c.hashSet(req, cb), req)`
*
* To avoid that boilerplate everywhere, prefer the `callRpc` helper which
* binds the method and infers the response type automatically.
*/
// eslint-disable-next-line @typescript-eslint/no-explicit-any
export function callUnary<Res = any>(
_client: grpc.Client,
// eslint-disable-next-line @typescript-eslint/no-explicit-any
method: (req: any, cb: (err: grpc.ServiceError | null, res: Res) => void) => grpc.ClientUnaryCall,
// eslint-disable-next-line @typescript-eslint/no-explicit-any
req: any
): Promise<Res> {
return new Promise<Res>((resolve, reject) => {
method(req, (err, res) => {
if (err) {
reject(rpcError(err));
} else {
resolve(res as Res);
}
});
});
}
/**
* Read all events from a grpc-js `ClientReadableStream<T>` into an
* async generator. Each `yield` fires as soon as a message arrives;
* the generator ends when the stream closes (or rejects on error).
*/
export async function* streamToAsyncIter<T>(
stream: grpc.ClientReadableStream<T>
): AsyncGenerator<T> {
const queue: T[] = [];
let ended = false;
let error: Error | null = null;
let resolve: (() => void) | null = null;
stream.on("data", (msg: T) => {
queue.push(msg);
resolve?.();
resolve = null;
});
stream.on("end", () => {
ended = true;
resolve?.();
resolve = null;
});
stream.on("error", (err: Error) => {
error = err;
resolve?.();
resolve = null;
});
while (true) {
while (queue.length > 0) {
yield queue.shift() as T;
}
if (error) throw rpcError(error as { message: string; code?: number });
if (ended) return;
await new Promise<void>((res) => {
resolve = res;
});
}
}
// Re-export error helpers so callers can import them from the main
// package without reaching into the error module.
export { serverError, rpcError, invalidError };

542
ts/src/collections.ts Normal file
View File

@ -0,0 +1,542 @@
/**
* Collections subsystem Redis-shape Hash / Set / Queue types.
*
* Entry points on `WaymakerClient`:
* - `client.createHashStore(config)` / `getOrCreateHashStore` / `deleteHashStore`
* - `client.createSetStore(config)` / `getOrCreateSetStore` / `deleteSetStore`
* - `client.createQueue(config)` / `getOrCreateQueue` / `deleteQueue`
* - `client.hashStore(name)` / `setStore(name)` / `queue(name)` handles without creation
*/
import { WaymakerClient, callUnary } from "./client";
import { serverError } from "./error";
// ------------------------------------------------------------------
// Hash
// ------------------------------------------------------------------
export interface HashStoreConfig {
name: string;
maxBytes?: number;
ephemeral?: boolean;
}
export class HashStore {
readonly name: string;
/** @internal */
constructor(private readonly _client: WaymakerClient, name: string) {
this.name = name;
}
/** Return a `Hash` handle for `hashKey` within this store. */
hash(hashKey: string): Hash {
return new Hash(this._client, this.name, hashKey);
}
}
export class Hash {
/** @internal */
constructor(
private readonly _client: WaymakerClient,
private readonly _bucket: string,
readonly hashKey: string
) {}
/** Set `field` = `value`. Returns the new revision. */
async set(field: string, value: Uint8Array | Buffer | string): Promise<number> {
const c = this._client._collectionsClient();
try {
const res = await callUnary(c, (req, cb) => c.hashSet(req, cb), {
bucket: this._bucket,
hashKey: this.hashKey,
field,
value: toBuffer(value),
});
if (!res.success) throw serverError(res.resultCode, res.message);
return res.revision;
} finally {
c.close();
}
}
/** Get value for `field`. `null` when absent. */
async get(field: string): Promise<Uint8Array | null> {
const c = this._client._collectionsClient();
try {
const res = await callUnary(c, (req, cb) => c.hashGet(req, cb), {
bucket: this._bucket,
hashKey: this.hashKey,
field,
});
if (!res.success) throw serverError(res.resultCode, res.message);
return res.value ?? null;
} finally {
c.close();
}
}
/** Check if `field` exists. */
async exists(field: string): Promise<boolean> {
const c = this._client._collectionsClient();
try {
const res = await callUnary(c, (req, cb) => c.hashExists(req, cb), {
bucket: this._bucket,
hashKey: this.hashKey,
field,
});
if (!res.success) throw serverError(res.resultCode, res.message);
return res.exists;
} finally {
c.close();
}
}
/** Delete `field`. */
async deleteField(field: string): Promise<void> {
const c = this._client._collectionsClient();
try {
const res = await callUnary(c, (req, cb) => c.hashDelete(req, cb), {
bucket: this._bucket,
hashKey: this.hashKey,
field,
});
if (!res.success) throw serverError(res.resultCode, res.message);
} finally {
c.close();
}
}
/** List all field names. */
async fields(): Promise<string[]> {
const c = this._client._collectionsClient();
try {
const res = await callUnary(c, (req, cb) => c.hashFields(req, cb), {
bucket: this._bucket,
hashKey: this.hashKey,
});
if (!res.success) throw serverError(res.resultCode, res.message);
return res.fields;
} finally {
c.close();
}
}
/** Number of fields. */
async len(): Promise<number> {
const c = this._client._collectionsClient();
try {
const res = await callUnary(c, (req, cb) => c.hashLen(req, cb), {
bucket: this._bucket,
hashKey: this.hashKey,
});
if (!res.success) throw serverError(res.resultCode, res.message);
return res.count;
} finally {
c.close();
}
}
/** Get all field→value pairs. */
async getAll(): Promise<Map<string, Uint8Array>> {
const c = this._client._collectionsClient();
try {
const res = await callUnary(c, (req, cb) => c.hashGetAll(req, cb), {
bucket: this._bucket,
hashKey: this.hashKey,
});
if (!res.success) throw serverError(res.resultCode, res.message);
const map = new Map<string, Uint8Array>();
for (const e of res.entries) map.set(e.field, e.value);
return map;
} finally {
c.close();
}
}
}
// ------------------------------------------------------------------
// Set
// ------------------------------------------------------------------
export interface SetStoreConfig {
name: string;
maxBytes?: number;
ephemeral?: boolean;
}
export class SetStore {
readonly name: string;
/** @internal */
constructor(private readonly _client: WaymakerClient, name: string) {
this.name = name;
}
/** Return a `Set` handle for `setKey` within this store. */
set(setKey: string): SetHandle {
return new SetHandle(this._client, this.name, setKey);
}
}
export class SetHandle {
/** @internal */
constructor(
private readonly _client: WaymakerClient,
private readonly _bucket: string,
readonly setKey: string
) {}
/** Add `member`. */
async add(member: string): Promise<void> {
const c = this._client._collectionsClient();
try {
const res = await callUnary(c, (req, cb) => c.setAdd(req, cb), {
bucket: this._bucket,
setKey: this.setKey,
member,
});
if (!res.success) throw serverError(res.resultCode, res.message);
} finally {
c.close();
}
}
/** Remove `member`. */
async remove(member: string): Promise<void> {
const c = this._client._collectionsClient();
try {
const res = await callUnary(c, (req, cb) => c.setRemove(req, cb), {
bucket: this._bucket,
setKey: this.setKey,
member,
});
if (!res.success) throw serverError(res.resultCode, res.message);
} finally {
c.close();
}
}
/** Test membership. */
async isMember(member: string): Promise<boolean> {
const c = this._client._collectionsClient();
try {
const res = await callUnary(c, (req, cb) => c.setIsMember(req, cb), {
bucket: this._bucket,
setKey: this.setKey,
member,
});
if (!res.success) throw serverError(res.resultCode, res.message);
return res.isMember;
} finally {
c.close();
}
}
/** List all members. */
async members(): Promise<string[]> {
const c = this._client._collectionsClient();
try {
const res = await callUnary(c, (req, cb) => c.setMembers(req, cb), {
bucket: this._bucket,
setKey: this.setKey,
});
if (!res.success) throw serverError(res.resultCode, res.message);
return res.members;
} finally {
c.close();
}
}
/** Cardinality. */
async len(): Promise<number> {
const c = this._client._collectionsClient();
try {
const res = await callUnary(c, (req, cb) => c.setLen(req, cb), {
bucket: this._bucket,
setKey: this.setKey,
});
if (!res.success) throw serverError(res.resultCode, res.message);
return res.count;
} finally {
c.close();
}
}
}
// ------------------------------------------------------------------
// Queue
// ------------------------------------------------------------------
export interface QueueConfig {
name: string;
maxBytes?: number;
maxMessages?: number;
ephemeral?: boolean;
}
export class Queue {
readonly name: string;
/** @internal */
constructor(private readonly _client: WaymakerClient, name: string) {
this.name = name;
}
/** Push `value` to the tail. Returns the new sequence. */
async push(value: Uint8Array | Buffer | string): Promise<number> {
const c = this._client._collectionsClient();
try {
const res = await callUnary(c, (req, cb) => c.queuePush(req, cb), {
bucket: this.name,
value: toBuffer(value),
});
if (!res.success) throw serverError(res.resultCode, res.message);
return res.sequence;
} finally {
c.close();
}
}
/** Pop from the head. Returns `null` when empty. */
async pop(): Promise<Uint8Array | null> {
const c = this._client._collectionsClient();
try {
const res = await callUnary(c, (req, cb) => c.queuePop(req, cb), {
bucket: this.name,
});
if (!res.success) throw serverError(res.resultCode, res.message);
return res.value ?? null;
} finally {
c.close();
}
}
/** Read a range without consuming. */
async range(from: number, limit: number): Promise<Uint8Array[]> {
const c = this._client._collectionsClient();
try {
const res = await callUnary(c, (req, cb) => c.queueRange(req, cb), {
bucket: this.name,
fromSequence: from,
limit,
});
if (!res.success) throw serverError(res.resultCode, res.message);
return res.values;
} finally {
c.close();
}
}
/** Queue depth. */
async len(): Promise<number> {
const c = this._client._collectionsClient();
try {
const res = await callUnary(c, (req, cb) => c.queueLen(req, cb), {
bucket: this.name,
});
if (!res.success) throw serverError(res.resultCode, res.message);
return res.count;
} finally {
c.close();
}
}
}
// ------------------------------------------------------------------
// Client extension methods
// ------------------------------------------------------------------
declare module "./client" {
interface WaymakerClient {
createHashStore(config: HashStoreConfig): Promise<HashStore>;
getOrCreateHashStore(config: HashStoreConfig): Promise<HashStore>;
hashStore(name: string): HashStore;
deleteHashStore(name: string): Promise<void>;
createSetStore(config: SetStoreConfig): Promise<SetStore>;
getOrCreateSetStore(config: SetStoreConfig): Promise<SetStore>;
setStore(name: string): SetStore;
deleteSetStore(name: string): Promise<void>;
createQueue(config: QueueConfig): Promise<Queue>;
getOrCreateQueue(config: QueueConfig): Promise<Queue>;
queue(name: string): Queue;
deleteQueue(name: string): Promise<void>;
}
}
// Hash store
WaymakerClient.prototype.createHashStore = async function (
this: WaymakerClient,
config: HashStoreConfig
): Promise<HashStore> {
const c = this._collectionsClient();
try {
const res = await callUnary(c, (req, cb) => c.createHashStore(req, cb), {
name: config.name,
maxBytes: config.maxBytes ?? 0,
ephemeral: config.ephemeral ?? false,
});
if (!res.success) throw serverError(res.resultCode, res.message);
return new HashStore(this, config.name);
} finally {
c.close();
}
};
WaymakerClient.prototype.getOrCreateHashStore = async function (
this: WaymakerClient,
config: HashStoreConfig
): Promise<HashStore> {
try {
return await (this as WaymakerClient).createHashStore(config);
} catch (e) {
if (isServerErrorCode(e, "already_exists")) return new HashStore(this, config.name);
throw e;
}
};
WaymakerClient.prototype.hashStore = function (
this: WaymakerClient,
name: string
): HashStore {
return new HashStore(this, name);
};
WaymakerClient.prototype.deleteHashStore = async function (
this: WaymakerClient,
name: string
): Promise<void> {
const c = this._collectionsClient();
try {
const res = await callUnary(c, (req, cb) => c.deleteHashStore(req, cb), { name });
if (!res.success) throw serverError(res.resultCode, res.message);
} finally {
c.close();
}
};
// Set store
WaymakerClient.prototype.createSetStore = async function (
this: WaymakerClient,
config: SetStoreConfig
): Promise<SetStore> {
const c = this._collectionsClient();
try {
const res = await callUnary(c, (req, cb) => c.createSetStore(req, cb), {
name: config.name,
maxBytes: config.maxBytes ?? 0,
ephemeral: config.ephemeral ?? false,
});
if (!res.success) throw serverError(res.resultCode, res.message);
return new SetStore(this, config.name);
} finally {
c.close();
}
};
WaymakerClient.prototype.getOrCreateSetStore = async function (
this: WaymakerClient,
config: SetStoreConfig
): Promise<SetStore> {
try {
return await (this as WaymakerClient).createSetStore(config);
} catch (e) {
if (isServerErrorCode(e, "already_exists")) return new SetStore(this, config.name);
throw e;
}
};
WaymakerClient.prototype.setStore = function (
this: WaymakerClient,
name: string
): SetStore {
return new SetStore(this, name);
};
WaymakerClient.prototype.deleteSetStore = async function (
this: WaymakerClient,
name: string
): Promise<void> {
const c = this._collectionsClient();
try {
const res = await callUnary(c, (req, cb) => c.deleteSetStore(req, cb), { name });
if (!res.success) throw serverError(res.resultCode, res.message);
} finally {
c.close();
}
};
// Queue
WaymakerClient.prototype.createQueue = async function (
this: WaymakerClient,
config: QueueConfig
): Promise<Queue> {
const c = this._collectionsClient();
try {
const res = await callUnary(c, (req, cb) => c.createQueue(req, cb), {
name: config.name,
maxBytes: config.maxBytes ?? 0,
maxMessages: config.maxMessages ?? 0,
ephemeral: config.ephemeral ?? false,
});
if (!res.success) throw serverError(res.resultCode, res.message);
return new Queue(this, config.name);
} finally {
c.close();
}
};
WaymakerClient.prototype.getOrCreateQueue = async function (
this: WaymakerClient,
config: QueueConfig
): Promise<Queue> {
try {
return await (this as WaymakerClient).createQueue(config);
} catch (e) {
if (isServerErrorCode(e, "already_exists")) return new Queue(this, config.name);
throw e;
}
};
WaymakerClient.prototype.queue = function (
this: WaymakerClient,
name: string
): Queue {
return new Queue(this, name);
};
WaymakerClient.prototype.deleteQueue = async function (
this: WaymakerClient,
name: string
): Promise<void> {
const c = this._collectionsClient();
try {
const res = await callUnary(c, (req, cb) => c.deleteQueue(req, cb), { name });
if (!res.success) throw serverError(res.resultCode, res.message);
} finally {
c.close();
}
};
// ------------------------------------------------------------------
// Helpers
// ------------------------------------------------------------------
function toBuffer(v: Uint8Array | Buffer | string): Buffer {
if (typeof v === "string") return Buffer.from(v);
if (Buffer.isBuffer(v)) return v;
return Buffer.from(v);
}
function isServerErrorCode(err: unknown, code: string): boolean {
return (
typeof err === "object" &&
err !== null &&
"kind" in err &&
(err as { kind: string }).kind === "server" &&
"code" in err &&
(err as { code: string }).code === code
);
}

47
ts/src/error.ts Normal file
View File

@ -0,0 +1,47 @@
/**
* Waymaker client error types.
*
* `WaymakerError` is the single error type the client surfaces.
* The `kind` discriminant lets callers match specific cases:
*
* - `server` the server returned success=false. `code` matches the
* server's result_code string (e.g. "expired", "wrong_revision").
* - `rpc` a gRPC transport error (network, auth, etc.).
* - `invalid` bad call-site argument (no RPC was sent).
*/
export type WaymakerError =
| { kind: "server"; code: string; message: string }
| { kind: "rpc"; message: string; status?: number }
| { kind: "invalid"; message: string };
/** Create a server-logic error (success=false response). */
export function serverError(code: string, message: string): WaymakerError {
return { kind: "server", code, message };
}
/** Wrap a gRPC ServiceError. */
export function rpcError(err: { message: string; code?: number }): WaymakerError {
return { kind: "rpc", message: err.message, status: err.code };
}
/** Create an invalid-argument error. */
export function invalidError(message: string): WaymakerError {
return { kind: "invalid", message };
}
/** Type guard: is this a server-logic error with the given code? */
export function isServerError(err: unknown, code?: string): err is { kind: "server"; code: string; message: string } {
if (!isWaymakerError(err) || err.kind !== "server") return false;
if (code !== undefined) return err.code === code;
return true;
}
export function isWaymakerError(err: unknown): err is WaymakerError {
return (
typeof err === "object" &&
err !== null &&
"kind" in err &&
(err as WaymakerError).kind !== undefined
);
}

946
ts/src/genpb/cache.ts Normal file
View File

@ -0,0 +1,946 @@
// Code generated by protoc-gen-ts_proto. DO NOT EDIT.
// versions:
// protoc-gen-ts_proto v2.11.8
// protoc v7.34.1
// source: cache.proto
/* eslint-disable */
import { BinaryReader, BinaryWriter } from "@bufbuild/protobuf/wire";
import {
type CallOptions,
type ChannelCredentials,
Client,
type ClientOptions,
type ClientUnaryCall,
type handleUnaryCall,
makeGenericClientConstructor,
type Metadata,
type ServiceError,
type UntypedServiceImplementation,
} from "@grpc/grpc-js";
export const protobufPackage = "waymaker.cache";
export interface AttachPolicyRequest {
bucket: string;
policyId: string;
/**
* Policy-specific knobs (e.g. `max_entries`, `default_ttl_ms`)
* interpretation is server-side.
*/
params: { [key: string]: string };
}
export interface AttachPolicyRequest_ParamsEntry {
key: string;
value: string;
}
export interface AttachPolicyResponse {
success: boolean;
resultCode: string;
message: string;
}
export interface DetachPolicyRequest {
bucket: string;
}
export interface DetachPolicyResponse {
success: boolean;
resultCode: string;
message: string;
}
export interface StatsRequest {
bucket: string;
}
export interface StatsResponse {
success: boolean;
resultCode: string;
message: string;
hitCount: number;
missCount: number;
evictionCount: number;
sizeBytes: number;
entryCount: number;
}
function createBaseAttachPolicyRequest(): AttachPolicyRequest {
return { bucket: "", policyId: "", params: {} };
}
export const AttachPolicyRequest: MessageFns<AttachPolicyRequest> = {
encode(message: AttachPolicyRequest, writer: BinaryWriter = new BinaryWriter()): BinaryWriter {
if (message.bucket !== "") {
writer.uint32(10).string(message.bucket);
}
if (message.policyId !== "") {
writer.uint32(18).string(message.policyId);
}
globalThis.Object.entries(message.params).forEach(([key, value]: [string, string]) => {
AttachPolicyRequest_ParamsEntry.encode({ key: key as any, value }, writer.uint32(26).fork()).join();
});
return writer;
},
decode(input: BinaryReader | Uint8Array, length?: number): AttachPolicyRequest {
const reader = input instanceof BinaryReader ? input : new BinaryReader(input);
const end = length === undefined ? reader.len : reader.pos + length;
const message = createBaseAttachPolicyRequest();
while (reader.pos < end) {
const tag = reader.uint32();
switch (tag >>> 3) {
case 1: {
if (tag !== 10) {
break;
}
message.bucket = reader.string();
continue;
}
case 2: {
if (tag !== 18) {
break;
}
message.policyId = reader.string();
continue;
}
case 3: {
if (tag !== 26) {
break;
}
const entry3 = AttachPolicyRequest_ParamsEntry.decode(reader, reader.uint32());
if (entry3.value !== undefined) {
message.params[entry3.key] = entry3.value;
}
continue;
}
}
if ((tag & 7) === 4 || tag === 0) {
break;
}
reader.skip(tag & 7);
}
return message;
},
fromJSON(object: any): AttachPolicyRequest {
return {
bucket: isSet(object.bucket) ? globalThis.String(object.bucket) : "",
policyId: isSet(object.policyId)
? globalThis.String(object.policyId)
: isSet(object.policy_id)
? globalThis.String(object.policy_id)
: "",
params: isObject(object.params)
? (globalThis.Object.entries(object.params) as [string, any][]).reduce(
(acc: { [key: string]: string }, [key, value]: [string, any]) => {
acc[key] = globalThis.String(value);
return acc;
},
{},
)
: {},
};
},
toJSON(message: AttachPolicyRequest): unknown {
const obj: any = {};
if (message.bucket !== "") {
obj.bucket = message.bucket;
}
if (message.policyId !== "") {
obj.policyId = message.policyId;
}
if (message.params) {
const entries = globalThis.Object.entries(message.params) as [string, string][];
if (entries.length > 0) {
obj.params = {};
entries.forEach(([k, v]) => {
obj.params[k] = v;
});
}
}
return obj;
},
create(base?: DeepPartial<AttachPolicyRequest>): AttachPolicyRequest {
return AttachPolicyRequest.fromPartial(base ?? {});
},
fromPartial(object: DeepPartial<AttachPolicyRequest>): AttachPolicyRequest {
const message = createBaseAttachPolicyRequest();
message.bucket = object.bucket ?? "";
message.policyId = object.policyId ?? "";
message.params = (globalThis.Object.entries(object.params ?? {}) as [string, string][]).reduce(
(acc: { [key: string]: string }, [key, value]: [string, string]) => {
if (value !== undefined) {
acc[key] = globalThis.String(value);
}
return acc;
},
{},
);
return message;
},
};
function createBaseAttachPolicyRequest_ParamsEntry(): AttachPolicyRequest_ParamsEntry {
return { key: "", value: "" };
}
export const AttachPolicyRequest_ParamsEntry: MessageFns<AttachPolicyRequest_ParamsEntry> = {
encode(message: AttachPolicyRequest_ParamsEntry, writer: BinaryWriter = new BinaryWriter()): BinaryWriter {
if (message.key !== "") {
writer.uint32(10).string(message.key);
}
if (message.value !== "") {
writer.uint32(18).string(message.value);
}
return writer;
},
decode(input: BinaryReader | Uint8Array, length?: number): AttachPolicyRequest_ParamsEntry {
const reader = input instanceof BinaryReader ? input : new BinaryReader(input);
const end = length === undefined ? reader.len : reader.pos + length;
const message = createBaseAttachPolicyRequest_ParamsEntry();
while (reader.pos < end) {
const tag = reader.uint32();
switch (tag >>> 3) {
case 1: {
if (tag !== 10) {
break;
}
message.key = reader.string();
continue;
}
case 2: {
if (tag !== 18) {
break;
}
message.value = reader.string();
continue;
}
}
if ((tag & 7) === 4 || tag === 0) {
break;
}
reader.skip(tag & 7);
}
return message;
},
fromJSON(object: any): AttachPolicyRequest_ParamsEntry {
return {
key: isSet(object.key) ? globalThis.String(object.key) : "",
value: isSet(object.value) ? globalThis.String(object.value) : "",
};
},
toJSON(message: AttachPolicyRequest_ParamsEntry): unknown {
const obj: any = {};
if (message.key !== "") {
obj.key = message.key;
}
if (message.value !== "") {
obj.value = message.value;
}
return obj;
},
create(base?: DeepPartial<AttachPolicyRequest_ParamsEntry>): AttachPolicyRequest_ParamsEntry {
return AttachPolicyRequest_ParamsEntry.fromPartial(base ?? {});
},
fromPartial(object: DeepPartial<AttachPolicyRequest_ParamsEntry>): AttachPolicyRequest_ParamsEntry {
const message = createBaseAttachPolicyRequest_ParamsEntry();
message.key = object.key ?? "";
message.value = object.value ?? "";
return message;
},
};
function createBaseAttachPolicyResponse(): AttachPolicyResponse {
return { success: false, resultCode: "", message: "" };
}
export const AttachPolicyResponse: MessageFns<AttachPolicyResponse> = {
encode(message: AttachPolicyResponse, writer: BinaryWriter = new BinaryWriter()): BinaryWriter {
if (message.success !== false) {
writer.uint32(8).bool(message.success);
}
if (message.resultCode !== "") {
writer.uint32(18).string(message.resultCode);
}
if (message.message !== "") {
writer.uint32(26).string(message.message);
}
return writer;
},
decode(input: BinaryReader | Uint8Array, length?: number): AttachPolicyResponse {
const reader = input instanceof BinaryReader ? input : new BinaryReader(input);
const end = length === undefined ? reader.len : reader.pos + length;
const message = createBaseAttachPolicyResponse();
while (reader.pos < end) {
const tag = reader.uint32();
switch (tag >>> 3) {
case 1: {
if (tag !== 8) {
break;
}
message.success = reader.bool();
continue;
}
case 2: {
if (tag !== 18) {
break;
}
message.resultCode = reader.string();
continue;
}
case 3: {
if (tag !== 26) {
break;
}
message.message = reader.string();
continue;
}
}
if ((tag & 7) === 4 || tag === 0) {
break;
}
reader.skip(tag & 7);
}
return message;
},
fromJSON(object: any): AttachPolicyResponse {
return {
success: isSet(object.success) ? globalThis.Boolean(object.success) : false,
resultCode: isSet(object.resultCode)
? globalThis.String(object.resultCode)
: isSet(object.result_code)
? globalThis.String(object.result_code)
: "",
message: isSet(object.message) ? globalThis.String(object.message) : "",
};
},
toJSON(message: AttachPolicyResponse): unknown {
const obj: any = {};
if (message.success !== false) {
obj.success = message.success;
}
if (message.resultCode !== "") {
obj.resultCode = message.resultCode;
}
if (message.message !== "") {
obj.message = message.message;
}
return obj;
},
create(base?: DeepPartial<AttachPolicyResponse>): AttachPolicyResponse {
return AttachPolicyResponse.fromPartial(base ?? {});
},
fromPartial(object: DeepPartial<AttachPolicyResponse>): AttachPolicyResponse {
const message = createBaseAttachPolicyResponse();
message.success = object.success ?? false;
message.resultCode = object.resultCode ?? "";
message.message = object.message ?? "";
return message;
},
};
function createBaseDetachPolicyRequest(): DetachPolicyRequest {
return { bucket: "" };
}
export const DetachPolicyRequest: MessageFns<DetachPolicyRequest> = {
encode(message: DetachPolicyRequest, writer: BinaryWriter = new BinaryWriter()): BinaryWriter {
if (message.bucket !== "") {
writer.uint32(10).string(message.bucket);
}
return writer;
},
decode(input: BinaryReader | Uint8Array, length?: number): DetachPolicyRequest {
const reader = input instanceof BinaryReader ? input : new BinaryReader(input);
const end = length === undefined ? reader.len : reader.pos + length;
const message = createBaseDetachPolicyRequest();
while (reader.pos < end) {
const tag = reader.uint32();
switch (tag >>> 3) {
case 1: {
if (tag !== 10) {
break;
}
message.bucket = reader.string();
continue;
}
}
if ((tag & 7) === 4 || tag === 0) {
break;
}
reader.skip(tag & 7);
}
return message;
},
fromJSON(object: any): DetachPolicyRequest {
return { bucket: isSet(object.bucket) ? globalThis.String(object.bucket) : "" };
},
toJSON(message: DetachPolicyRequest): unknown {
const obj: any = {};
if (message.bucket !== "") {
obj.bucket = message.bucket;
}
return obj;
},
create(base?: DeepPartial<DetachPolicyRequest>): DetachPolicyRequest {
return DetachPolicyRequest.fromPartial(base ?? {});
},
fromPartial(object: DeepPartial<DetachPolicyRequest>): DetachPolicyRequest {
const message = createBaseDetachPolicyRequest();
message.bucket = object.bucket ?? "";
return message;
},
};
function createBaseDetachPolicyResponse(): DetachPolicyResponse {
return { success: false, resultCode: "", message: "" };
}
export const DetachPolicyResponse: MessageFns<DetachPolicyResponse> = {
encode(message: DetachPolicyResponse, writer: BinaryWriter = new BinaryWriter()): BinaryWriter {
if (message.success !== false) {
writer.uint32(8).bool(message.success);
}
if (message.resultCode !== "") {
writer.uint32(18).string(message.resultCode);
}
if (message.message !== "") {
writer.uint32(26).string(message.message);
}
return writer;
},
decode(input: BinaryReader | Uint8Array, length?: number): DetachPolicyResponse {
const reader = input instanceof BinaryReader ? input : new BinaryReader(input);
const end = length === undefined ? reader.len : reader.pos + length;
const message = createBaseDetachPolicyResponse();
while (reader.pos < end) {
const tag = reader.uint32();
switch (tag >>> 3) {
case 1: {
if (tag !== 8) {
break;
}
message.success = reader.bool();
continue;
}
case 2: {
if (tag !== 18) {
break;
}
message.resultCode = reader.string();
continue;
}
case 3: {
if (tag !== 26) {
break;
}
message.message = reader.string();
continue;
}
}
if ((tag & 7) === 4 || tag === 0) {
break;
}
reader.skip(tag & 7);
}
return message;
},
fromJSON(object: any): DetachPolicyResponse {
return {
success: isSet(object.success) ? globalThis.Boolean(object.success) : false,
resultCode: isSet(object.resultCode)
? globalThis.String(object.resultCode)
: isSet(object.result_code)
? globalThis.String(object.result_code)
: "",
message: isSet(object.message) ? globalThis.String(object.message) : "",
};
},
toJSON(message: DetachPolicyResponse): unknown {
const obj: any = {};
if (message.success !== false) {
obj.success = message.success;
}
if (message.resultCode !== "") {
obj.resultCode = message.resultCode;
}
if (message.message !== "") {
obj.message = message.message;
}
return obj;
},
create(base?: DeepPartial<DetachPolicyResponse>): DetachPolicyResponse {
return DetachPolicyResponse.fromPartial(base ?? {});
},
fromPartial(object: DeepPartial<DetachPolicyResponse>): DetachPolicyResponse {
const message = createBaseDetachPolicyResponse();
message.success = object.success ?? false;
message.resultCode = object.resultCode ?? "";
message.message = object.message ?? "";
return message;
},
};
function createBaseStatsRequest(): StatsRequest {
return { bucket: "" };
}
export const StatsRequest: MessageFns<StatsRequest> = {
encode(message: StatsRequest, writer: BinaryWriter = new BinaryWriter()): BinaryWriter {
if (message.bucket !== "") {
writer.uint32(10).string(message.bucket);
}
return writer;
},
decode(input: BinaryReader | Uint8Array, length?: number): StatsRequest {
const reader = input instanceof BinaryReader ? input : new BinaryReader(input);
const end = length === undefined ? reader.len : reader.pos + length;
const message = createBaseStatsRequest();
while (reader.pos < end) {
const tag = reader.uint32();
switch (tag >>> 3) {
case 1: {
if (tag !== 10) {
break;
}
message.bucket = reader.string();
continue;
}
}
if ((tag & 7) === 4 || tag === 0) {
break;
}
reader.skip(tag & 7);
}
return message;
},
fromJSON(object: any): StatsRequest {
return { bucket: isSet(object.bucket) ? globalThis.String(object.bucket) : "" };
},
toJSON(message: StatsRequest): unknown {
const obj: any = {};
if (message.bucket !== "") {
obj.bucket = message.bucket;
}
return obj;
},
create(base?: DeepPartial<StatsRequest>): StatsRequest {
return StatsRequest.fromPartial(base ?? {});
},
fromPartial(object: DeepPartial<StatsRequest>): StatsRequest {
const message = createBaseStatsRequest();
message.bucket = object.bucket ?? "";
return message;
},
};
function createBaseStatsResponse(): StatsResponse {
return {
success: false,
resultCode: "",
message: "",
hitCount: 0,
missCount: 0,
evictionCount: 0,
sizeBytes: 0,
entryCount: 0,
};
}
export const StatsResponse: MessageFns<StatsResponse> = {
encode(message: StatsResponse, writer: BinaryWriter = new BinaryWriter()): BinaryWriter {
if (message.success !== false) {
writer.uint32(8).bool(message.success);
}
if (message.resultCode !== "") {
writer.uint32(18).string(message.resultCode);
}
if (message.message !== "") {
writer.uint32(26).string(message.message);
}
if (message.hitCount !== 0) {
writer.uint32(32).uint64(message.hitCount);
}
if (message.missCount !== 0) {
writer.uint32(40).uint64(message.missCount);
}
if (message.evictionCount !== 0) {
writer.uint32(48).uint64(message.evictionCount);
}
if (message.sizeBytes !== 0) {
writer.uint32(56).uint64(message.sizeBytes);
}
if (message.entryCount !== 0) {
writer.uint32(64).uint64(message.entryCount);
}
return writer;
},
decode(input: BinaryReader | Uint8Array, length?: number): StatsResponse {
const reader = input instanceof BinaryReader ? input : new BinaryReader(input);
const end = length === undefined ? reader.len : reader.pos + length;
const message = createBaseStatsResponse();
while (reader.pos < end) {
const tag = reader.uint32();
switch (tag >>> 3) {
case 1: {
if (tag !== 8) {
break;
}
message.success = reader.bool();
continue;
}
case 2: {
if (tag !== 18) {
break;
}
message.resultCode = reader.string();
continue;
}
case 3: {
if (tag !== 26) {
break;
}
message.message = reader.string();
continue;
}
case 4: {
if (tag !== 32) {
break;
}
message.hitCount = longToNumber(reader.uint64());
continue;
}
case 5: {
if (tag !== 40) {
break;
}
message.missCount = longToNumber(reader.uint64());
continue;
}
case 6: {
if (tag !== 48) {
break;
}
message.evictionCount = longToNumber(reader.uint64());
continue;
}
case 7: {
if (tag !== 56) {
break;
}
message.sizeBytes = longToNumber(reader.uint64());
continue;
}
case 8: {
if (tag !== 64) {
break;
}
message.entryCount = longToNumber(reader.uint64());
continue;
}
}
if ((tag & 7) === 4 || tag === 0) {
break;
}
reader.skip(tag & 7);
}
return message;
},
fromJSON(object: any): StatsResponse {
return {
success: isSet(object.success) ? globalThis.Boolean(object.success) : false,
resultCode: isSet(object.resultCode)
? globalThis.String(object.resultCode)
: isSet(object.result_code)
? globalThis.String(object.result_code)
: "",
message: isSet(object.message) ? globalThis.String(object.message) : "",
hitCount: isSet(object.hitCount)
? globalThis.Number(object.hitCount)
: isSet(object.hit_count)
? globalThis.Number(object.hit_count)
: 0,
missCount: isSet(object.missCount)
? globalThis.Number(object.missCount)
: isSet(object.miss_count)
? globalThis.Number(object.miss_count)
: 0,
evictionCount: isSet(object.evictionCount)
? globalThis.Number(object.evictionCount)
: isSet(object.eviction_count)
? globalThis.Number(object.eviction_count)
: 0,
sizeBytes: isSet(object.sizeBytes)
? globalThis.Number(object.sizeBytes)
: isSet(object.size_bytes)
? globalThis.Number(object.size_bytes)
: 0,
entryCount: isSet(object.entryCount)
? globalThis.Number(object.entryCount)
: isSet(object.entry_count)
? globalThis.Number(object.entry_count)
: 0,
};
},
toJSON(message: StatsResponse): unknown {
const obj: any = {};
if (message.success !== false) {
obj.success = message.success;
}
if (message.resultCode !== "") {
obj.resultCode = message.resultCode;
}
if (message.message !== "") {
obj.message = message.message;
}
if (message.hitCount !== 0) {
obj.hitCount = Math.round(message.hitCount);
}
if (message.missCount !== 0) {
obj.missCount = Math.round(message.missCount);
}
if (message.evictionCount !== 0) {
obj.evictionCount = Math.round(message.evictionCount);
}
if (message.sizeBytes !== 0) {
obj.sizeBytes = Math.round(message.sizeBytes);
}
if (message.entryCount !== 0) {
obj.entryCount = Math.round(message.entryCount);
}
return obj;
},
create(base?: DeepPartial<StatsResponse>): StatsResponse {
return StatsResponse.fromPartial(base ?? {});
},
fromPartial(object: DeepPartial<StatsResponse>): StatsResponse {
const message = createBaseStatsResponse();
message.success = object.success ?? false;
message.resultCode = object.resultCode ?? "";
message.message = object.message ?? "";
message.hitCount = object.hitCount ?? 0;
message.missCount = object.missCount ?? 0;
message.evictionCount = object.evictionCount ?? 0;
message.sizeBytes = object.sizeBytes ?? 0;
message.entryCount = object.entryCount ?? 0;
return message;
},
};
export type WaymakerCacheServiceService = typeof WaymakerCacheServiceService;
export const WaymakerCacheServiceService = {
/**
* Apply a TTL policy to a bucket. `policy_id` selects from
* server-configured policies (initially: `lru`, `expiry`).
*/
attachPolicy: {
path: "/waymaker.cache.WaymakerCacheService/AttachPolicy" as const,
requestStream: false as const,
responseStream: false as const,
requestSerialize: (value: AttachPolicyRequest): Buffer => Buffer.from(AttachPolicyRequest.encode(value).finish()),
requestDeserialize: (value: Buffer): AttachPolicyRequest => AttachPolicyRequest.decode(value),
responseSerialize: (value: AttachPolicyResponse): Buffer =>
Buffer.from(AttachPolicyResponse.encode(value).finish()),
responseDeserialize: (value: Buffer): AttachPolicyResponse => AttachPolicyResponse.decode(value),
},
/**
* Detach the policy currently bound to `bucket` (no-op if
* none).
*/
detachPolicy: {
path: "/waymaker.cache.WaymakerCacheService/DetachPolicy" as const,
requestStream: false as const,
responseStream: false as const,
requestSerialize: (value: DetachPolicyRequest): Buffer => Buffer.from(DetachPolicyRequest.encode(value).finish()),
requestDeserialize: (value: Buffer): DetachPolicyRequest => DetachPolicyRequest.decode(value),
responseSerialize: (value: DetachPolicyResponse): Buffer =>
Buffer.from(DetachPolicyResponse.encode(value).finish()),
responseDeserialize: (value: Buffer): DetachPolicyResponse => DetachPolicyResponse.decode(value),
},
/**
* Report current cache stats (hit/miss/eviction counters,
* memory footprint) for a bucket.
*/
stats: {
path: "/waymaker.cache.WaymakerCacheService/Stats" as const,
requestStream: false as const,
responseStream: false as const,
requestSerialize: (value: StatsRequest): Buffer => Buffer.from(StatsRequest.encode(value).finish()),
requestDeserialize: (value: Buffer): StatsRequest => StatsRequest.decode(value),
responseSerialize: (value: StatsResponse): Buffer => Buffer.from(StatsResponse.encode(value).finish()),
responseDeserialize: (value: Buffer): StatsResponse => StatsResponse.decode(value),
},
} as const;
export interface WaymakerCacheServiceServer extends UntypedServiceImplementation {
/**
* Apply a TTL policy to a bucket. `policy_id` selects from
* server-configured policies (initially: `lru`, `expiry`).
*/
attachPolicy: handleUnaryCall<AttachPolicyRequest, AttachPolicyResponse>;
/**
* Detach the policy currently bound to `bucket` (no-op if
* none).
*/
detachPolicy: handleUnaryCall<DetachPolicyRequest, DetachPolicyResponse>;
/**
* Report current cache stats (hit/miss/eviction counters,
* memory footprint) for a bucket.
*/
stats: handleUnaryCall<StatsRequest, StatsResponse>;
}
export interface WaymakerCacheServiceClient extends Client {
/**
* Apply a TTL policy to a bucket. `policy_id` selects from
* server-configured policies (initially: `lru`, `expiry`).
*/
attachPolicy(
request: AttachPolicyRequest,
callback: (error: ServiceError | null, response: AttachPolicyResponse) => void,
): ClientUnaryCall;
attachPolicy(
request: AttachPolicyRequest,
metadata: Metadata,
callback: (error: ServiceError | null, response: AttachPolicyResponse) => void,
): ClientUnaryCall;
attachPolicy(
request: AttachPolicyRequest,
metadata: Metadata,
options: Partial<CallOptions>,
callback: (error: ServiceError | null, response: AttachPolicyResponse) => void,
): ClientUnaryCall;
/**
* Detach the policy currently bound to `bucket` (no-op if
* none).
*/
detachPolicy(
request: DetachPolicyRequest,
callback: (error: ServiceError | null, response: DetachPolicyResponse) => void,
): ClientUnaryCall;
detachPolicy(
request: DetachPolicyRequest,
metadata: Metadata,
callback: (error: ServiceError | null, response: DetachPolicyResponse) => void,
): ClientUnaryCall;
detachPolicy(
request: DetachPolicyRequest,
metadata: Metadata,
options: Partial<CallOptions>,
callback: (error: ServiceError | null, response: DetachPolicyResponse) => void,
): ClientUnaryCall;
/**
* Report current cache stats (hit/miss/eviction counters,
* memory footprint) for a bucket.
*/
stats(
request: StatsRequest,
callback: (error: ServiceError | null, response: StatsResponse) => void,
): ClientUnaryCall;
stats(
request: StatsRequest,
metadata: Metadata,
callback: (error: ServiceError | null, response: StatsResponse) => void,
): ClientUnaryCall;
stats(
request: StatsRequest,
metadata: Metadata,
options: Partial<CallOptions>,
callback: (error: ServiceError | null, response: StatsResponse) => void,
): ClientUnaryCall;
}
export const WaymakerCacheServiceClient = makeGenericClientConstructor(
WaymakerCacheServiceService,
"waymaker.cache.WaymakerCacheService",
) as unknown as {
new (address: string, credentials: ChannelCredentials, options?: Partial<ClientOptions>): WaymakerCacheServiceClient;
service: typeof WaymakerCacheServiceService;
serviceName: string;
};
type Builtin = Date | Function | Uint8Array | string | number | boolean | undefined;
export type DeepPartial<T> = T extends Builtin ? T
: T extends globalThis.Array<infer U> ? globalThis.Array<DeepPartial<U>>
: T extends ReadonlyArray<infer U> ? ReadonlyArray<DeepPartial<U>>
: T extends {} ? { [K in keyof T]?: DeepPartial<T[K]> }
: Partial<T>;
function longToNumber(int64: { toString(): string }): number {
const num = globalThis.Number(int64.toString());
if (num > globalThis.Number.MAX_SAFE_INTEGER) {
throw new globalThis.Error("Value is larger than Number.MAX_SAFE_INTEGER");
}
if (num < globalThis.Number.MIN_SAFE_INTEGER) {
throw new globalThis.Error("Value is smaller than Number.MIN_SAFE_INTEGER");
}
return num;
}
function isObject(value: any): boolean {
return typeof value === "object" && value !== null;
}
function isSet(value: any): boolean {
return value !== null && value !== undefined;
}
export interface MessageFns<T> {
encode(message: T, writer?: BinaryWriter): BinaryWriter;
decode(input: BinaryReader | Uint8Array, length?: number): T;
fromJSON(object: any): T;
toJSON(message: T): unknown;
create(base?: DeepPartial<T>): T;
fromPartial(object: DeepPartial<T>): T;
}

5216
ts/src/genpb/collections.ts Normal file

File diff suppressed because it is too large Load Diff

2985
ts/src/genpb/kv.ts Normal file

File diff suppressed because it is too large Load Diff

6336
ts/src/genpb/sketches.ts Normal file

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

83
ts/src/index.ts Normal file
View File

@ -0,0 +1,83 @@
/**
* @waymaker/client Official TypeScript client for waymaker.
*
* Import the client and subsystem modules you need:
*
* ```ts
* import { WaymakerClient, Scope } from "@waymaker/client";
*
* const client = WaymakerClient.connect("localhost:8818");
* const lock = await client.acquireLock("leader:myjob", {
* maxWaitMs: 0,
* leaseTtlMs: 60_000,
* scope: Scope.Local,
* });
* const renewal = lock.spawnRenewal(30_000);
* // ... do work
* await renewal.stop();
* await lock.unlock();
* ```
*/
// Core
export { WaymakerClient } from "./client";
export type { ConnectOptions, WaymakerError } from "./client";
export { isServerError, isWaymakerError, serverError, rpcError, invalidError } from "./client";
// Locks — import the module to register the extension methods
import "./lock";
export { Lock, RenewalHandle, Scope } from "./lock";
export type { LockConfig, Lease, LockState, AcquiredLock } from "./lock";
// Streams — import the module to register the extension methods
import "./stream";
export { Stream, Consumer, Message } from "./stream";
export type {
StreamConfig,
StreamUpdate,
ConsumerConfig,
PublishAck,
SourceStatus,
StreamSource,
SubjectTransform,
RetentionPolicy,
DeliverPolicy,
OnDropPolicy,
} from "./stream";
// KV — import the module to register the extension methods
import "./kv";
export { KvBucket } from "./kv";
export type { KvConfig, HistoryEntry, KvEvent } from "./kv";
// Collections — import the module to register the extension methods
import "./collections";
export { HashStore, Hash, SetStore, SetHandle, Queue } from "./collections";
export type { HashStoreConfig, SetStoreConfig, QueueConfig } from "./collections";
// Sketches — import the module to register the extension methods
import "./sketches";
export { Bloom, Hll, Cms, TopK, TDigest } from "./sketches";
export type {
BloomConfig,
BloomInfo,
HllConfig,
CmsConfig,
TopKConfig,
TopKListEntry,
TDigestConfig,
} from "./sketches";
// Object store — import the module to register the extension methods
import "./object";
export { ObjectStore } from "./object";
export type {
ObjectStoreConfig,
ObjectInfo,
ObjectEntry,
ObjectRevision,
PutOptions,
} from "./object";
// Cache — import the module to register the extension methods
import "./cache";

351
ts/src/kv.ts Normal file
View File

@ -0,0 +1,351 @@
/**
* KV subsystem Put/Get/Create/Update(CAS)/Delete/Keys/History/Touch/Watch.
*
* Entry points on `WaymakerClient`:
* - `client.createKv(config)` `KvBucket`
* - `client.getOrCreateKv(config)` idempotent
* - `client.kv(name)` handle without creation check
* - `client.deleteKv(name)`
*
* Wire conventions (subject format, tombstone/TTL headers) live
* entirely server-side. The client just calls the typed RPCs.
*/
import * as grpc from "@grpc/grpc-js";
import { WaymakerClient, callUnary, streamToAsyncIter } from "./client";
import { serverError } from "./error";
import type { KvWatchEvent } from "./genpb/kv";
// ------------------------------------------------------------------
// Config
// ------------------------------------------------------------------
export interface KvConfig {
name: string;
/** Cap on total bucket bytes. `undefined` = unbounded. */
maxBytes?: number;
/** Cap on per-value bytes. `undefined` = no cap. */
maxValueSize?: number;
/** Bucket-level TTL (ms). `undefined` = unbounded. */
maxAgeMs?: number;
/** Memory-only bucket. */
ephemeral?: boolean;
/**
* Per-key revision cap. 0 = unbounded. When N > 0, after each
* write the older revisions of that key beyond N most recent are
* dropped.
*/
maxRevisions?: number;
}
// ------------------------------------------------------------------
// History + Watch types
// ------------------------------------------------------------------
export interface HistoryEntry {
value: Uint8Array;
revision: number;
tsMs: number;
tombstone: boolean;
}
export type KvEvent =
| { kind: "put"; key: string; value: Uint8Array; revision: number }
| { kind: "delete"; key: string; revision: number };
// ------------------------------------------------------------------
// KvBucket
// ------------------------------------------------------------------
export class KvBucket {
readonly name: string;
/** @internal */
constructor(private readonly _client: WaymakerClient, name: string) {
this.name = name;
}
/**
* Put `value` under `key`. Latest-write-wins. Returns the new revision.
*/
async put(key: string, value: Uint8Array | Buffer | string): Promise<number> {
return this._put(key, value, 0);
}
/** Put with per-key TTL (ms). */
async putWithTtl(key: string, value: Uint8Array | Buffer | string, ttlMs: number): Promise<number> {
return this._put(key, value, ttlMs);
}
private async _put(key: string, value: Uint8Array | Buffer | string, ttlMs: number): Promise<number> {
const c = this._client._kvClient();
try {
const res = await callUnary(c, (req, cb) => c.put(req, cb), {
bucket: this.name,
key,
value: toBuffer(value),
ttlMs,
});
if (!res.success) throw serverError(res.resultCode, res.message);
return res.revision;
} finally {
c.close();
}
}
/**
* Atomic create fails with `{ kind: "server", code: "wrong_revision" }`
* if the key already exists.
*/
async create(key: string, value: Uint8Array | Buffer | string): Promise<number> {
const c = this._client._kvClient();
try {
const res = await callUnary(c, (req, cb) => c.create(req, cb), {
bucket: this.name,
key,
value: toBuffer(value),
ttlMs: 0,
});
if (!res.success) throw serverError(res.resultCode, res.message);
return res.revision;
} finally {
c.close();
}
}
/**
* CAS update succeeds only if current revision matches
* `expectedRevision`.
*/
async update(key: string, value: Uint8Array | Buffer | string, expectedRevision: number): Promise<number> {
const c = this._client._kvClient();
try {
const res = await callUnary(c, (req, cb) => c.update(req, cb), {
bucket: this.name,
key,
value: toBuffer(value),
expectedRevision,
ttlMs: 0,
});
if (!res.success) throw serverError(res.resultCode, res.message);
return res.revision;
} finally {
c.close();
}
}
/** Get the latest value. `null` when absent or tombstoned. */
async get(key: string): Promise<Uint8Array | null> {
const entry = await this.getWithRevision(key);
return entry ? entry.value : null;
}
/** Get value + revision. `null` when absent or tombstoned. */
async getWithRevision(key: string): Promise<{ value: Uint8Array; revision: number } | null> {
const c = this._client._kvClient();
try {
const res = await callUnary(c, (req, cb) => c.get(req, cb), {
bucket: this.name,
key,
});
if (!res.success) throw serverError(res.resultCode, res.message);
if (!res.entry) return null;
return { value: res.entry.value, revision: res.entry.revision };
} finally {
c.close();
}
}
/** Tombstone `key`. */
async delete(key: string): Promise<void> {
const c = this._client._kvClient();
try {
const res = await callUnary(c, (req, cb) => c.delete(req, cb), {
bucket: this.name,
key,
});
if (!res.success) throw serverError(res.resultCode, res.message);
} finally {
c.close();
}
}
/** Extend TTL on `key` without changing its value. */
async touch(key: string, ttlMs: number): Promise<number> {
const c = this._client._kvClient();
try {
const res = await callUnary(c, (req, cb) => c.touch(req, cb), {
bucket: this.name,
key,
ttlMs,
});
if (!res.success) throw serverError(res.resultCode, res.message);
return res.revision;
} finally {
c.close();
}
}
/** List every live key in the bucket (excludes tombstoned). */
async keys(): Promise<string[]> {
const c = this._client._kvClient();
try {
const res = await callUnary(c, (req, cb) => c.keys(req, cb), {
bucket: this.name,
});
if (!res.success) throw serverError(res.resultCode, res.message);
// eslint-disable-next-line @typescript-eslint/no-explicit-any
return res.entries.filter((e: any) => !e.deleted).map((e: any) => e.key as string);
} finally {
c.close();
}
}
/** Historical values at `key` in publish order. */
async history(key: string): Promise<HistoryEntry[]> {
const c = this._client._kvClient();
try {
const res = await callUnary(c, (req, cb) => c.history(req, cb), {
bucket: this.name,
key,
fromRevision: 0,
limit: 0,
});
if (!res.success) throw serverError(res.resultCode, res.message);
// eslint-disable-next-line @typescript-eslint/no-explicit-any
return res.entries.map((e: any) => ({
value: e.value as Uint8Array,
revision: e.revision as number,
tsMs: e.tsMs as number,
tombstone: e.deleted as boolean,
}));
} finally {
c.close();
}
}
/**
* Watch live changes at `key`. Returns an async iterable of events.
* Ends when the server closes the stream (or on error).
*/
async watch(key: string): Promise<AsyncIterable<KvEvent>> {
return this._watchInner(key);
}
/** Watch every key in the bucket. */
async watchAll(): Promise<AsyncIterable<KvEvent>> {
return this._watchInner("");
}
private async _watchInner(key: string): Promise<AsyncIterable<KvEvent>> {
const c = this._client._kvClient();
const rpcStream = c.watch({ bucket: this.name, key });
async function* gen(): AsyncGenerator<KvEvent> {
try {
for await (const raw of streamToAsyncIter(rpcStream)) {
const ev = raw as KvWatchEvent;
if (ev.put !== undefined) {
yield { kind: "put", key: ev.put.key, value: ev.put.value, revision: ev.put.revision };
} else if (ev.delete !== undefined) {
yield { kind: "delete", key: ev.delete.key, revision: ev.delete.revision };
}
}
} finally {
c.close();
}
}
return gen();
}
}
// ------------------------------------------------------------------
// Client extension methods
// ------------------------------------------------------------------
declare module "./client" {
interface WaymakerClient {
createKv(config: KvConfig): Promise<KvBucket>;
getOrCreateKv(config: KvConfig): Promise<KvBucket>;
/** Return a bucket handle without verifying it exists. */
kv(name: string): KvBucket;
deleteKv(name: string): Promise<void>;
}
}
WaymakerClient.prototype.createKv = async function (
this: WaymakerClient,
config: KvConfig
): Promise<KvBucket> {
const c = this._kvClient();
try {
const res = await callUnary(c, (req, cb) => c.createBucket(req, cb), {
bucket: config.name,
maxBytes: config.maxBytes ?? 0,
maxValueSize: config.maxValueSize ?? 0,
maxAgeMs: config.maxAgeMs ?? 0,
ephemeral: config.ephemeral ?? false,
maxRevisions: config.maxRevisions ?? 0,
});
if (!res.success) throw serverError(res.resultCode, res.message);
return new KvBucket(this, config.name);
} finally {
c.close();
}
};
WaymakerClient.prototype.getOrCreateKv = async function (
this: WaymakerClient,
config: KvConfig
): Promise<KvBucket> {
try {
return await (this as WaymakerClient).createKv(config);
} catch (e) {
if (isServerErrorCode(e, "already_exists")) {
return new KvBucket(this, config.name);
}
throw e;
}
};
WaymakerClient.prototype.kv = function (
this: WaymakerClient,
name: string
): KvBucket {
return new KvBucket(this, name);
};
WaymakerClient.prototype.deleteKv = async function (
this: WaymakerClient,
name: string
): Promise<void> {
const c = this._kvClient();
try {
const res = await callUnary(c, (req, cb) => c.deleteBucket(req, cb), { bucket: name });
if (!res.success) throw serverError(res.resultCode, res.message);
} finally {
c.close();
}
};
// ------------------------------------------------------------------
// Helpers
// ------------------------------------------------------------------
function toBuffer(v: Uint8Array | Buffer | string): Buffer {
if (typeof v === "string") return Buffer.from(v);
if (Buffer.isBuffer(v)) return v;
return Buffer.from(v);
}
function isServerErrorCode(err: unknown, code: string): boolean {
return (
typeof err === "object" &&
err !== null &&
"kind" in err &&
(err as { kind: string }).kind === "server" &&
"code" in err &&
(err as { code: string }).code === code
);
}

908
ts/src/lock.ts Normal file
View File

@ -0,0 +1,908 @@
/**
* Locks subsystem wraps the rwlock RPCs.
*
* Entry points on `WaymakerClient`:
* - `client.acquireLock(key, config)` exclusive (write) lock
* - `client.acquireReadLock(key, config)` shared (read) lock
* - `client.leaseStatus(key, id)` query a lease
* - `client.multiLock(keys, config)` atomic multi-key acquire
* - `client.listAcquiredLocks(keyPrefix?)` list held locks (operator)
*
* The returned `Lock` keeps a background loop that holds the server
* event stream open. On stream end/error it transparently re-binds
* (reusing the original `requestId` so a still-held lease is
* recovered, not re-contended). The lock's live state (fenceToken,
* leaseExpiresAtMs, lost) is readable at any time via the accessor
* methods and notified via an EventEmitter `on("change", cb)`.
*
* ## Leader-election pattern
*
* ```ts
* const lock = await client.acquireLock("leader:myjob", {
* maxWaitMs: 0, // try-acquire (fail fast if held)
* leaseTtlMs: 60_000,
* scope: Scope.Local,
* });
* const renewal = lock.spawnRenewal(30_000);
* try {
* await doWork(); // work can outlive a single lease window
* } finally {
* await renewal.stop();
* await lock.unlock();
* }
* ```
*/
import { EventEmitter } from "events";
import * as crypto from "crypto";
import * as grpc from "@grpc/grpc-js";
import { WaymakerClient, callUnary, streamToAsyncIter } from "./client";
import {
FenceScope,
LockEventType,
type LockRequest,
type LockEvent,
type AcquiredLock,
} from "./genpb/waymaker_locks";
import { serverError, rpcError, type WaymakerError } from "./error";
// ------------------------------------------------------------------
// Public types
// ------------------------------------------------------------------
/**
* Fence-scope tier controls durability of the per-key fence-token
* counter. Mirrors the proto `FenceScope`.
*/
export enum Scope {
/** Per-process RAM. Resets on restart / rebalance. Fastest. */
Ephemeral = FenceScope.ScopeEphemeral,
/** Disk-persisted on the owning node. Survives restart, not rebalance. */
Local = FenceScope.ScopeLocal,
/** Raft-replicated — cluster-wide monotonic. One Raft commit per acquire. */
Quorum = FenceScope.ScopeQuorum,
}
/** Configuration for `acquireLock` / `acquireReadLock`. */
export interface LockConfig {
/**
* Max time (ms) to wait for the lock. `0` = try-acquire (fail
* immediately if contended). `undefined` / very large = block up
* to `u32::MAX` ms (~49 days).
*/
maxWaitMs?: number;
/** Lease TTL (ms). Default 30 000 (30 s). */
leaseTtlMs?: number;
/** Priority class — higher jumps the queue. Default 0. */
priority?: number;
/** Fence-token durability. Default `Scope.Ephemeral`. */
scope?: Scope;
/** Free-form requester metadata for operator audit. */
requesterInfo?: string;
/** Application name shown in operator dashboards. */
requesterApplication?: string;
/**
* Idempotency key for retries of the same logical acquire.
* Auto-filled with a random UUID if left empty.
*/
requestId?: string;
}
/** Lease details returned by `leaseStatus` / `extend`. */
export interface Lease {
id: string;
key: string;
acquiredAtMs: number;
leaseExpiresAtMs: number;
fenceToken: number;
priority: number;
}
/** Live snapshot of a held lock's state, emitted on `"change"` events. */
export interface LockState {
/** Current lease id. Stable across re-bind; changes if truly re-won. */
id: string;
/**
* Current fence token. Re-read before every fenced side effect a
* lost-then-re-won lock carries a higher token.
*/
fenceToken: number;
/** Lease expiry (epoch ms), updated from heartbeats / re-acquire. */
leaseExpiresAtMs: number;
/**
* `true` once the client can no longer prove it holds the lock.
* A lost holder must stop acting as the holder.
*/
lost: boolean;
}
// ------------------------------------------------------------------
// Constants
// ------------------------------------------------------------------
const HOLD_BASE_BACKOFF_MS = 200;
const HOLD_MAX_BACKOFF_MS = 10_000;
const HOLD_RPC_TIMEOUT_MS = 10_000;
// ------------------------------------------------------------------
// Lock class
// ------------------------------------------------------------------
/**
* An acquired lock. The lease is kept alive server-side by TTL +
* renewal. This handle tracks live state and drives the background
* re-bind loop.
*
* Dropping the handle (without calling `unlock`) does NOT release
* the server-side lease the TTL will eventually expire it, or a
* caller to `spawnRenewal` will keep it alive. Call `unlock()`
* explicitly when done.
*/
export class Lock extends EventEmitter {
readonly key: string;
private _state: LockState;
private readonly _stopped: { value: boolean };
private readonly _stopCallbacks: Array<() => void> = [];
private _holdLoopPromise: Promise<void>;
/** @internal — constructed by acquireLock/acquireReadLock */
constructor(
private readonly _client: WaymakerClient,
key: string,
initState: LockState,
reacquireReq: LockRequest,
initStream: grpc.ClientReadableStream<LockEvent>,
read: boolean
) {
super();
this.key = key;
this._state = initState;
this._stopped = { value: false };
this._holdLoopPromise = holdLoop(
_client,
key,
read,
reacquireReq,
initStream,
initState,
this._stopped,
this._stopCallbacks,
(next) => this._publish(next)
);
}
// ---- Live state accessors ----------------------------------------
/** Current lease id (live). */
id(): string {
return this._state.id;
}
/**
* Current fence token (live). Re-read before every fenced side
* effect a lost-then-re-won lock carries a higher token.
*/
fenceToken(): number {
return this._state.fenceToken;
}
/** Current lease expiry, epoch ms (live). */
leaseExpiresAtMs(): number {
return this._state.leaseExpiresAtMs;
}
/** `true` once the client has lost the lock and cannot re-win it. */
isLost(): boolean {
return this._state.lost;
}
/** Current state snapshot. */
state(): Readonly<LockState> {
return this._state;
}
// ---- Operations --------------------------------------------------
/**
* Extend the lease by `additionalMs` milliseconds.
*/
async extend(additionalMs: number): Promise<Lease> {
const c = this._client._locksClient();
try {
const res = await callUnary(
c,
(req, cb) => c.extendLease(req, cb),
{
key: this.key,
id: this.id(),
leaseTimeout: clampMs(additionalMs),
}
);
if (!res.success) throw serverError(res.resultCode, res.message);
const lease = res.lease;
if (!lease) throw serverError("internal", "missing lease in ExtendLease response");
return {
id: lease.id,
key: lease.key,
acquiredAtMs: lease.createdAt,
leaseExpiresAtMs: lease.leaseExpiresAt,
fenceToken: lease.fenceToken,
priority: lease.priority,
};
} finally {
c.close();
}
}
/**
* Release the lock. Stops the background hold loop FIRST (so it
* cannot re-acquire a lock we're about to release), then sends
* `UnLock`. After this returns, the handle is invalid.
*/
async unlock(): Promise<void> {
this._stopHold();
const id = this.id();
const c = this._client._locksClient();
try {
const res = await callUnary(
c,
(req, cb) => c.unLock(req, cb),
{ key: this.key, id }
);
if (!res.success) throw serverError(res.resultCode, res.message);
} finally {
c.close();
}
}
/**
* Spawn a background task that periodically extends the lease.
* Returns a `RenewalHandle` call `.stop()` or drop it to halt.
*
* `everyMs` is the renewal interval; the TTL sent to the server is
* `everyMs * 2` so the lease always outlasts the next tick.
*
* The renewal reads the *live* lock id each tick, so it correctly
* tracks a re-won lock's new id after a primary failure.
*/
spawnRenewal(everyMs: number): RenewalHandle {
const ttlMs = Math.min(everyMs * 2, 0xffffffff);
const stopped = { value: false };
const callbacks: Array<() => void> = [];
const loop = async () => {
// Skip the first interval — caller just got a fresh lease.
await sleep(everyMs, stopped, callbacks);
while (!stopped.value) {
if (!this._state.lost) {
const id = this.id();
const c = this._client._locksClient();
try {
// Bound the RPC so a wedged server can't freeze the renewal.
await withTimeout(
callUnary(c, (req, cb) => c.extendLease(req, cb), {
key: this.key,
id,
leaseTimeout: clampMs(ttlMs),
}),
everyMs
);
} catch {
// Ignore renewal errors — keep trying next tick.
} finally {
c.close();
}
}
await sleep(everyMs, stopped, callbacks);
}
};
loop(); // fire and forget — errors are swallowed intentionally
return new RenewalHandle(stopped, callbacks);
}
// ------------------------------------------------------------------
private _publish(next: LockState): void {
const prev = this._state;
if (
prev.id !== next.id ||
prev.fenceToken !== next.fenceToken ||
prev.leaseExpiresAtMs !== next.leaseExpiresAtMs ||
prev.lost !== next.lost
) {
this._state = next;
this.emit("change", next);
}
}
private _stopHold(): void {
this._stopped.value = true;
for (const cb of this._stopCallbacks) cb();
this._stopCallbacks.length = 0;
}
/**
* Wait for the background hold loop to exit. Useful in tests or
* when you need to ensure the loop has fully stopped after
* `unlock()`.
*/
async waitForHoldLoop(): Promise<void> {
await this._holdLoopPromise;
}
}
// ------------------------------------------------------------------
// RenewalHandle
// ------------------------------------------------------------------
/**
* Returned by `Lock.spawnRenewal`. Drop or call `stop()` to halt.
*/
export class RenewalHandle {
private _stopped: { value: boolean };
private _callbacks: Array<() => void>;
/** @internal */
constructor(stopped: { value: boolean }, callbacks: Array<() => void>) {
this._stopped = stopped;
this._callbacks = callbacks;
}
/** Halt renewal. Returns after any in-flight RPC times out. */
stop(): void {
this._stopped.value = true;
for (const cb of this._callbacks) cb();
this._callbacks.length = 0;
}
}
// ------------------------------------------------------------------
// Background hold loop (mirrors Rust hold_loop / drain_stream /
// reacquire / confirm_ownership)
// ------------------------------------------------------------------
type PublishFn = (state: LockState) => void;
async function holdLoop(
client: WaymakerClient,
key: string,
read: boolean,
reacquireReq: LockRequest,
initStream: grpc.ClientReadableStream<LockEvent>,
initState: LockState,
stopped: { value: boolean },
stopCallbacks: Array<() => void>,
publish: PublishFn
): Promise<void> {
let cur = { ...initState };
let stream = initStream;
// Register a stop callback so unlock() can cancel the current
// blocking wait inside drain_stream.
let drainCancel: (() => void) | null = null;
stopCallbacks.push(() => {
drainCancel?.();
});
while (true) {
if (stopped.value) return;
const drained = await drainStream(
stream,
cur,
stopped,
(cancel) => { drainCancel = cancel; },
(next) => {
cur = { ...next };
publish(cur);
}
);
if (drained === "stopped") return;
// Stream ended or lease gone — attempt re-bind with backoff.
let backoff = HOLD_BASE_BACKOFF_MS;
while (true) {
if (stopped.value) return;
const rebound = await withRaceStop(
reacquire(client, read, reacquireReq),
stopped,
stopCallbacks
);
if (rebound === null) return; // stopped
if (rebound.kind === "bound") {
const next: LockState = {
id: rebound.id,
fenceToken: rebound.fenceToken,
leaseExpiresAtMs: rebound.leaseExpiresAtMs,
lost: false,
};
cur = { ...next };
publish(cur);
stream = rebound.stream;
break; // resume draining the fresh stream
}
// NotBound — check if we still own the lease.
const ownership = await withRaceStop(
confirmOwnership(client, key, cur.id),
stopped,
stopCallbacks
);
if (ownership === null) return; // stopped
if (ownership.kind === "held") {
const next: LockState = { ...cur, leaseExpiresAtMs: ownership.leaseExpiresAtMs };
cur = { ...next };
publish(cur);
// Still ours; keep monitoring with backoff.
} else if (ownership.kind === "lost") {
const next: LockState = { ...cur, lost: true };
cur = { ...next };
publish(cur);
return;
}
// Unknown — transient; back off and retry.
const slept = await withRaceStop(sleep(backoff, stopped, stopCallbacks), stopped, stopCallbacks);
if (slept === null) return;
backoff = Math.min(backoff * 2, HOLD_MAX_BACKOFF_MS);
}
}
}
// ------------------------------------------------------------------
// drain_stream: consume the event stream, updating state on
// Heartbeat/Acquired events, returning "stopped" or "disconnected".
// ------------------------------------------------------------------
type Drained = "stopped" | "disconnected";
function drainStream(
stream: grpc.ClientReadableStream<LockEvent>,
cur: LockState,
stopped: { value: boolean },
setCancelFn: (fn: () => void) => void,
onUpdate: (next: LockState) => void
): Promise<Drained> {
return new Promise<Drained>((resolve) => {
let done = false;
const finish = (result: Drained) => {
if (done) return;
done = true;
stream.removeAllListeners();
resolve(result);
};
setCancelFn(() => finish("stopped"));
if (stopped.value) {
stream.destroy();
finish("stopped");
return;
}
stream.on("data", (ev: LockEvent) => {
if (stopped.value) {
stream.destroy();
finish("stopped");
return;
}
const et = ev.eventType;
if (et === LockEventType.Heartbeat) {
onUpdate({ ...cur, leaseExpiresAtMs: ev.leaseExpiresAt });
cur = { ...cur, leaseExpiresAtMs: ev.leaseExpiresAt };
} else if (et === LockEventType.Acquired) {
const next: LockState = {
id: ev.id,
fenceToken: ev.fenceToken,
leaseExpiresAtMs: ev.leaseExpiresAt,
lost: false,
};
onUpdate(next);
cur = { ...next };
} else if (et === LockEventType.Expired || et === LockEventType.Failed) {
finish("disconnected");
}
// Waiting / Unknown — ignore.
});
stream.on("end", () => finish("disconnected"));
stream.on("error", () => finish("disconnected"));
});
}
// ------------------------------------------------------------------
// reacquire: one idempotent re-acquire attempt (maxWait=0, same
// requestId). Returns "bound" with a fresh stream, or "not_bound".
// ------------------------------------------------------------------
type Rebound =
| { kind: "bound"; stream: grpc.ClientReadableStream<LockEvent>; id: string; fenceToken: number; leaseExpiresAtMs: number }
| { kind: "not_bound" };
async function reacquire(
client: WaymakerClient,
read: boolean,
req: LockRequest
): Promise<Rebound> {
const attempt = (): Promise<Rebound> =>
new Promise<Rebound>((resolve) => {
const c = client._locksClient();
const stream: grpc.ClientReadableStream<LockEvent> = read
? c.readLock(req)
: c.lock(req);
let resolved = false;
const finish = (r: Rebound) => {
if (resolved) return;
resolved = true;
if (r.kind !== "bound") {
stream.destroy();
c.close();
}
resolve(r);
};
stream.on("data", (ev: LockEvent) => {
const et = ev.eventType;
if (et === LockEventType.Acquired) {
// Detach listeners before handing stream to caller.
stream.removeAllListeners("end");
stream.removeAllListeners("error");
finish({
kind: "bound",
stream,
id: ev.id,
fenceToken: ev.fenceToken,
leaseExpiresAtMs: ev.leaseExpiresAt,
});
} else if (et === LockEventType.Failed || et === LockEventType.Expired) {
finish({ kind: "not_bound" });
}
// Waiting / Heartbeat — keep reading.
});
stream.on("end", () => finish({ kind: "not_bound" }));
stream.on("error", () => finish({ kind: "not_bound" }));
});
try {
return await withTimeout(attempt(), HOLD_RPC_TIMEOUT_MS);
} catch {
return { kind: "not_bound" };
}
}
// ------------------------------------------------------------------
// confirmOwnership: query lease_status to see if we still hold the lock.
// ------------------------------------------------------------------
type Ownership =
| { kind: "held"; leaseExpiresAtMs: number }
| { kind: "lost" }
| { kind: "unknown" };
async function confirmOwnership(
client: WaymakerClient,
key: string,
id: string
): Promise<Ownership> {
const c = client._locksClient();
try {
const res = await withTimeout(
callUnary(c, (req, cb) => c.leaseStatus(req, cb), { key, id }),
HOLD_RPC_TIMEOUT_MS
);
if (res.success && res.lease) {
return { kind: "held", leaseExpiresAtMs: res.lease.leaseExpiresAt };
}
return { kind: "lost" };
} catch {
return { kind: "unknown" };
} finally {
c.close();
}
}
// ------------------------------------------------------------------
// Helpers
// ------------------------------------------------------------------
/** Sleep for `ms`; resolve early if `stopped` becomes true. */
function sleep(
ms: number,
stopped: { value: boolean },
callbacks: Array<() => void>
): Promise<void> {
return new Promise<void>((resolve) => {
if (stopped.value) { resolve(); return; }
const t = setTimeout(() => {
const idx = callbacks.indexOf(cancelFn);
if (idx !== -1) callbacks.splice(idx, 1);
resolve();
}, ms);
const cancelFn = () => {
clearTimeout(t);
resolve();
};
callbacks.push(cancelFn);
});
}
/**
* Race `promise` against `stopped`. Returns the promise's value, or
* `null` if `stopped` becomes true first.
*/
function withRaceStop<T>(
promise: Promise<T>,
stopped: { value: boolean },
callbacks: Array<() => void>
): Promise<T | null> {
if (stopped.value) return Promise.resolve(null);
let resolve: (v: T | null) => void;
const outer = new Promise<T | null>((res) => { resolve = res; });
const cancelFn = () => resolve(null);
callbacks.push(cancelFn);
promise.then(
(v) => {
const idx = callbacks.indexOf(cancelFn);
if (idx !== -1) callbacks.splice(idx, 1);
resolve(v);
},
() => {
const idx = callbacks.indexOf(cancelFn);
if (idx !== -1) callbacks.splice(idx, 1);
resolve(null);
}
);
return outer;
}
/** Reject if `promise` doesn't resolve within `ms`. */
function withTimeout<T>(promise: Promise<T>, ms: number): Promise<T> {
return new Promise<T>((resolve, reject) => {
const t = setTimeout(() => reject(new Error("timeout")), ms);
promise.then(
(v) => { clearTimeout(t); resolve(v); },
(e) => { clearTimeout(t); reject(e); }
);
});
}
/** Clamp a duration in ms to uint32 range. */
function clampMs(ms: number): number {
return Math.min(Math.max(0, Math.round(ms)), 0xffffffff);
}
/** Generate a random UUID v4. */
function randomUuid(): string {
return crypto.randomUUID();
}
/** Build a `LockRequest` proto from config + key. */
function buildLockRequest(key: string, config: LockConfig): { req: LockRequest; requestId: string } {
const requestId = config.requestId && config.requestId.length > 0
? config.requestId
: randomUuid();
const req: LockRequest = {
key,
maxWaitPeriod: clampMs(config.maxWaitMs ?? 3_600_000),
maxLeasePeriod: clampMs(config.leaseTtlMs ?? 30_000),
priority: config.priority ?? 0,
requesterInfo: config.requesterInfo ?? "",
requesterApplication: config.requesterApplication ?? "waymaker-ts-client",
requestId,
fenceScope: (config.scope as unknown as FenceScope) ?? FenceScope.ScopeEphemeral,
};
return { req, requestId };
}
// ------------------------------------------------------------------
// Client extension methods (acquire_lock, multi_lock, etc.)
// ------------------------------------------------------------------
declare module "./client" {
interface WaymakerClient {
/**
* Acquire an exclusive (write) lock. Blocks until granted or
* `maxWaitMs` elapses. A `maxWaitMs=0` fails immediately if the
* lock is contended.
*/
acquireLock(key: string, config?: LockConfig): Promise<Lock>;
/**
* Acquire a shared (read) lock. Multiple readers can hold
* concurrently; any write-lock waiter blocks new readers.
*/
acquireReadLock(key: string, config?: LockConfig): Promise<Lock>;
/** Query the current state of a lease by key + id. */
leaseStatus(key: string, id: string): Promise<Lease>;
/**
* Acquire N locks atomically. Server sorts keys lexicographically
* to guarantee deadlock-free ordering. Returns the leases on
* success; on failure every partial lock is already released.
*/
multiLock(
keys: Array<{ key: string; writeLock: boolean }>,
config?: LockConfig
): Promise<Lease[]>;
/**
* List held locks on the serving node. Operator introspection
* surface. In a multi-node cluster, call on each node to see the
* full picture.
*/
listAcquiredLocks(keyPrefix?: string): Promise<AcquiredLock[]>;
}
}
WaymakerClient.prototype.acquireLock = function (
this: WaymakerClient,
key: string,
config: LockConfig = {}
): Promise<Lock> {
return acquireLockInner(this, key, config, false);
};
WaymakerClient.prototype.acquireReadLock = function (
this: WaymakerClient,
key: string,
config: LockConfig = {}
): Promise<Lock> {
return acquireLockInner(this, key, config, true);
};
WaymakerClient.prototype.leaseStatus = async function (
this: WaymakerClient,
key: string,
id: string
): Promise<Lease> {
const c = this._locksClient();
try {
const res = await callUnary(c, (req, cb) => c.leaseStatus(req, cb), { key, id });
if (!res.success) throw serverError(res.resultCode, res.message);
const lease = res.lease;
if (!lease) throw serverError("internal", "missing lease in LeaseStatus response");
return {
id: lease.id,
key: lease.key,
acquiredAtMs: lease.createdAt,
leaseExpiresAtMs: lease.leaseExpiresAt,
fenceToken: lease.fenceToken,
priority: lease.priority,
};
} finally {
c.close();
}
};
WaymakerClient.prototype.multiLock = async function (
this: WaymakerClient,
keys: Array<{ key: string; writeLock: boolean }>,
config: LockConfig = {}
): Promise<Lease[]> {
const requestId = config.requestId && config.requestId.length > 0
? config.requestId
: randomUuid();
const c = this._locksClient();
try {
const res = await callUnary(c, (req, cb) => c.multiLock(req, cb), {
keys,
maxWaitPeriod: clampMs(config.maxWaitMs ?? 3_600_000),
maxLeasePeriod: clampMs(config.leaseTtlMs ?? 30_000),
priority: config.priority ?? 0,
requesterInfo: config.requesterInfo ?? "",
requesterApplication: config.requesterApplication ?? "waymaker-ts-client",
requestId,
fenceScope: (config.scope as unknown as FenceScope) ?? FenceScope.ScopeEphemeral,
});
if (!res.success) throw serverError(res.resultCode, res.message);
// eslint-disable-next-line @typescript-eslint/no-explicit-any
return res.leases.map((lease: any) => ({
id: lease.id as string,
key: lease.key as string,
acquiredAtMs: lease.createdAt as number,
leaseExpiresAtMs: lease.leaseExpiresAt as number,
fenceToken: lease.fenceToken as number,
priority: lease.priority as number,
}));
} finally {
c.close();
}
};
WaymakerClient.prototype.listAcquiredLocks = async function (
this: WaymakerClient,
keyPrefix = ""
): Promise<AcquiredLock[]> {
const c = this._locksClient();
try {
const res = await callUnary(c, (req, cb) => c.listAcquiredLocks(req, cb), { keyPrefix });
if (!res.success) throw serverError("rpc_error", "listAcquiredLocks failed");
return res.locks;
} finally {
c.close();
}
};
// ------------------------------------------------------------------
// Core acquire logic
// ------------------------------------------------------------------
async function acquireLockInner(
client: WaymakerClient,
key: string,
config: LockConfig,
read: boolean
): Promise<Lock> {
const { req, requestId } = buildLockRequest(key, config);
// Template for transparent re-acquire after stream drop: same
// requestId (so a still-held lease is recovered idempotently) but
// never block (maxWaitPeriod=0).
const reacquireReq: LockRequest = { ...req, maxWaitPeriod: 0 };
const c = client._locksClient();
const stream: grpc.ClientReadableStream<LockEvent> = read
? c.readLock(req)
: c.lock(req);
// Consume events until Acquired / Failed / Expired.
return new Promise<Lock>((resolve, reject) => {
let resolved = false;
const finish = (result: Lock | WaymakerError) => {
if (resolved) return;
resolved = true;
stream.removeAllListeners();
c.close();
if (result instanceof Lock) {
resolve(result);
} else {
reject(result);
}
};
stream.on("data", (ev: LockEvent) => {
const et = ev.eventType;
if (et === LockEventType.Acquired) {
const initState: LockState = {
id: ev.id,
fenceToken: ev.fenceToken,
leaseExpiresAtMs: ev.leaseExpiresAt,
lost: false,
};
// Detach our listeners — the Lock's hold loop takes over.
stream.removeAllListeners("end");
stream.removeAllListeners("error");
const lock = new Lock(client, key, initState, reacquireReq, stream, read);
finish(lock);
} else if (et === LockEventType.Failed) {
finish(serverError("failed", ev.message));
} else if (et === LockEventType.Expired) {
finish(serverError("expired", ev.message));
}
// Waiting / Heartbeat / Unknown — keep reading.
});
stream.on("end", () => {
if (!resolved) finish(serverError("stream_closed", "lock stream closed before acquire"));
});
stream.on("error", (err: Error) => {
if (!resolved) finish(rpcError(err as { message: string; code?: number }));
});
});
}
// Re-export types needed by callers.
export type { AcquiredLock } from "./genpb/waymaker_locks";

346
ts/src/object.ts Normal file
View File

@ -0,0 +1,346 @@
/**
* Object store subsystem chunked put/get with metadata.
*
* An object-store bucket is backed by a stream with subject filters
* `objm.>` (metadata) and `objc.>` (chunks). The wrapper hides
* the wire convention; create / get via the typed entry points.
*
* Entry points on `WaymakerClient`:
* - `client.createObjectStore(config)` create the backing stream
* - `client.getOrCreateObjectStore(config)` idempotent
* - `client.objectStore(name)` handle without creation
*
* `Store` carries per-object operations:
* - `store.put(name, payload)` / `store.putWith(name, payload, opts)`
* - `store.get(name)` `{ info, payload }`
* - `store.getRange(name, offset, len)` `{ info, offset, payload }`
* - `store.info(name)` `ObjectInfo | null`
* - `store.delete(name)` tombstoneSeq
* - `store.list(prefix)` / `store.listWithDeleted(prefix)`
* - `store.revisions(name)`
*/
import { WaymakerClient, callUnary } from "./client";
import { serverError } from "./error";
// ------------------------------------------------------------------
// Config & types
// ------------------------------------------------------------------
export interface ObjectStoreConfig {
name: string;
/** Cap on total bucket bytes. `undefined` = unbounded. */
maxBytes?: number;
/** Memory-only bucket. */
ephemeral?: boolean;
}
export interface ObjectInfo {
name: string;
totalBytes: number;
chunkCount: number;
chunkSize: number;
sha256: string;
tsMs: number;
headers: Array<[string, string]>;
revision: number;
deduped: boolean;
}
export interface ObjectEntry {
name: string;
totalBytes: number;
deleted: boolean;
}
export interface ObjectRevision {
metadataSeq: number;
totalBytes: number;
sha256: string;
tsMs: number;
deleted: boolean;
}
export interface PutOptions {
/** Bytes per chunk. 0 = server default (1 MiB). */
chunkSize?: number;
/** Optional headers on the object's metadata. */
headers?: Array<[string, string]>;
/** Pre-computed SHA-256 hex. Empty = server computes. */
sha256?: string;
/** Content-addressed chunk dedup. */
dedupe?: boolean;
}
// ------------------------------------------------------------------
// Store handle
// ------------------------------------------------------------------
export class ObjectStore {
readonly name: string;
/** @internal */
constructor(private readonly _client: WaymakerClient, name: string) {
this.name = name;
}
/** Put a whole object. Server chunks server-side. */
async put(name: string, payload: Uint8Array | Buffer | string): Promise<ObjectInfo> {
return this.putWith(name, payload, {});
}
/** Put with explicit options. */
async putWith(
name: string,
payload: Uint8Array | Buffer | string,
opts: PutOptions
): Promise<ObjectInfo> {
const c = this._client._streamsClient();
try {
const res = await callUnary(c, (req, cb) => c.putObject(req, cb), {
bucket: this.name,
name,
payload: toBuffer(payload),
chunkSize: opts.chunkSize ?? 0,
headers: (opts.headers ?? []).map(([key, value]) => ({ key, value })),
sha256: opts.sha256 ?? "",
dedupe: opts.dedupe ?? false,
});
if (!res.success) throw serverError(res.resultCode, res.message);
if (!res.info) throw serverError("internal", "missing info in PutObject response");
return fromPbInfo(res.info);
} finally {
c.close();
}
}
/** Get an object's full payload + metadata. */
async get(name: string): Promise<{ info: ObjectInfo; payload: Uint8Array }> {
const c = this._client._streamsClient();
try {
const res = await callUnary(c, (req, cb) => c.getObject(req, cb), {
bucket: this.name,
name,
});
if (!res.success) throw serverError(res.resultCode, res.message);
if (!res.info) throw serverError("internal", "missing info in GetObject response");
return { info: fromPbInfo(res.info), payload: res.payload };
} finally {
c.close();
}
}
/** Read a byte range. `len=0` reads from `offset` to end. */
async getRange(
name: string,
offset: number,
len: number
): Promise<{ info: ObjectInfo; actualOffset: number; payload: Uint8Array }> {
const c = this._client._streamsClient();
try {
const res = await callUnary(c, (req, cb) => c.getObjectRange(req, cb), {
bucket: this.name,
name,
offset,
len,
});
if (!res.success) throw serverError(res.resultCode, res.message);
if (!res.info) throw serverError("internal", "missing info in GetObjectRange response");
return { info: fromPbInfo(res.info), actualOffset: res.actualOffset, payload: res.payload };
} finally {
c.close();
}
}
/** Get metadata without payload. `null` if tombstoned. */
async info(name: string): Promise<ObjectInfo | null> {
const c = this._client._streamsClient();
try {
const res = await callUnary(c, (req, cb) => c.getObjectInfo(req, cb), {
bucket: this.name,
name,
});
if (!res.success) throw serverError(res.resultCode, res.message);
if (res.deleted) return null;
return res.info ? fromPbInfo(res.info) : null;
} finally {
c.close();
}
}
/** Tombstone an object. Returns the tombstone sequence. */
async delete(name: string): Promise<number> {
const c = this._client._streamsClient();
try {
const res = await callUnary(c, (req, cb) => c.deleteObject(req, cb), {
bucket: this.name,
name,
});
if (!res.success) throw serverError(res.resultCode, res.message);
return res.tombstoneSeq;
} finally {
c.close();
}
}
/** List objects (excludes tombstoned). */
async list(namePrefix = ""): Promise<ObjectEntry[]> {
return this._listInner(namePrefix, false);
}
/** List objects including tombstoned. */
async listWithDeleted(namePrefix = ""): Promise<ObjectEntry[]> {
return this._listInner(namePrefix, true);
}
private async _listInner(namePrefix: string, includeDeleted: boolean): Promise<ObjectEntry[]> {
const c = this._client._streamsClient();
try {
const res = await callUnary(c, (req, cb) => c.listObjects(req, cb), {
bucket: this.name,
namePrefix,
includeDeleted,
});
if (!res.success) throw serverError(res.resultCode, res.message);
// eslint-disable-next-line @typescript-eslint/no-explicit-any
return res.entries.map((e: any) => ({
name: e.name as string,
totalBytes: e.totalBytes as number,
deleted: e.deleted as boolean,
}));
} finally {
c.close();
}
}
/** List every metadata revision of `name` in sequence order. */
async revisions(name: string): Promise<ObjectRevision[]> {
const c = this._client._streamsClient();
try {
const res = await callUnary(c, (req, cb) => c.listObjectRevisions(req, cb), {
bucket: this.name,
name,
fromSeq: 0,
limit: 0,
});
if (!res.success) throw serverError(res.resultCode, res.message);
// eslint-disable-next-line @typescript-eslint/no-explicit-any
return res.revisions.map((r: any) => ({
metadataSeq: r.metadataSeq as number,
totalBytes: r.totalBytes as number,
sha256: r.sha256 as string,
tsMs: r.tsMs,
deleted: r.deleted,
}));
} finally {
c.close();
}
}
}
// ------------------------------------------------------------------
// Client extension methods
// ------------------------------------------------------------------
declare module "./client" {
interface WaymakerClient {
createObjectStore(config: ObjectStoreConfig): Promise<ObjectStore>;
getOrCreateObjectStore(config: ObjectStoreConfig): Promise<ObjectStore>;
objectStore(name: string): ObjectStore;
}
}
WaymakerClient.prototype.createObjectStore = async function (
this: WaymakerClient,
config: ObjectStoreConfig
): Promise<ObjectStore> {
const c = this._streamsClient();
try {
const streamConfig = buildStreamConfig(config);
const res = await callUnary(c, (req, cb) => c.createStream(req, cb), {
config: streamConfig,
});
if (!res.success) throw serverError(res.resultCode, res.message);
return new ObjectStore(this, config.name);
} finally {
c.close();
}
};
WaymakerClient.prototype.getOrCreateObjectStore = async function (
this: WaymakerClient,
config: ObjectStoreConfig
): Promise<ObjectStore> {
const c = this._streamsClient();
try {
// Try getStreamInfo first; create if missing.
const infoRes = await callUnary(c, (req, cb) => c.getStreamInfo(req, cb), { name: config.name });
if (infoRes.success) return new ObjectStore(this, config.name);
} catch {
// fall through to create
} finally {
c.close();
}
return (this as WaymakerClient).createObjectStore(config);
};
WaymakerClient.prototype.objectStore = function (
this: WaymakerClient,
name: string
): ObjectStore {
return new ObjectStore(this, name);
};
// ------------------------------------------------------------------
// Helpers
// ------------------------------------------------------------------
function toBuffer(v: Uint8Array | Buffer | string): Buffer {
if (typeof v === "string") return Buffer.from(v);
if (Buffer.isBuffer(v)) return v;
return Buffer.from(v);
}
function fromPbInfo(i: {
name: string;
totalBytes: number;
chunkCount: number;
chunkSize: number;
sha256: string;
tsMs: number;
headers: Array<{ key: string; value: string }>;
metadataSeq: number;
deduped: boolean;
}): ObjectInfo {
return {
name: i.name,
totalBytes: i.totalBytes,
chunkCount: i.chunkCount,
chunkSize: i.chunkSize,
sha256: i.sha256,
tsMs: i.tsMs,
headers: i.headers.map((h) => [h.key, h.value] as [string, string]),
revision: i.metadataSeq,
deduped: i.deduped,
};
}
function buildStreamConfig(config: ObjectStoreConfig) {
return {
name: config.name,
subjectsFilter: ["objm.>", "objc.>"],
retention: {
limits: {
maxAgeMs: 0,
maxMsgs: 0,
maxBytes: config.maxBytes ?? 0,
strictLimits: false,
},
},
blockSize: 0,
maxMsgBytes: 0,
ephemeral: config.ephemeral ?? false,
maxMsgsPerSubject: 0,
sources: [],
};
}

580
ts/src/sketches.ts Normal file
View File

@ -0,0 +1,580 @@
/**
* Sketches subsystem probabilistic data structures.
*
* Entry points on `WaymakerClient`:
* - Bloom: `createBloom` / `bloom` / `deleteBloom`
* - HLL: `createHll` / `hll` / `deleteHll`
* - CMS: `createCms` / `cms` / `deleteCms`
* - TopK: `createTopK` / `topk` / `deleteTopK`
* - t-digest: `createTDigest` / `tdigest` / `deleteTDigest`
*
* CMS / TopK / t-digest are proto-complete but server returns
* `unimplemented` on every call (implementation deferred).
*/
import { WaymakerClient, callUnary } from "./client";
import { serverError } from "./error";
// ------------------------------------------------------------------
// Bloom filter
// ------------------------------------------------------------------
export interface BloomConfig {
name: string;
capacity: number;
/** False-positive rate (e.g. 0.01). 0 = server default (0.01). */
errorRate?: number;
}
export interface BloomInfo {
capacity: number;
errorRate: number;
bitsSet: number;
bitCount: number;
hashCount: number;
itemsAdded: number;
}
export class Bloom {
readonly name: string;
/** @internal */
constructor(private readonly _client: WaymakerClient, name: string) {
this.name = name;
}
/** Add one item. */
async add(item: Uint8Array | Buffer | string): Promise<void> {
const c = this._client._sketchesClient();
try {
const res = await callUnary(c, (req, cb) => c.bloomAdd(req, cb), {
name: this.name,
item: toBuffer(item),
});
if (!res.success) throw serverError(res.resultCode, res.message);
} finally {
c.close();
}
}
/** Add many items. */
async addMany(items: Array<Uint8Array | Buffer | string>): Promise<void> {
const c = this._client._sketchesClient();
try {
const res = await callUnary(c, (req, cb) => c.bloomMultiAdd(req, cb), {
name: this.name,
items: items.map(toBuffer),
});
if (!res.success) throw serverError(res.resultCode, res.message);
} finally {
c.close();
}
}
/** Probabilistic membership test. `true` = probably present. */
async exists(item: Uint8Array | Buffer | string): Promise<boolean> {
const c = this._client._sketchesClient();
try {
const res = await callUnary(c, (req, cb) => c.bloomExists(req, cb), {
name: this.name,
item: toBuffer(item),
});
if (!res.success) throw serverError(res.resultCode, res.message);
return res.exists;
} finally {
c.close();
}
}
/** Probabilistic membership for many items. */
async existsMany(items: Array<Uint8Array | Buffer | string>): Promise<boolean[]> {
const c = this._client._sketchesClient();
try {
const res = await callUnary(c, (req, cb) => c.bloomMultiExists(req, cb), {
name: this.name,
items: items.map(toBuffer),
});
if (!res.success) throw serverError(res.resultCode, res.message);
return res.exists;
} finally {
c.close();
}
}
/** Get filter metadata. */
async info(): Promise<BloomInfo> {
const c = this._client._sketchesClient();
try {
const res = await callUnary(c, (req, cb) => c.bloomInfo(req, cb), {
name: this.name,
});
if (!res.success) throw serverError(res.resultCode, res.message);
return {
capacity: res.capacity,
errorRate: res.errorRate,
bitsSet: res.bitsSet,
bitCount: res.bitCount,
hashCount: res.hashCount,
itemsAdded: res.itemsAdded,
};
} finally {
c.close();
}
}
}
// ------------------------------------------------------------------
// HyperLogLog
// ------------------------------------------------------------------
export interface HllConfig {
name: string;
/** 2^precision = register count. Valid 4..18. 0 = server default (14). */
precision?: number;
}
export class Hll {
readonly name: string;
/** @internal */
constructor(private readonly _client: WaymakerClient, name: string) {
this.name = name;
}
/** Add items. */
async add(items: Array<Uint8Array | Buffer | string>): Promise<void> {
const c = this._client._sketchesClient();
try {
const res = await callUnary(c, (req, cb) => c.hllAdd(req, cb), {
name: this.name,
items: items.map(toBuffer),
});
if (!res.success) throw serverError(res.resultCode, res.message);
} finally {
c.close();
}
}
/** Estimated cardinality. */
async count(): Promise<number> {
const c = this._client._sketchesClient();
try {
const res = await callUnary(c, (req, cb) => c.hllCount(req, cb), {
name: this.name,
});
if (!res.success) throw serverError(res.resultCode, res.message);
return res.estimate;
} finally {
c.close();
}
}
/** Merge `sources` into this HLL (union of their registers). */
async mergeFrom(sources: string[]): Promise<void> {
const c = this._client._sketchesClient();
try {
const res = await callUnary(c, (req, cb) => c.hllMerge(req, cb), {
destination: this.name,
sources,
});
if (!res.success) throw serverError(res.resultCode, res.message);
} finally {
c.close();
}
}
}
// ------------------------------------------------------------------
// Count-Min Sketch
// ------------------------------------------------------------------
export interface CmsConfig {
name: string;
/** Sketch width (columns). 0 = server default. */
width?: number;
/** Sketch depth (rows). 0 = server default. */
depth?: number;
}
export class Cms {
readonly name: string;
/** @internal */
constructor(private readonly _client: WaymakerClient, name: string) {
this.name = name;
}
/** Increment item counts. Returns the new count per item. */
async incr(items: Array<{ item: Uint8Array | Buffer | string; count: number }>): Promise<number[]> {
const c = this._client._sketchesClient();
try {
const res = await callUnary(c, (req, cb) => c.cmsIncrBy(req, cb), {
name: this.name,
items: items.map(({ item, count }) => ({ item: toBuffer(item), count })),
});
if (!res.success) throw serverError(res.resultCode, res.message);
return res.counts;
} finally {
c.close();
}
}
/** Query estimated counts. */
async query(items: Array<Uint8Array | Buffer | string>): Promise<number[]> {
const c = this._client._sketchesClient();
try {
const res = await callUnary(c, (req, cb) => c.cmsQuery(req, cb), {
name: this.name,
items: items.map(toBuffer),
});
if (!res.success) throw serverError(res.resultCode, res.message);
return res.counts;
} finally {
c.close();
}
}
}
// ------------------------------------------------------------------
// Top-K
// ------------------------------------------------------------------
export interface TopKConfig {
name: string;
k: number;
/** Underlying sketch width. 0 = server default. */
width?: number;
/** Underlying sketch depth. 0 = server default. */
depth?: number;
/** Probability-decay factor (0..1). 0 = server default. */
decay?: number;
}
export interface TopKListEntry {
item: Uint8Array;
count: number;
}
export class TopK {
readonly name: string;
/** @internal */
constructor(private readonly _client: WaymakerClient, name: string) {
this.name = name;
}
/** Add items. Returns evicted item (if any) per slot. */
async add(items: Array<Uint8Array | Buffer | string>): Promise<Uint8Array[]> {
const c = this._client._sketchesClient();
try {
const res = await callUnary(c, (req, cb) => c.topKAdd(req, cb), {
name: this.name,
items: items.map(toBuffer),
});
if (!res.success) throw serverError(res.resultCode, res.message);
return res.evicted;
} finally {
c.close();
}
}
/** Test membership in top-K. */
async query(items: Array<Uint8Array | Buffer | string>): Promise<boolean[]> {
const c = this._client._sketchesClient();
try {
const res = await callUnary(c, (req, cb) => c.topKQuery(req, cb), {
name: this.name,
items: items.map(toBuffer),
});
if (!res.success) throw serverError(res.resultCode, res.message);
return res.inTopK;
} finally {
c.close();
}
}
/** List the current top-K entries. */
async list(): Promise<TopKListEntry[]> {
const c = this._client._sketchesClient();
try {
const res = await callUnary(c, (req, cb) => c.topKList(req, cb), {
name: this.name,
});
if (!res.success) throw serverError(res.resultCode, res.message);
// eslint-disable-next-line @typescript-eslint/no-explicit-any
return res.entries.map((e: any) => ({ item: e.item as Buffer, count: e.count as number }));
} finally {
c.close();
}
}
}
// ------------------------------------------------------------------
// t-digest
// ------------------------------------------------------------------
export interface TDigestConfig {
name: string;
/** Compression. Higher = better tail accuracy. 0 = server default. */
compression?: number;
}
export class TDigest {
readonly name: string;
/** @internal */
constructor(private readonly _client: WaymakerClient, name: string) {
this.name = name;
}
/** Add values. */
async add(values: number[]): Promise<void> {
const c = this._client._sketchesClient();
try {
const res = await callUnary(c, (req, cb) => c.tDigestAdd(req, cb), {
name: this.name,
values,
});
if (!res.success) throw serverError(res.resultCode, res.message);
} finally {
c.close();
}
}
/** Estimate quantiles (0..1). */
async quantile(quantiles: number[]): Promise<number[]> {
const c = this._client._sketchesClient();
try {
const res = await callUnary(c, (req, cb) => c.tDigestQuantile(req, cb), {
name: this.name,
quantiles,
});
if (!res.success) throw serverError(res.resultCode, res.message);
return res.values;
} finally {
c.close();
}
}
/** Get min and max values. */
async minMax(): Promise<{ min: number; max: number }> {
const c = this._client._sketchesClient();
try {
const res = await callUnary(c, (req, cb) => c.tDigestMinMax(req, cb), {
name: this.name,
});
if (!res.success) throw serverError(res.resultCode, res.message);
return { min: res.min, max: res.max };
} finally {
c.close();
}
}
}
// ------------------------------------------------------------------
// Client extension methods
// ------------------------------------------------------------------
declare module "./client" {
interface WaymakerClient {
createBloom(config: BloomConfig): Promise<Bloom>;
bloom(name: string): Bloom;
deleteBloom(name: string): Promise<void>;
createHll(config: HllConfig): Promise<Hll>;
hll(name: string): Hll;
deleteHll(name: string): Promise<void>;
createCms(config: CmsConfig): Promise<Cms>;
cms(name: string): Cms;
deleteCms(name: string): Promise<void>;
createTopK(config: TopKConfig): Promise<TopK>;
topk(name: string): TopK;
deleteTopK(name: string): Promise<void>;
createTDigest(config: TDigestConfig): Promise<TDigest>;
tdigest(name: string): TDigest;
deleteTDigest(name: string): Promise<void>;
}
}
// Bloom
WaymakerClient.prototype.createBloom = async function (
this: WaymakerClient,
config: BloomConfig
): Promise<Bloom> {
const c = this._sketchesClient();
try {
const res = await callUnary(c, (req, cb) => c.bloomReserve(req, cb), {
name: config.name,
capacity: config.capacity,
errorRate: config.errorRate ?? 0,
});
if (!res.success) throw serverError(res.resultCode, res.message);
return new Bloom(this, config.name);
} finally {
c.close();
}
};
WaymakerClient.prototype.bloom = function (this: WaymakerClient, name: string): Bloom {
return new Bloom(this, name);
};
WaymakerClient.prototype.deleteBloom = async function (
this: WaymakerClient,
name: string
): Promise<void> {
const c = this._sketchesClient();
try {
const res = await callUnary(c, (req, cb) => c.bloomDelete(req, cb), { name });
if (!res.success) throw serverError(res.resultCode, res.message);
} finally {
c.close();
}
};
// HLL
WaymakerClient.prototype.createHll = async function (
this: WaymakerClient,
config: HllConfig
): Promise<Hll> {
const c = this._sketchesClient();
try {
const res = await callUnary(c, (req, cb) => c.hllReserve(req, cb), {
name: config.name,
precision: config.precision ?? 0,
});
if (!res.success) throw serverError(res.resultCode, res.message);
return new Hll(this, config.name);
} finally {
c.close();
}
};
WaymakerClient.prototype.hll = function (this: WaymakerClient, name: string): Hll {
return new Hll(this, name);
};
WaymakerClient.prototype.deleteHll = async function (
this: WaymakerClient,
name: string
): Promise<void> {
const c = this._sketchesClient();
try {
const res = await callUnary(c, (req, cb) => c.hllDelete(req, cb), { name });
if (!res.success) throw serverError(res.resultCode, res.message);
} finally {
c.close();
}
};
// CMS
WaymakerClient.prototype.createCms = async function (
this: WaymakerClient,
config: CmsConfig
): Promise<Cms> {
const c = this._sketchesClient();
try {
const res = await callUnary(c, (req, cb) => c.cmsReserve(req, cb), {
name: config.name,
width: config.width ?? 0,
depth: config.depth ?? 0,
});
if (!res.success) throw serverError(res.resultCode, res.message);
return new Cms(this, config.name);
} finally {
c.close();
}
};
WaymakerClient.prototype.cms = function (this: WaymakerClient, name: string): Cms {
return new Cms(this, name);
};
WaymakerClient.prototype.deleteCms = async function (
this: WaymakerClient,
name: string
): Promise<void> {
const c = this._sketchesClient();
try {
const res = await callUnary(c, (req, cb) => c.cmsDelete(req, cb), { name });
if (!res.success) throw serverError(res.resultCode, res.message);
} finally {
c.close();
}
};
// TopK
WaymakerClient.prototype.createTopK = async function (
this: WaymakerClient,
config: TopKConfig
): Promise<TopK> {
const c = this._sketchesClient();
try {
const res = await callUnary(c, (req, cb) => c.topKReserve(req, cb), {
name: config.name,
k: config.k,
width: config.width ?? 0,
depth: config.depth ?? 0,
decay: config.decay ?? 0,
});
if (!res.success) throw serverError(res.resultCode, res.message);
return new TopK(this, config.name);
} finally {
c.close();
}
};
WaymakerClient.prototype.topk = function (this: WaymakerClient, name: string): TopK {
return new TopK(this, name);
};
WaymakerClient.prototype.deleteTopK = async function (
this: WaymakerClient,
name: string
): Promise<void> {
const c = this._sketchesClient();
try {
const res = await callUnary(c, (req, cb) => c.topKDelete(req, cb), { name });
if (!res.success) throw serverError(res.resultCode, res.message);
} finally {
c.close();
}
};
// TDigest
WaymakerClient.prototype.createTDigest = async function (
this: WaymakerClient,
config: TDigestConfig
): Promise<TDigest> {
const c = this._sketchesClient();
try {
const res = await callUnary(c, (req, cb) => c.tDigestCreate(req, cb), {
name: config.name,
compression: config.compression ?? 0,
});
if (!res.success) throw serverError(res.resultCode, res.message);
return new TDigest(this, config.name);
} finally {
c.close();
}
};
WaymakerClient.prototype.tdigest = function (this: WaymakerClient, name: string): TDigest {
return new TDigest(this, name);
};
WaymakerClient.prototype.deleteTDigest = async function (
this: WaymakerClient,
name: string
): Promise<void> {
const c = this._sketchesClient();
try {
const res = await callUnary(c, (req, cb) => c.tDigestDelete(req, cb), { name });
if (!res.success) throw serverError(res.resultCode, res.message);
} finally {
c.close();
}
};
// ------------------------------------------------------------------
// Helpers
// ------------------------------------------------------------------
function toBuffer(v: Uint8Array | Buffer | string): Buffer {
if (typeof v === "string") return Buffer.from(v);
if (Buffer.isBuffer(v)) return v;
return Buffer.from(v);
}

647
ts/src/stream.ts Normal file
View File

@ -0,0 +1,647 @@
/**
* Streams subsystem publish, consumers, ack/nak, stream management.
*
* Entry points on `WaymakerClient`:
* - `client.createStream(config)` create a new stream
* - `client.getStream(name)` handle to existing stream
* - `client.getOrCreateStream(config)` idempotent
* - `client.updateStream(name, update)` mutate limits
* - `client.deleteStream(name)`
* - `client.getStreamSources()` per-(sourcing, source) tail status
*
* Returned `Stream` carries per-stream operations:
* - `stream.publish(subject, payload)`
* - `stream.createConsumer(config)` `Consumer`
* - `stream.getConsumer(name)` / `getOrCreateConsumer` / `deleteConsumer`
* - `stream.sourcesStatus()`
*
* Returned `Consumer` carries:
* - `consumer.messages()` `AsyncIterable<Message>` (push/subscribe)
* - `consumer.fetch(batchSize)` `Message[]` (pull)
* - `message.ack()` / `nak()` / `term()` / `inProgress()`
*/
import * as grpc from "@grpc/grpc-js";
import { WaymakerClient, callUnary, streamToAsyncIter } from "./client";
import { serverError, rpcError } from "./error";
import type {
MessagePb,
MessageHeader,
SubscribeEvent,
} from "./genpb/waymaker_streams";
// ------------------------------------------------------------------
// Enums
// ------------------------------------------------------------------
export enum RetentionPolicy {
Limits = "limits",
WorkQueue = "work_queue",
Interest = "interest",
}
export enum DeliverPolicy {
All = "all",
New = "new",
Last = "last",
ByStartSequence = "by_start_sequence",
ByStartTime = "by_start_time",
}
export enum OnDropPolicy {
Halt = 0,
SkipToFirstAvailable = 1,
}
// ------------------------------------------------------------------
// Config types
// ------------------------------------------------------------------
export interface SubjectTransform {
sourcePattern: string;
destination: string;
}
export interface StreamSource {
sourceStream: string;
/** NATS wildcard pattern. Empty = all. */
filterSubject?: string;
/** First source seq. 0 = beginning. Mutually exclusive with startTimeMs. */
startSeq?: number;
/** Wall-clock start time (ms since epoch). Mutually exclusive with startSeq. */
startTimeMs?: number;
/** Cap on initial backfill. 0 = no cap. */
maxInitialBackfill?: number;
/** Subject rewrite. */
subjectTransform?: SubjectTransform;
/** What to do when source retention drops messages past our watermark. */
onDrop?: OnDropPolicy;
/** Dead-letter stream. */
dlqStream?: string;
}
export interface StreamConfig {
name: string;
subjects?: string[];
retention?: RetentionPolicy;
maxAgeMs?: number;
maxMessages?: number;
maxBytes?: number;
maxMessageSize?: number;
/** Block size override. 0 = server default. */
blockSize?: number;
/** Reject publishes over limits instead of dropping oldest. */
strictLimits?: boolean;
/** Memory-only; survives failover via replication. */
ephemeral?: boolean;
/** Per-subject revision cap. 0 = unbounded. */
maxMsgsPerSubject?: number;
/** Cross-stream sources. */
sources?: StreamSource[];
}
export interface StreamUpdate {
maxAgeMs?: number;
maxMsgs?: number;
maxBytes?: number;
maxMsgBytes?: number;
strictLimits?: boolean;
}
export interface ConsumerConfig {
durableName: string;
filterSubject?: string;
deliverPolicy?: DeliverPolicy;
/** Start sequence for ByStartSequence. */
startSeq?: number;
/** Start time (ms) for ByStartTime. */
startTimeMs?: number;
/** Ack-wait window (ms). Default 30 000. */
ackWaitMs?: number;
/** Max delivery attempts. Default 5. */
maxDeliver?: number;
/** Queue group label for shared consumption. */
deliverGroup?: string;
/** Dead-letter subject within the stream. */
deadLetterSubject?: string;
}
// ------------------------------------------------------------------
// PublishAck
// ------------------------------------------------------------------
export interface PublishAck {
sequence: number;
}
// ------------------------------------------------------------------
// SourceStatus
// ------------------------------------------------------------------
export interface SourceStatus {
sourcingStream: string;
sourceStream: string;
lastSourcedSeq: number;
pulledTotal: number;
lastError: string;
lastErrorTsMs: number;
}
// ------------------------------------------------------------------
// Message
// ------------------------------------------------------------------
export class Message {
readonly subject: string;
readonly payload: Buffer;
readonly headers: Array<[string, string]>;
readonly sequence: number;
readonly tsMs: number;
readonly deliverCount: number;
/** @internal */
constructor(
private readonly _client: WaymakerClient,
private readonly _stream: string,
private readonly _consumer: string,
pb: MessagePb
) {
this.subject = pb.subject;
this.payload = Buffer.from(pb.payload);
this.headers = pb.headers.map((h: MessageHeader) => [h.key, h.value]);
this.sequence = pb.seq;
this.tsMs = pb.tsMs;
this.deliverCount = pb.deliverCount;
}
/** Positive acknowledgment — message is consumed. */
async ack(): Promise<void> {
const c = this._client._streamsClient();
try {
const res = await callUnary(c, (req, cb) => c.ack(req, cb), {
stream: this._stream,
consumer: this._consumer,
seq: this.sequence,
});
if (!res.success) throw serverError(res.resultCode, res.message);
} finally {
c.close();
}
}
/** Negative acknowledgment — request immediate redelivery. */
async nak(): Promise<void> {
await this.nakWithDelay(0);
}
/** Negative acknowledgment with delay (ms). */
async nakWithDelay(delayMs: number): Promise<void> {
const c = this._client._streamsClient();
try {
const res = await callUnary(c, (req, cb) => c.nak(req, cb), {
stream: this._stream,
consumer: this._consumer,
seq: this.sequence,
delayMs,
});
if (!res.success) throw serverError(res.resultCode, res.message);
} finally {
c.close();
}
}
/** Terminal acknowledgment — permanently drop regardless of maxDeliver. */
async term(): Promise<void> {
const c = this._client._streamsClient();
try {
const res = await callUnary(c, (req, cb) => c.term(req, cb), {
stream: this._stream,
consumer: this._consumer,
seq: this.sequence,
});
if (!res.success) throw serverError(res.resultCode, res.message);
} finally {
c.close();
}
}
/** Heartbeat — extend ack_wait without acking. */
async inProgress(): Promise<void> {
const c = this._client._streamsClient();
try {
const res = await callUnary(c, (req, cb) => c.inProgress(req, cb), {
stream: this._stream,
consumer: this._consumer,
seq: this.sequence,
});
if (!res.success) throw serverError(res.resultCode, res.message);
} finally {
c.close();
}
}
}
// ------------------------------------------------------------------
// Consumer
// ------------------------------------------------------------------
export class Consumer {
/** @internal */
constructor(
private readonly _client: WaymakerClient,
private readonly _stream: string,
readonly name: string
) {}
/** Parent stream name. */
get stream(): string { return this._stream; }
/**
* Open a push-mode subscription. Returns an `AsyncIterable<Message>`
* that yields delivered messages until the underlying RPC closes.
*
* ```ts
* for await (const msg of await consumer.messages()) {
* await msg.ack();
* }
* ```
*/
async messages(batchSize = 0): Promise<AsyncIterable<Message>> {
const c = this._client._streamsClient();
const rpcStream: grpc.ClientReadableStream<SubscribeEvent> = c.subscribe({
stream: this._stream,
consumer: this.name,
batchSize,
stopWhenEmpty: false,
});
const client = this._client;
const streamName = this._stream;
const consumerName = this.name;
async function* gen(): AsyncGenerator<Message> {
try {
for await (const event of streamToAsyncIter(rpcStream)) {
const ev = event as SubscribeEvent;
if (ev.message !== undefined) {
yield new Message(client, streamName, consumerName, ev.message);
} else if (ev.stopped !== undefined) {
if (ev.stopped.reason && ev.stopped.reason.length > 0) {
throw serverError("subscribe_stopped", ev.stopped.reason);
}
return;
}
}
} finally {
c.close();
}
}
return gen();
}
/**
* Pull-style: fetch up to `batchSize` messages right now.
* Returns the batch immediately (possibly empty).
*/
async fetch(batchSize: number): Promise<Message[]> {
const c = this._client._streamsClient();
try {
const res = await callUnary(c, (req, cb) => c.fetch(req, cb), {
stream: this._stream,
consumer: this.name,
batchSize,
});
if (!res.success) throw serverError(res.resultCode, res.message);
return res.messages.map(
(m: MessagePb) => new Message(this._client, this._stream, this.name, m)
);
} finally {
c.close();
}
}
}
// ------------------------------------------------------------------
// Stream handle
// ------------------------------------------------------------------
export class Stream {
readonly name: string;
/** @internal */
constructor(private readonly _client: WaymakerClient, name: string) {
this.name = name;
}
/** Publish a message. */
async publish(subject: string, payload: Uint8Array | Buffer | string): Promise<PublishAck> {
return this.publishWithHeaders(subject, [], payload);
}
/** Publish with explicit headers. */
async publishWithHeaders(
subject: string,
headers: Array<[string, string]>,
payload: Uint8Array | Buffer | string
): Promise<PublishAck> {
const c = this._client._streamsClient();
try {
const res = await callUnary(c, (req, cb) => c.publish(req, cb), {
stream: this.name,
subject,
payload: toBuffer(payload),
headers: headers.map(([key, value]) => ({ key, value })),
tsMs: 0,
expectedLastSeq: undefined,
});
if (!res.success) throw serverError(res.resultCode, res.message);
return { sequence: res.seq };
} finally {
c.close();
}
}
/** Create a new consumer. */
async createConsumer(config: ConsumerConfig): Promise<Consumer> {
const c = this._client._streamsClient();
try {
const res = await callUnary(c, (req, cb) => c.createConsumer(req, cb), {
stream: this.name,
config: buildConsumerConfigPb(config),
});
if (!res.success) throw serverError(res.resultCode, res.message);
return new Consumer(this._client, this.name, config.durableName);
} finally {
c.close();
}
}
/** Idempotent: create if not exists, otherwise return handle. */
async getOrCreateConsumer(config: ConsumerConfig): Promise<Consumer> {
try {
return await this.createConsumer(config);
} catch (e) {
if (isServerErrorCode(e, "already_exists")) {
return new Consumer(this._client, this.name, config.durableName);
}
throw e;
}
}
/** Return a handle to an existing consumer. */
async getConsumer(name: string): Promise<Consumer> {
const c = this._client._streamsClient();
try {
const res = await callUnary(c, (req, cb) => c.getConsumerInfo(req, cb), {
stream: this.name,
consumer: name,
});
if (!res.success) throw serverError(res.resultCode, res.message);
return new Consumer(this._client, this.name, name);
} finally {
c.close();
}
}
/** Delete a consumer. */
async deleteConsumer(name: string): Promise<void> {
const c = this._client._streamsClient();
try {
const res = await callUnary(c, (req, cb) => c.deleteConsumer(req, cb), {
stream: this.name,
consumer: name,
});
if (!res.success) throw serverError(res.resultCode, res.message);
} finally {
c.close();
}
}
/** Per-source tail status for this stream. */
async sourcesStatus(): Promise<SourceStatus[]> {
const c = this._client._streamsClient();
try {
const res = await callUnary(c, (req, cb) => c.getStreamInfo(req, cb), {
name: this.name,
});
if (!res.success) throw serverError(res.resultCode, res.message);
// eslint-disable-next-line @typescript-eslint/no-explicit-any
return (res.sourcesStatus ?? []).map((s: any) => ({
sourcingStream: this.name,
sourceStream: s.sourceStream as string,
lastSourcedSeq: s.lastSourcedSeq as number,
pulledTotal: s.pulledTotal as number,
lastError: s.lastError as string,
lastErrorTsMs: s.lastErrorTsMs as number,
}));
} finally {
c.close();
}
}
}
// ------------------------------------------------------------------
// Proto conversion helpers
// ------------------------------------------------------------------
function buildStreamConfigPb(config: StreamConfig) {
const retention = config.retention ?? RetentionPolicy.Limits;
let retentionPb: object;
if (retention === RetentionPolicy.Limits) {
retentionPb = {
limits: {
maxAgeMs: config.maxAgeMs,
maxMsgs: config.maxMessages,
maxBytes: config.maxBytes,
strictLimits: config.strictLimits ?? false,
},
};
} else if (retention === RetentionPolicy.WorkQueue) {
retentionPb = { workQueue: {} };
} else {
retentionPb = { interest: {} };
}
return {
name: config.name,
subjectsFilter: config.subjects ?? [],
retention: retentionPb,
blockSize: config.blockSize ?? 0,
maxMsgBytes: config.maxMessageSize ?? 0,
ephemeral: config.ephemeral ?? false,
maxMsgsPerSubject: config.maxMsgsPerSubject ?? 0,
sources: (config.sources ?? []).map((s) => ({
sourceStream: s.sourceStream,
filterSubject: s.filterSubject ?? "",
startSeq: s.startSeq ?? 0,
startTimeMs: s.startTimeMs ?? 0,
subjectTransform: s.subjectTransform
? { sourcePattern: s.subjectTransform.sourcePattern, destination: s.subjectTransform.destination }
: undefined,
maxInitialBackfill: s.maxInitialBackfill ?? 0,
onDrop: s.onDrop ?? OnDropPolicy.Halt,
dlqStream: s.dlqStream ?? "",
})),
};
}
function buildConsumerConfigPb(config: ConsumerConfig) {
const policy = config.deliverPolicy ?? DeliverPolicy.All;
let ty = 0; // DeliveryAll
let startSeq = 0;
let startTimeMs = 0;
if (policy === DeliverPolicy.New) {
ty = 3; // DeliveryByStartTime
startTimeMs = Date.now();
} else if (policy === DeliverPolicy.Last) {
ty = 2; // DeliveryLast
} else if (policy === DeliverPolicy.ByStartSequence) {
ty = 1; // DeliveryByStartSeq
startSeq = config.startSeq ?? 0;
} else if (policy === DeliverPolicy.ByStartTime) {
ty = 3; // DeliveryByStartTime
startTimeMs = config.startTimeMs ?? 0;
}
return {
name: config.durableName,
filterSubject: config.filterSubject ?? "",
deliveryPolicy: { type: ty, startSeq, startTimeMs },
ackWaitMs: config.ackWaitMs ?? 30_000,
maxDeliver: config.maxDeliver ?? 5,
deliverGroup: config.deliverGroup ?? "",
deadLetterSubject: config.deadLetterSubject ?? "",
};
}
function toBuffer(v: Uint8Array | Buffer | string): Buffer {
if (typeof v === "string") return Buffer.from(v);
if (Buffer.isBuffer(v)) return v;
return Buffer.from(v);
}
function isServerErrorCode(err: unknown, code: string): boolean {
return (
typeof err === "object" &&
err !== null &&
"kind" in err &&
(err as { kind: string }).kind === "server" &&
"code" in err &&
(err as { code: string }).code === code
);
}
// ------------------------------------------------------------------
// Client extension methods
// ------------------------------------------------------------------
declare module "./client" {
interface WaymakerClient {
createStream(config: StreamConfig): Promise<Stream>;
getStream(name: string): Promise<Stream>;
getOrCreateStream(config: StreamConfig): Promise<Stream>;
updateStream(name: string, update: StreamUpdate): Promise<void>;
deleteStream(name: string): Promise<void>;
getStreamSources(): Promise<SourceStatus[]>;
}
}
WaymakerClient.prototype.createStream = async function (
this: WaymakerClient,
config: StreamConfig
): Promise<Stream> {
const c = this._streamsClient();
try {
const res = await callUnary(c, (req, cb) => c.createStream(req, cb), {
config: buildStreamConfigPb(config),
});
if (!res.success) throw serverError(res.resultCode, res.message);
return new Stream(this, config.name);
} finally {
c.close();
}
};
WaymakerClient.prototype.getStream = async function (
this: WaymakerClient,
name: string
): Promise<Stream> {
const c = this._streamsClient();
try {
const res = await callUnary(c, (req, cb) => c.getStreamInfo(req, cb), { name });
if (!res.success) throw serverError(res.resultCode, res.message);
return new Stream(this, name);
} finally {
c.close();
}
};
WaymakerClient.prototype.getOrCreateStream = async function (
this: WaymakerClient,
config: StreamConfig
): Promise<Stream> {
try {
return await (this as WaymakerClient).getStream(config.name);
} catch (e) {
if (isServerErrorCode(e, "no_such_stream")) {
return (this as WaymakerClient).createStream(config);
}
throw e;
}
};
WaymakerClient.prototype.updateStream = async function (
this: WaymakerClient,
name: string,
update: StreamUpdate
): Promise<void> {
const c = this._streamsClient();
try {
const res = await callUnary(c, (req, cb) => c.updateStream(req, cb), {
name,
maxAgeMs: update.maxAgeMs,
maxMsgs: update.maxMsgs,
maxBytes: update.maxBytes,
maxMsgBytes: update.maxMsgBytes,
strictLimits: update.strictLimits,
});
if (!res.success) throw serverError(res.resultCode, res.message);
} finally {
c.close();
}
};
WaymakerClient.prototype.deleteStream = async function (
this: WaymakerClient,
name: string
): Promise<void> {
const c = this._streamsClient();
try {
const res = await callUnary(c, (req, cb) => c.deleteStream(req, cb), { name });
if (!res.success) throw serverError(res.resultCode, res.message);
} finally {
c.close();
}
};
WaymakerClient.prototype.getStreamSources = async function (
this: WaymakerClient
): Promise<SourceStatus[]> {
const c = this._streamsClient();
try {
const res = await callUnary(c, (req, cb) => c.getStreamSources(req, cb), {});
if (!res.success) throw serverError(res.resultCode, res.message);
// eslint-disable-next-line @typescript-eslint/no-explicit-any
return (res.entries ?? []).map((e: any) => ({
sourcingStream: e.sourcingStream as string,
sourceStream: e.sourceStream as string,
lastSourcedSeq: e.lastSourcedSeq as number,
pulledTotal: e.pulledTotal as number,
lastError: e.lastError as string,
lastErrorTsMs: e.lastErrorTsMs as number,
}));
} finally {
c.close();
}
};

18
ts/tsconfig.json Normal file
View File

@ -0,0 +1,18 @@
{
"compilerOptions": {
"target": "ES2020",
"module": "commonjs",
"moduleResolution": "node",
"lib": ["ES2020"],
"declaration": true,
"outDir": "dist",
"rootDir": "src",
"strict": true,
"esModuleInterop": true,
"skipLibCheck": true,
"forceConsistentCasingInFileNames": true,
"resolveJsonModule": true
},
"include": ["src/**/*.ts"],
"exclude": ["node_modules", "dist"]
}