A released lock could come back as a ghost. `Lock::unlock` signalled the background hold task and sent UnLock immediately; if the task had an idempotent re-acquire RPC in flight, `race_stop` dropped the future client-side but the server kept processing it — once UnLock had removed the original lease, the same request_id re-WON the key with a fresh id that no client held. On graylin prod that ghost writer sat on `shutdown:pod:<id>:barrier` (created ~1 ms after "released :barrier writer", then "lease_ttl pq has a removed node" and never reaped) and parked every proxied request on the pod for the full 120 s restart wait. - hold_loop no longer races the acquire RPC against stop; it lets the RPC finish (bounded by HOLD_RPC_TIMEOUT), then honours a level `stopped` flag and releases any lease it won while stopping. - unlock() sets `stopped`, notifies, JOINS the hold task (abort only as a backstop past every bound), then UnLocks the *current* id. - Drop sets the flag and notifies but no longer aborts mid-RPC. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MCwe5Sf5NU8Ei13oqTcesj |
||
|---|---|---|
| .. | ||
| src | ||
| Cargo.toml | ||
| README.md | ||
| build.rs | ||
README.md
waymaker-client (Rust)
Official Rust client for waymaker.
[dependencies]
waymaker-client = { git = "https://git.awesomike.com/pub/waymaker-client", tag = "v0.1.31" }
# (the crate lives in the rust/ subdir; cargo resolves it automatically)
Proto stubs are generated from ../proto at build time via build.rs — no
dependency on the waymaker server workspace.
Surfaces
lock— read/write locks, leader election, TTL leases. TheLockhandle keeps a background task that transparently re-binds its event stream across a primary bounce (reusing therequest_id) and exposes live state:lock.fence_token(),lock.watch()(awatch::Receiver<LockState>),lock.is_lost().LockState::generationdistinguishes a same-generation re-bind from a fresh transparent acquire, whileLockState::renewalexposes typed renewal success/failure and the authoritative renewed deadline. Pin the id/token/generation before protected work and reject any change.stream— JetStream-lite publish + pull/push consumers.kv— Put/Get/Create/Update(CAS)/Delete/Keys/History/Watch.cache— Redis-shape Hash / Set / Queue (collections).probabilistic— Bloom / HLL / CMS / TopK / t-digest.object— chunked object Put/Get.
Raw generated stubs are available under server / streams_server /
kv_server / collections_server / sketches_server / cache_server.
Leader election
use std::time::Duration;
use waymaker_client::{Client, lock};
let client = Client::connect("http://127.0.0.1:8818").await?;
let lock = client.acquire_lock("leader:reports", lock::Config {
max_wait: Duration::ZERO, // try-acquire
lease_ttl: Duration::from_secs(30),
scope: lock::Scope::Quorum, // Raft-replicated fence
..Default::default()
}).await?;
let _renewal = lock.spawn_renewal(Duration::from_secs(15));
// Pin one ownership generation; a transparent re-win cannot inherit work.
let mut state = lock.watch();
let admitted = state.borrow().clone();
loop {
let current = state.borrow().clone();
if current.lost
|| current.generation != admitted.generation
|| current.id != admitted.id
|| current.fence_token != admitted.fence_token
|| matches!(current.renewal, lock::RenewalState::Failed(_))
{
break;
}
do_fenced_write(admitted.fence_token).await?;
state.changed().await.ok();
}