Runs live, versioned Python application objects on one VM, with a suite of apps: projects, notes, tasks, files and more.

0 downloads

dbbasic-object-server/object_change_dispatch.py
Download zip
Clone
Clone with git
git clone https://gitstable.com/git/askrobots/dbbasic-object-server.git
HTTPS
https://gitstable.com/git/askrobots/dbbasic-object-server.git
SSH
ssh://git@gitstable.com:2222/askrobots/dbbasic-object-server.git

Cloning needs no sign-in. Pushing over HTTPS asks for your username and an access token. SSH needs your key under SSH keys.

danq9 signup: the person chooses the password and name on the confirm page 83c7aa6 2 weeks, 5 days ago History
object_change_dispatch.py · 13.0 KB · 315 lines
1"""Change-log event dispatch -- the real fix for docs/logic-decisions.md #9.
2
3The problem this closes: HANDLES-declared event handlers only fire from
4object_server's synchronous HTTP write path (see object_handlers.py and
5object_server._dispatch_event_handlers). A runner or an action that writes
6through object_records directly never passes through that path, so its
7write never triggers the reactions a handler would give an equivalent HTTP
8write. Doctrine #9 tracked two workarounds that paid for this gap by hand
9(action_apply_count composing its own inventory-variance journal;
10action_resolve_bank_line composing the NSF bounce reversal) and said the
11third instance must buy the real fix instead of a third workaround: a
12daemon pass that dispatches handlers from the record-change log, so a
13write reaches its reactions no matter which path wrote it -- the same
14shape object_daemon.process_notifications already uses to avoid depending
15on synchronous dispatch for notify_rules.
16
17Payload parity with the HTTP path (read this twice, it bit production
18once): object_server's dispatcher builds
19
20 {"event": "<collection>.record.<created|updated|deleted>",
21 "collection": collection, "record_id": record_id, "action": action}
22
23where ``action`` is the RAW verb from the write ("create"/"update"/
24"delete") and ``event`` uses the past-participle form. A handler that
25mixed the two up (checking ``action == "created"`` instead of mapping
26"create" -> "created" first) silently matched nothing and skipped every
27entry in production -- see the comment in
28packages/app-payments/objects/system/books.py's EVENT(). This module
29builds the exact same shape from the exact same raw fields a record-change
30entry already carries (object_record_changes stores the raw action
31untouched), so a handler cannot tell which path dispatched it.
32
33Double-dispatch, honestly: an HTTP write already dispatched its handlers
34synchronously before this pass ever sees the resulting change-log entry.
35This module has no way to know that happened -- object_record_changes
36does not record whether HANDLES already ran for an entry, and this file
37is not allowed to touch object_server.py to add that bookkeeping. So this
38pass does NOT prevent a handler from running twice across the two paths.
39What it promises instead is the same thing doctrine #7 already requires
40of every composer here: idempotency by provenance (a generated_from
41marker such as "payments/{id}" or "payments/{id}:bounced"), which is
42exactly how action_apply_count and action_resolve_bank_line stay correct
43under the two-workaround world already. At-least-once is therefore the
44right guarantee to promise here, not exactly-once -- a handler that isn't
45idempotent under redelivery is a bug in the handler, not in this
46dispatcher.
47
48What this module DOES guard against is re-firing the SAME (change_id,
49handler_object_id) pair across this pass's own polls -- a cursor file
50(same posture as process_notifications: first run stamps "now" and never
51backfills history) plus a small marker file recording pairs dispatched
52since the cursor last advanced. The marker exists because the cursor only
53advances once per successful batch; without it, a crash between
54dispatching entry 3 of 10 and persisting the new cursor would replay
55entries 1-10 (not just 4-10) on the next poll. The marker file is cleared
56every time the cursor successfully advances, so it never grows past one
57in-flight batch.
58
59Finally, this whole pass is opt-in: DBBASIC_ENABLE_CHANGE_DISPATCH is
60unset (OFF) by default, mirroring object_handlers.HANDLERS_ENABLED_ENV's
61posture that a dispatch behavior this new should be a deliberate operator
62act, not a silent default.
63"""
64from __future__ import annotations
65
66import os
67from datetime import datetime, timezone
68from pathlib import Path
69from typing import Any, Iterable
70
71import object_collections
72import object_handlers
73import object_record_changes
74from object_execution import ObjectExecutionRequest, execute_object
75from python_object_runtime import PythonObjectRuntime
76
77ENABLE_ENV = "DBBASIC_ENABLE_CHANGE_DISPATCH"
78_TRUE_VALUES = {"1", "true", "yes", "on"}
79
80CURSOR_FILE_NAME = ".change_dispatch_cursor"
81MARKER_FILE_NAME = ".change_dispatch_dispatched"
82
83DEFAULT_LIMIT = 200
84
85# One runtime, module-level, mirrors object_server.py's `_runtime =
86# PythonObjectRuntime()` and the test fixtures' `RUNTIME =
87# python_object_runtime.PythonObjectRuntime()` -- cheap to construct, no
88# per-call state worth re-creating on every poll.
89_RUNTIME = PythonObjectRuntime()
90
91
92def change_dispatch_enabled(*, base_dir: Path | str) -> bool:
93 """Return True when the change-log dispatch pass may run.
94
95 Env-only and OFF by default -- unlike object_notify's
96 notify_pass_enabled (a feature_flags row, default ON: a brownout lever
97 for a feature already adopted), this pass makes an installed handler
98 start reacting to writes that never used to reach it. That is a
99 behavior change for existing objects, not a lever on an existing one,
100 so it needs an explicit operator opt-in rather than a data default.
101 ``base_dir`` is accepted for signature symmetry with the other
102 ``*_enabled(*, base_dir)`` checks in this codebase; this particular
103 check does not read anything under it.
104 """
105 value = os.environ.get(ENABLE_ENV, "")
106 return value.strip().lower() in _TRUE_VALUES
107
108
109def dispatch_pending(
110 *,
111 base_dir: Path | str,
112 roots: Iterable[Path] | None = None,
113 limit: int = DEFAULT_LIMIT,
114) -> dict[str, Any]:
115 """Dispatch HANDLES handlers for record-change entries newer than the cursor.
116
117 Returns {"processed", "dispatched", "skipped", "errors", "results"}:
118
119 - processed: change-log entries examined this pass (across every
120 collection any installed handler declares interest in).
121 - dispatched: handler EVENT calls actually attempted (success or
122 failure both count -- this is "how much work happened", not "how
123 much succeeded").
124 - skipped: entries whose event had no matching handler, plus
125 (change_id, handler_id) pairs already recorded as dispatched by an
126 earlier, not-yet-committed pass over the same batch.
127 - errors: dispatched calls whose handler raised or returned a
128 non-ok ObjectExecutionResult.
129 - results: one dict per attempted dispatch (change_id, collection,
130 record_id, action, event, handler_id, ok, error).
131
132 Gracefully returns the all-zero shape (never raises) when: the pass is
133 disabled, no installed object declares a HANDLES entry shaped like
134 "<collection>.record.<created|updated|deleted>", or a collection has
135 no change log yet. One raising handler, or one unreadable change-log
136 file, never stops the rest of the batch -- same posture as
137 object_daemon.process_notifications.
138 """
139 base_dir = Path(base_dir)
140 result: dict[str, Any] = {
141 "processed": 0,
142 "dispatched": 0,
143 "skipped": 0,
144 "errors": 0,
145 "results": [],
146 }
147
148 if not change_dispatch_enabled(base_dir=base_dir):
149 return result
150
151 try:
152 index = object_handlers.build_index(roots)
153 except Exception:
154 # Source discovery is best-effort here -- a broken package source
155 # elsewhere must not stop this pass either.
156 return result
157
158 collections = _collections_with_record_handlers(index)
159 if not collections:
160 return result
161
162 cursor_path = base_dir / CURSOR_FILE_NAME
163 try:
164 cursor = cursor_path.read_text(encoding="utf-8").strip()
165 except OSError:
166 cursor = ""
167
168 if not cursor:
169 # First run: stamp "now" and dispatch only FUTURE changes. A brand
170 # new install (or a flag flipped on long after go-live) must never
171 # replay years of history through every handler the moment it
172 # turns on.
173 now_iso = datetime.now(timezone.utc).isoformat()
174 cursor_path.parent.mkdir(parents=True, exist_ok=True)
175 cursor_path.write_text(now_iso, encoding="utf-8")
176 return result
177
178 fresh: list[dict[str, Any]] = []
179 for collection in collections:
180 try:
181 payload = object_record_changes.list_record_changes(
182 collection, base_dir=base_dir, limit=limit
183 )
184 except (object_collections.InvalidCollectionNameError, OSError, ValueError):
185 continue
186 for change in payload.get("changes") or []:
187 ts = str(change.get("timestamp") or "")
188 if ts > cursor:
189 fresh.append(change)
190 fresh.sort(key=lambda c: str(c.get("timestamp") or ""))
191
192 result["processed"] = len(fresh)
193 if not fresh:
194 return result
195
196 marker_path = base_dir / MARKER_FILE_NAME
197 dispatched_pairs = _load_markers(marker_path)
198
199 max_ts = cursor
200 for change in fresh:
201 # Per-entry isolation: a malformed change or a handler exception
202 # must never stop the rest of the batch or prevent the cursor from
203 # advancing past everything that DID work.
204 try:
205 ts = str(change.get("timestamp") or "")
206 if ts > max_ts:
207 max_ts = ts
208
209 collection = str(change.get("collection") or "")
210 record_id = str(change.get("record_id") or "")
211 action = str(change.get("action") or "")
212 change_id = str(change.get("change_id") or "")
213
214 event = object_handlers.event_name(collection, action)
215 handler_ids = index.get(event) if event else None
216 if not handler_ids:
217 result["skipped"] += 1
218 continue
219
220 # Same signal-shaped payload object_server._dispatch_event_handlers
221 # builds: no record body, RAW action, participle event name.
222 event_payload = {
223 "event": event,
224 "collection": collection,
225 "record_id": record_id,
226 "action": action,
227 }
228
229 for handler_id in handler_ids:
230 pair_key = f"{change_id}\t{handler_id}"
231 if pair_key in dispatched_pairs:
232 result["skipped"] += 1
233 continue
234
235 entry: dict[str, Any] = {
236 "change_id": change_id,
237 "collection": collection,
238 "record_id": record_id,
239 "action": action,
240 "event": event,
241 "handler_id": handler_id,
242 "ok": False,
243 "error": None,
244 }
245 try:
246 request = ObjectExecutionRequest(
247 object_id=handler_id,
248 method="EVENT",
249 payload=event_payload,
250 correlation_id=change.get("correlation_id") or None,
251 )
252 exec_result = execute_object(_RUNTIME, request, roots)
253 entry["ok"] = bool(exec_result.ok)
254 if not exec_result.ok and exec_result.error is not None:
255 entry["error"] = exec_result.error.message
256 except Exception as exc: # noqa: BLE001 -- one bad handler must not stop others
257 entry["ok"] = False
258 entry["error"] = str(exc)[:200]
259
260 result["dispatched"] += 1
261 if not entry["ok"]:
262 result["errors"] += 1
263 result["results"].append(entry)
264 dispatched_pairs.add(pair_key)
265 _append_marker(marker_path, pair_key)
266 except Exception: # noqa: BLE001 -- one bad change must not stop the batch
267 continue
268
269 if max_ts != cursor:
270 cursor_path.write_text(max_ts, encoding="utf-8")
271 # The whole batch between the old and new cursor just committed;
272 # nothing in it can ever be re-read (list_record_changes only
273 # returns entries newer than the cursor), so the markers guarding
274 # against a mid-batch crash are no longer needed.
275 _clear_markers(marker_path)
276
277 return result
278
279
280def _collections_with_record_handlers(index: dict[str, list[str]]) -> list[str]:
281 """Return the sorted set of collections any HANDLES entry names.
282
283 Only "<collection>.record.<created|updated|deleted>" shaped events are
284 record-change events this pass can act on; anything else in HANDLES
285 (a future, non-record event kind) is silently not our concern here.
286 """
287 participles = {"created", "updated", "deleted"}
288 collections: set[str] = set()
289 for event in index:
290 parts = event.split(".record.", 1)
291 if len(parts) != 2 or parts[1] not in participles or not parts[0]:
292 continue
293 collections.add(parts[0])
294 return sorted(collections)
295
296
297def _load_markers(path: Path) -> set[str]:
298 try:
299 text = path.read_text(encoding="utf-8")
300 except OSError:
301 return set()
302 return {line for line in text.splitlines() if line.strip()}
303
304
305def _append_marker(path: Path, key: str) -> None:
306 path.parent.mkdir(parents=True, exist_ok=True)
307 with path.open("a", encoding="utf-8") as handle:
308 handle.write(key + "\n")
309
310
311def _clear_markers(path: Path) -> None:
312 try:
313 path.write_text("", encoding="utf-8")
314 except OSError:
315 pass