askrobots / dbbasic-object-server
Public
Runs live, versioned Python application objects on one VM, with a suite of apps: projects, notes, tasks, files and more.
danq9
signup: the person chooses the password and name on the confirm page
83c7aa6
2 weeks, 5 days ago
History
mail_sync.py · 25.6 KB · 593 lines
| 1 | """system_mail_sync -- copy new mail from each person's IMAP mailboxes into |
| 2 | their app-messaging inbox, without touching the mailbox. |
| 3 | |
| 4 | EVENT/POST: the scheduled pass declared in this package's `schedules`, |
| 5 | every two minutes. (The home-page count of failing accounts is its own |
| 6 | object, system_mail_sync_attention, as every attention provider is.) |
| 7 | Everything that can be decided without a server -- parsing, |
| 8 | dedupe, threading, where the cursor goes, what to download -- is |
| 9 | object_mail's, which is pure and tested on its own. This object is the I/O: |
| 10 | it opens the connection, asks, and writes rows. |
| 11 | |
| 12 | READ-ONLY TOWARDS THE MAILBOX. Folders are opened with EXAMINE (imaplib's |
| 13 | ``select(readonly=True)``) and messages fetched with BODY.PEEK[], so the |
| 14 | sync can never set \\Seen, and ``is_read`` on each copied message is the |
| 15 | server's own \\Seen flag at the moment of copying. The predecessor searched |
| 16 | UNSEEN and fetched RFC822, which marked every synced message read in the |
| 17 | person's own mail program and never copied anything they had read |
| 18 | elsewhere first. Messages are chosen by UID, never by flag. The sync sends |
| 19 | nothing, stores no link it does not open, and opens none. |
| 20 | |
| 21 | BOUNDED, because it runs inside the daemon's loop, between the other |
| 22 | passes: at most BATCH_LIMIT messages and TICK_BYTE_BUDGET bytes per account |
| 23 | per pass, socket timeouts on every connection, messages over |
| 24 | MAX_MESSAGE_BYTES copied as headers only (the body then says so), a first |
| 25 | sync that takes the newest FIRST_WINDOW messages of the last |
| 26 | FIRST_WINDOW_DAYS days rather than the whole history, and no new account |
| 27 | started after TICK_SECONDS. A backlog drains over several passes. |
| 28 | |
| 29 | AT LEAST ONCE, MADE SAFE. The folder cursor (mail_sync_state.last_uid) moves |
| 30 | only after the rows for those messages are written. A pass that dies |
| 31 | between the two repeats that work next time, and the repeat writes nothing |
| 32 | new: every row id is object_mail.stable_id(owner, account, dedupe key), so |
| 33 | the second create of the same message is refused as a duplicate. |
| 34 | |
| 35 | QUIET WHEN NOTHING HAPPENS. An account is checked read-only first (EXAMINE |
| 36 | answers UIDNEXT); only when there is something to copy does the pass claim |
| 37 | the account -- a compare-and-set on its row with expected_rev, so a manual |
| 38 | run and the scheduled one cannot both write -- and only then are rows |
| 39 | written. An idle account refreshes last_synced_at at most hourly. Without |
| 40 | that, two account writes every two minutes would grow mail_accounts' change |
| 41 | log by ~1,400 entries a day per account for nothing (AGENTS.md: a writer |
| 42 | says how it is bounded). |
| 43 | |
| 44 | FAILURES ARE RECORDED, NEVER SWALLOWED. Two kinds of silence are told |
| 45 | apart, the way system_tracking_poll does: |
| 46 | |
| 47 | not set up the feature is off, the account has no password in |
| 48 | the vault, no owner, or an id that cannot name a |
| 49 | vault entry. Nothing is counted against the account; |
| 50 | the pass says why in its report. |
| 51 | this account failed a refused login, an unreachable or refused host, a |
| 52 | missing folder, a message that would not store. |
| 53 | sync_attempts goes up, sync_error says what happened, |
| 54 | and the account waits longer between tries |
| 55 | (object_mail.account_due) so a wrong password is not |
| 56 | tried every two minutes until the provider locks the |
| 57 | mailbox. At object_mail.ATTENTION_AFTER failures |
| 58 | in a row it is |
| 59 | counted on the home page. |
| 60 | |
| 61 | The password is read from the owner's identity vault under |
| 62 | ``imap-<account id>``, resolved by the ACCOUNT ROW's owner_id and never by |
| 63 | anything a caller passed, used for the login, and never written, logged or |
| 64 | returned; error text from the server is scrubbed of it before it is stored. |
| 65 | """ |
| 66 | |
| 67 | import imaplib |
| 68 | import os |
| 69 | import socket |
| 70 | import ssl |
| 71 | import time |
| 72 | from datetime import datetime, timedelta, timezone |
| 73 | |
| 74 | import object_feature_flags |
| 75 | import object_mail |
| 76 | import object_records |
| 77 | import object_schemas |
| 78 | import object_service_keys |
| 79 | |
| 80 | ACTOR = "system_mail_sync" |
| 81 | |
| 82 | ACCOUNTS = "mail_accounts" |
| 83 | STATE = "mail_sync_state" |
| 84 | SYNCED = "mail_synced_messages" |
| 85 | MESSAGES = "messages" |
| 86 | THREADS = "message_threads" |
| 87 | RECIPIENTS = "message_recipients" |
| 88 | |
| 89 | BATCH_LIMIT = 50 |
| 90 | FIRST_WINDOW = 200 |
| 91 | FIRST_WINDOW_DAYS = 30 |
| 92 | MAX_MESSAGE_BYTES = 10 * 1024 * 1024 |
| 93 | TICK_BYTE_BUDGET = 25 * 1024 * 1024 |
| 94 | SOCKET_TIMEOUT_SECONDS = 30 |
| 95 | TICK_SECONDS = 60 |
| 96 | CLAIM_TTL_SECONDS = 15 * 60 |
| 97 | RETRY_BASE_SECONDS = 120 |
| 98 | RETRY_MAX_SECONDS = 30 * 60 |
| 99 | HEARTBEAT_SECONDS = 60 * 60 |
| 100 | MAX_ERROR = 500 |
| 101 | |
| 102 | _DEFAULT_PORTS = {"ssl": 993, "starttls": 143, "plain": 143} |
| 103 | |
| 104 | |
| 105 | class SyncError(Exception): |
| 106 | """A failure this account's owner needs to read; the text is stored.""" |
| 107 | |
| 108 | |
| 109 | def _base_dir(): |
| 110 | return os.environ.get("DBBASIC_DATA_DIR", "data") |
| 111 | |
| 112 | |
| 113 | def _text(value): |
| 114 | return str(value if value is not None else "").strip() |
| 115 | |
| 116 | |
| 117 | def _int(value, default=0): |
| 118 | try: |
| 119 | return int(_text(value) or default) |
| 120 | except (TypeError, ValueError): |
| 121 | return default |
| 122 | |
| 123 | |
| 124 | def _now(): |
| 125 | return datetime.now(timezone.utc) |
| 126 | |
| 127 | |
| 128 | def _stamp(moment): |
| 129 | return object_mail.format_timestamp(moment) |
| 130 | |
| 131 | |
| 132 | def _collection_exists(base, name): |
| 133 | if object_records.collection_has_records(name, base_dir=base): |
| 134 | return True |
| 135 | try: |
| 136 | object_schemas.get_schema(name, base_dir=base) |
| 137 | return True |
| 138 | except (object_schemas.SchemaNotFoundError, OSError, ValueError): |
| 139 | return False |
| 140 | |
| 141 | |
| 142 | def _get(base, collection, record_id): |
| 143 | try: |
| 144 | return object_records.get_collection_record(collection, record_id, base_dir=base) |
| 145 | except object_records.RecordNotFoundError: |
| 146 | return None |
| 147 | |
| 148 | |
| 149 | def _create(base, collection, row): |
| 150 | """True when written, False when a row with this id already exists -- |
| 151 | the replay of a batch a previous pass did not finish.""" |
| 152 | try: |
| 153 | object_records.create_collection_record(collection, row, base_dir=base, actor=ACTOR) |
| 154 | return True |
| 155 | except object_records.DuplicateRecordIdError: |
| 156 | return False |
| 157 | |
| 158 | |
| 159 | def _scrub(text, secrets): |
| 160 | text = _text(text) |
| 161 | for secret in secrets: |
| 162 | if secret and len(secret) >= 3: |
| 163 | text = text.replace(secret, "***") |
| 164 | return text[:MAX_ERROR] |
| 165 | |
| 166 | |
| 167 | # --- the connection ------------------------------------------------------------ |
| 168 | |
| 169 | def _connect(account, password): |
| 170 | host = _text(account.get("host")) |
| 171 | security = _text(account.get("security")) or "ssl" |
| 172 | if security not in _DEFAULT_PORTS: |
| 173 | raise SyncError(f"security must be ssl, starttls or plain, not {security!r}") |
| 174 | port = _int(account.get("port")) or _DEFAULT_PORTS[security] |
| 175 | username = _text(account.get("username")) or _text(account.get("email_address")) |
| 176 | if not host: |
| 177 | raise SyncError("the account has no IMAP server") |
| 178 | if any(char in username for char in "\r\n\x00"): |
| 179 | # imaplib quotes a login but does not refuse a line break, which |
| 180 | # would end the command and start another on the mailbox. |
| 181 | raise SyncError("the user name cannot contain a line break") |
| 182 | |
| 183 | try: |
| 184 | infos = socket.getaddrinfo(host, port, type=socket.SOCK_STREAM) |
| 185 | except (socket.gaierror, UnicodeError, OSError) as exc: |
| 186 | raise SyncError(f"could not find the server {host!r}: {exc}") from exc |
| 187 | refusal = object_mail.host_refusal( |
| 188 | host, sorted({info[4][0] for info in infos}), security=security, |
| 189 | allow_private=object_feature_flags.env_opt_in(object_mail.ALLOW_PRIVATE_HOSTS_ENV)) |
| 190 | if refusal: |
| 191 | raise SyncError(refusal) |
| 192 | |
| 193 | conn = None |
| 194 | try: |
| 195 | if security == "ssl": |
| 196 | conn = imaplib.IMAP4_SSL(host, port, ssl_context=ssl.create_default_context(), |
| 197 | timeout=SOCKET_TIMEOUT_SECONDS) |
| 198 | else: |
| 199 | conn = imaplib.IMAP4(host, port, timeout=SOCKET_TIMEOUT_SECONDS) |
| 200 | if security == "starttls": |
| 201 | conn.starttls(ssl_context=ssl.create_default_context()) |
| 202 | except (OSError, imaplib.IMAP4.error, ssl.SSLError) as exc: |
| 203 | if conn is not None: |
| 204 | _logout(conn) |
| 205 | raise SyncError(f"could not connect to {host}:{port} ({security}): {exc}") from exc |
| 206 | |
| 207 | try: |
| 208 | conn.login(username, password) |
| 209 | except UnicodeError as exc: |
| 210 | _logout(conn) |
| 211 | raise SyncError("the user name or password has characters an IMAP login " |
| 212 | "cannot send (only plain ASCII is allowed)") from exc |
| 213 | except imaplib.IMAP4.error as exc: |
| 214 | _logout(conn) |
| 215 | raise SyncError(f"the server refused the login: {_scrub(exc, [password])}") from exc |
| 216 | return conn |
| 217 | |
| 218 | |
| 219 | def _logout(conn): |
| 220 | try: |
| 221 | conn.logout() |
| 222 | except Exception: # noqa: BLE001 -- the work is done or already failed; closing is best effort |
| 223 | return |
| 224 | |
| 225 | |
| 226 | def _ok(typ, data, what): |
| 227 | if typ != "OK": |
| 228 | detail = b" ".join(d for d in (data or []) if isinstance(d, bytes)).decode("utf-8", "replace") |
| 229 | raise SyncError(f"{what} failed: {detail or typ}") |
| 230 | return data |
| 231 | |
| 232 | |
| 233 | def _examine(conn, folder): |
| 234 | """Open a folder read-only; (uidvalidity, uidnext or 0).""" |
| 235 | try: |
| 236 | quoted = object_mail.quote_mailbox(folder) |
| 237 | except ValueError as exc: |
| 238 | raise SyncError(f"the folder name {folder!r} cannot be used: {exc}") from exc |
| 239 | typ, data = conn.select(quoted, readonly=True) |
| 240 | _ok(typ, data, f"opening the folder {folder!r}") |
| 241 | uidvalidity = object_mail.parse_search(conn.response("UIDVALIDITY")[1]) |
| 242 | uidnext = object_mail.parse_search(conn.response("UIDNEXT")[1]) |
| 243 | if not uidvalidity: |
| 244 | raise SyncError(f"the server gave no UIDVALIDITY for {folder!r}") |
| 245 | return str(uidvalidity[0]), (uidnext[0] if uidnext else 0) |
| 246 | |
| 247 | |
| 248 | # --- one account ----------------------------------------------------------------- |
| 249 | |
| 250 | class _Pass: |
| 251 | """One account's sync: what to check, then (only if needed) the writes.""" |
| 252 | |
| 253 | def __init__(self, base, account, password, now): |
| 254 | self.base = base |
| 255 | self.account = account |
| 256 | self.password = password |
| 257 | self.now = now |
| 258 | self.account_id = account["id"] |
| 259 | self.owner_id = _text(account.get("owner_id")) |
| 260 | self.address = _text(account.get("email_address")) |
| 261 | self.imported = 0 |
| 262 | self.duplicates = 0 |
| 263 | self.errors = [] |
| 264 | self.discovered = None |
| 265 | self._seen = None |
| 266 | self._index = None |
| 267 | |
| 268 | # -- read-only check ------------------------------------------------------- |
| 269 | |
| 270 | def check(self, conn): |
| 271 | """Folders with something to do, found without writing anything.""" |
| 272 | names = object_mail.split_folders(self.account.get("folders")) |
| 273 | entries = None |
| 274 | if not names: |
| 275 | entries = object_mail.parse_list(_ok(*conn.list(), "LIST")) |
| 276 | chosen = object_mail.choose_folders(entries) |
| 277 | self.discovered = ", ".join(name for name, _ in chosen) |
| 278 | names = [name for name, _ in chosen] |
| 279 | |
| 280 | work = [] |
| 281 | for name in names: |
| 282 | state_id = object_mail.stable_id("folder", self.account_id, name) |
| 283 | state = _get(self.base, STATE, state_id) |
| 284 | try: |
| 285 | uidvalidity, uidnext = _examine(conn, name) |
| 286 | except SyncError as exc: |
| 287 | self.errors.append(str(exc)) |
| 288 | continue |
| 289 | decision = object_mail.plan_folder(state, uidvalidity=uidvalidity) |
| 290 | last_uid = _int((state or {}).get("last_uid")) |
| 291 | if decision == "continue" and uidnext and uidnext - 1 <= last_uid: |
| 292 | continue |
| 293 | role = (state or {}).get("role") |
| 294 | if not role: |
| 295 | if entries is None: |
| 296 | entries = object_mail.parse_list(_ok(*conn.list(), "LIST")) |
| 297 | role = object_mail.folder_role(name, entries) |
| 298 | work.append({"folder": name, "state_id": state_id, "state": state, |
| 299 | "decision": decision, "uidvalidity": uidvalidity, |
| 300 | "uidnext": uidnext, "role": role}) |
| 301 | return work |
| 302 | |
| 303 | # -- the writes -------------------------------------------------------------- |
| 304 | |
| 305 | def run(self, conn, work): |
| 306 | budget = BATCH_LIMIT |
| 307 | bytes_left = TICK_BYTE_BUDGET |
| 308 | for item in work: |
| 309 | if budget <= 0 or bytes_left <= 0: |
| 310 | break |
| 311 | try: |
| 312 | done, spent = self._folder(conn, item, budget, bytes_left) |
| 313 | except SyncError as exc: |
| 314 | self.errors.append(str(exc)) |
| 315 | continue |
| 316 | except (imaplib.IMAP4.error, OSError) as exc: |
| 317 | # The connection itself failed; later folders cannot work. |
| 318 | self.errors.append(f"{item['folder']}: {_scrub(exc, [self.password])}") |
| 319 | break |
| 320 | budget -= done |
| 321 | bytes_left -= spent |
| 322 | |
| 323 | def _folder(self, conn, item, budget, bytes_left): |
| 324 | folder = item["folder"] |
| 325 | uidvalidity, uidnext = _examine(conn, folder) |
| 326 | if uidvalidity != item["uidvalidity"]: |
| 327 | raise SyncError(f"{folder!r} changed UIDVALIDITY while it was being read; " |
| 328 | f"the next pass starts it again") |
| 329 | state = item["state"] |
| 330 | last_uid = _int((state or {}).get("last_uid")) |
| 331 | |
| 332 | if item["decision"] in ("fresh", "reset"): |
| 333 | since = object_mail.imap_date(self.now - timedelta(days=FIRST_WINDOW_DAYS)) |
| 334 | window = object_mail.parse_search(_ok(*conn.uid("SEARCH", "SINCE", since), "SEARCH")) |
| 335 | highest = uidnext - 1 if uidnext else max( |
| 336 | object_mail.parse_search(_ok(*conn.uid("SEARCH", "ALL"), "SEARCH")) or [0]) |
| 337 | last_uid = object_mail.window_baseline(window, first_window=FIRST_WINDOW, |
| 338 | highest_uid=highest) |
| 339 | row = {"account_id": self.account_id, "folder": folder, "role": item["role"], |
| 340 | "uidvalidity": uidvalidity, "last_uid": str(last_uid), |
| 341 | "owner_id": self.owner_id} |
| 342 | if state is None: |
| 343 | _create(self.base, STATE, {"id": item["state_id"], **row}) |
| 344 | else: |
| 345 | object_records.update_collection_record( |
| 346 | STATE, item["state_id"], row, base_dir=self.base, actor=ACTOR) |
| 347 | |
| 348 | found = object_mail.parse_search( |
| 349 | _ok(*conn.uid("SEARCH", "UID", f"{last_uid + 1}:*"), "SEARCH")) |
| 350 | uids = object_mail.next_uids(found, last_uid=last_uid, limit=budget) |
| 351 | if not uids: |
| 352 | return 0, 0 |
| 353 | metas = object_mail.parse_fetch_meta(_ok(*conn.uid( |
| 354 | "FETCH", ",".join(str(uid) for uid in uids), |
| 355 | "(UID FLAGS RFC822.SIZE INTERNALDATE)"), "FETCH")) |
| 356 | by_uid = {meta["uid"]: meta for meta in metas} |
| 357 | plan = object_mail.plan_downloads(metas, max_message_bytes=MAX_MESSAGE_BYTES, |
| 358 | byte_budget=bytes_left) |
| 359 | |
| 360 | advanced = last_uid |
| 361 | done = spent = 0 |
| 362 | try: |
| 363 | for uid, mode in plan: |
| 364 | meta = by_uid[uid] |
| 365 | part = "BODY.PEEK[]" if mode == "full" else "BODY.PEEK[HEADER]" |
| 366 | raw = object_mail.fetch_literal(_ok(*conn.uid("FETCH", str(uid), f"(UID {part})"), |
| 367 | "FETCH")) |
| 368 | if raw is None: |
| 369 | raise SyncError(f"{folder!r}: the server sent no message for UID {uid}") |
| 370 | parsed = object_mail.parse_message( |
| 371 | raw, flags=meta["flags"], internaldate=meta["internaldate"], |
| 372 | header_only_size=meta["size"] if mode == "header" else None) |
| 373 | try: |
| 374 | self._store(parsed, folder=folder, uid=uid, uidvalidity=uidvalidity, |
| 375 | role=item["role"]) |
| 376 | except (object_records.InvalidRecordPayloadError, ValueError, LookupError) as exc: |
| 377 | raise SyncError(f"{folder!r}: UID {uid} could not be stored: " |
| 378 | f"{str(exc)[:200]}") from exc |
| 379 | advanced = uid |
| 380 | done += 1 |
| 381 | spent += meta["size"] if mode == "full" else 0 |
| 382 | finally: |
| 383 | # The cursor moves to the last message whose rows are written, |
| 384 | # and no further, whatever stopped the loop. |
| 385 | if advanced > last_uid: |
| 386 | object_records.update_collection_record( |
| 387 | STATE, item["state_id"], {"last_uid": str(advanced)}, |
| 388 | base_dir=self.base, actor=ACTOR) |
| 389 | return done, spent |
| 390 | |
| 391 | def _prepare(self): |
| 392 | """The dedupe keys and thread index, built once per pass and only |
| 393 | when there is something to import -- they are the expensive reads.""" |
| 394 | if self._seen is not None: |
| 395 | return |
| 396 | synced = object_records.read_collection_records(SYNCED, base_dir=self.base) |
| 397 | own = [row for row in synced if row.get("owner_id") == self.owner_id] |
| 398 | self._seen = {row.get("dedupe_key") for row in own |
| 399 | if row.get("account_id") == self.account_id} |
| 400 | messages = object_records.read_collection_records(MESSAGES, base_dir=self.base) |
| 401 | self._index = object_mail.ThreadIndex.from_rows(self.owner_id, messages, own) |
| 402 | |
| 403 | def _store(self, parsed, *, folder, uid, uidvalidity, role): |
| 404 | self._prepare() |
| 405 | key = object_mail.dedupe_key(parsed) |
| 406 | if key in self._seen: |
| 407 | self.duplicates += 1 |
| 408 | return |
| 409 | owner, account_id = self.owner_id, self.account_id |
| 410 | direction = object_mail.direction_for(parsed, account_address=self.address, |
| 411 | folder_role=role) |
| 412 | message_id = object_mail.stable_id("message", owner, account_id, key) |
| 413 | thread_id = object_mail.match_thread(parsed, self._index) |
| 414 | joined = thread_id is not None |
| 415 | if not joined: |
| 416 | thread_id = object_mail.stable_id("thread", owner, account_id, key) |
| 417 | _create(self.base, THREADS, object_mail.new_thread_row( |
| 418 | parsed, owner_id=owner, thread_id=thread_id, direction=direction, |
| 419 | account_address=self.address)) |
| 420 | |
| 421 | row, recipients = object_mail.message_rows( |
| 422 | parsed, owner_id=owner, message_id=message_id, thread_id=thread_id, |
| 423 | direction=direction) |
| 424 | created = _create(self.base, MESSAGES, row) |
| 425 | for recipient in recipients: |
| 426 | _create(self.base, RECIPIENTS, recipient) |
| 427 | _create(self.base, SYNCED, object_mail.synced_row( |
| 428 | parsed, row_id=object_mail.stable_id("synced", owner, account_id, key), |
| 429 | owner_id=owner, account_id=account_id, folder=folder, uid=uid, |
| 430 | uidvalidity=uidvalidity, message_id=message_id, key=key)) |
| 431 | if created and joined: |
| 432 | thread = _get(self.base, THREADS, thread_id) |
| 433 | if thread is not None: |
| 434 | changes = object_mail.thread_changes(thread, parsed, direction=direction, |
| 435 | account_address=self.address) |
| 436 | object_records.update_collection_record( |
| 437 | THREADS, thread_id, changes, base_dir=self.base, actor=ACTOR) |
| 438 | self._index.add(parsed, thread_id) |
| 439 | self._seen.add(key) |
| 440 | if created: |
| 441 | self.imported += 1 |
| 442 | else: |
| 443 | self.duplicates += 1 |
| 444 | |
| 445 | |
| 446 | # --- the pass ------------------------------------------------------------------------ |
| 447 | |
| 448 | def _finish(base, account, changes, *, expected_rev=None): |
| 449 | try: |
| 450 | object_records.update_collection_record( |
| 451 | ACCOUNTS, account["id"], changes, base_dir=base, actor=ACTOR, |
| 452 | preserve_read_only=True, expected_rev=expected_rev) |
| 453 | return True |
| 454 | except object_records.VersionConflictError: |
| 455 | return False |
| 456 | |
| 457 | |
| 458 | def _failed(base, account, message, now): |
| 459 | attempts = _int(account.get("sync_attempts")) + 1 |
| 460 | _finish(base, account, {"sync_attempts": str(attempts), "sync_error": message[:MAX_ERROR], |
| 461 | "sync_finished_at": _stamp(now)}) |
| 462 | return attempts |
| 463 | |
| 464 | |
| 465 | def _sync_one(base, account, now): |
| 466 | """One account's outcome, as a report entry.""" |
| 467 | entry = {"account": account["id"]} |
| 468 | owner_id = _text(account.get("owner_id")) |
| 469 | if not owner_id: |
| 470 | entry["declined"] = "the account has no owner, so there is no vault to read a password from" |
| 471 | return entry |
| 472 | service = object_mail.password_service(account["id"]) |
| 473 | if not object_service_keys.validate_service_name(service): |
| 474 | entry["declined"] = (f"the account id cannot name a vault entry ({service!r} is not a " |
| 475 | f"valid service name); recreate the account from /mail/accounts") |
| 476 | return entry |
| 477 | password = object_service_keys.get_service_key(owner_id, service, base_dir=base) |
| 478 | if not password: |
| 479 | entry["declined"] = f"no password set (the owner's vault has no {service!r})" |
| 480 | return entry |
| 481 | |
| 482 | rev = object_records.compute_record_rev(account) |
| 483 | conn = None |
| 484 | try: |
| 485 | conn = _connect(account, password) |
| 486 | sync = _Pass(base, account, password, now) |
| 487 | work = sync.check(conn) |
| 488 | if not work and sync.discovered is None: |
| 489 | # Nothing to copy. Write only what changed: a recovery clears the |
| 490 | # failure count; otherwise last_synced_at is refreshed hourly. |
| 491 | if sync.errors: |
| 492 | raise SyncError("; ".join(sync.errors)) |
| 493 | last = object_mail.parse_timestamp(account.get("last_synced_at")) |
| 494 | recovering = _int(account.get("sync_attempts")) or _text(account.get("sync_error")) |
| 495 | if recovering or last is None or (now - last).total_seconds() >= HEARTBEAT_SECONDS: |
| 496 | _finish(base, account, {"last_synced_at": _stamp(now), "sync_finished_at": _stamp(now), |
| 497 | "sync_attempts": "0", "sync_error": ""}, expected_rev=rev) |
| 498 | entry["up_to_date"] = True |
| 499 | return entry |
| 500 | |
| 501 | # Something to write: claim the account first. expected_rev is the |
| 502 | # row as this pass read it, so a second pass (or an edit) that got |
| 503 | # there first makes this one stand down instead of writing beside it. |
| 504 | if not _finish(base, account, {"sync_started_at": _stamp(now)}, expected_rev=rev): |
| 505 | entry["declined"] = "the account changed while this pass read it; the next pass will try" |
| 506 | return entry |
| 507 | try: |
| 508 | sync.run(conn, work) |
| 509 | except Exception as exc: # noqa: BLE001 -- recorded on the account below, not hidden |
| 510 | sync.errors.append(_scrub(f"{type(exc).__name__}: {exc}", [password])) |
| 511 | entry["imported"] = sync.imported |
| 512 | if sync.duplicates: |
| 513 | entry["already_copied"] = sync.duplicates |
| 514 | changes = {"sync_finished_at": _stamp(_now())} |
| 515 | if sync.discovered is not None: |
| 516 | changes["folders"] = sync.discovered |
| 517 | if sync.errors: |
| 518 | message = _scrub("; ".join(sync.errors), [password]) |
| 519 | changes.update({"sync_attempts": str(_int(account.get("sync_attempts")) + 1), |
| 520 | "sync_error": message}) |
| 521 | entry["error"] = message |
| 522 | else: |
| 523 | changes.update({"last_synced_at": _stamp(now), "sync_attempts": "0", "sync_error": ""}) |
| 524 | _finish(base, account, changes) |
| 525 | return entry |
| 526 | except SyncError as exc: |
| 527 | message = _scrub(exc, [password]) |
| 528 | entry["error"] = message |
| 529 | entry["attempts"] = _failed(base, account, message, now) |
| 530 | return entry |
| 531 | except Exception as exc: # noqa: BLE001 -- written on the account below, never hidden |
| 532 | # Anything else (an answer imaplib could not parse, a bug here) is |
| 533 | # still this account's failure: recorded like the rest, and the |
| 534 | # other accounts still get their turn this pass. |
| 535 | message = _scrub(f"{type(exc).__name__}: {exc}", [password]) |
| 536 | entry["error"] = message |
| 537 | entry["attempts"] = _failed(base, account, message, now) |
| 538 | return entry |
| 539 | finally: |
| 540 | if conn is not None: |
| 541 | _logout(conn) |
| 542 | |
| 543 | |
| 544 | def EVENT(request): |
| 545 | base = _base_dir() |
| 546 | refusal = object_feature_flags.refusal( |
| 547 | env_name=object_mail.ENABLE_ENV, flag=object_mail.KILL_SWITCH_FLAG, base_dir=base) |
| 548 | if refusal: |
| 549 | return {"ok": True, "skipped": f"mail sync is {refusal}"} |
| 550 | if not _collection_exists(base, ACCOUNTS): |
| 551 | return {"ok": True, "skipped": "app-mail-sync is not installed (no mail_accounts)"} |
| 552 | missing = [name for name in (MESSAGES, THREADS, RECIPIENTS, STATE, SYNCED) |
| 553 | if not _collection_exists(base, name)] |
| 554 | if missing: |
| 555 | return {"ok": True, "skipped": f"missing collections {missing}: install app-messaging " |
| 556 | f"and app-mail-sync"} |
| 557 | |
| 558 | accounts = object_records.read_collection_records(ACCOUNTS, base_dir=base) |
| 559 | epoch = datetime(1970, 1, 1, tzinfo=timezone.utc) |
| 560 | accounts.sort(key=lambda row: object_mail.parse_timestamp(row.get("last_synced_at")) or epoch) |
| 561 | |
| 562 | report = {"ok": True, "accounts": 0, "imported": 0, "failed": 0, "declined": 0, |
| 563 | "waiting": 0, "results": []} |
| 564 | deadline = time.monotonic() + TICK_SECONDS |
| 565 | for account in accounts: |
| 566 | if not _text(account.get("id")): |
| 567 | continue |
| 568 | now = _now() |
| 569 | due, why = object_mail.account_due( |
| 570 | account, now, claim_ttl_seconds=CLAIM_TTL_SECONDS, |
| 571 | retry_base_seconds=RETRY_BASE_SECONDS, retry_max_seconds=RETRY_MAX_SECONDS) |
| 572 | if not due: |
| 573 | if why != "disabled": |
| 574 | report["waiting"] += 1 |
| 575 | report["results"].append({"account": account["id"], "waiting": why}) |
| 576 | continue |
| 577 | if time.monotonic() > deadline: |
| 578 | report["results"].append({"account": account["id"], |
| 579 | "waiting": "this pass ran out of time; the next one starts here"}) |
| 580 | report["waiting"] += 1 |
| 581 | continue |
| 582 | report["accounts"] += 1 |
| 583 | entry = _sync_one(base, account, now) |
| 584 | report["imported"] += entry.get("imported", 0) |
| 585 | if "error" in entry: |
| 586 | report["failed"] += 1 |
| 587 | if "declined" in entry: |
| 588 | report["declined"] += 1 |
| 589 | report["results"].append(entry) |
| 590 | return report |
| 591 | |
| 592 | |
| 593 | POST = EVENT |