Runs live, versioned Python application objects on one VM, with a suite of apps: projects, notes, tasks, files and more.

0 downloads

Download zip
Clone
Clone with git
git clone https://gitstable.com/git/askrobots/dbbasic-object-server.git
HTTPS
https://gitstable.com/git/askrobots/dbbasic-object-server.git
SSH
ssh://git@gitstable.com:2222/askrobots/dbbasic-object-server.git

Cloning needs no sign-in. Pushing over HTTPS asks for your username and an access token. SSH needs your key under SSH keys.

danq9 signup: the person chooses the password and name on the confirm page 83c7aa6 2 weeks, 5 days ago History
mail_sync.py · 25.6 KB · 593 lines
1"""system_mail_sync -- copy new mail from each person's IMAP mailboxes into
2their app-messaging inbox, without touching the mailbox.
3
4EVENT/POST: the scheduled pass declared in this package's `schedules`,
5every two minutes. (The home-page count of failing accounts is its own
6object, system_mail_sync_attention, as every attention provider is.)
7Everything that can be decided without a server -- parsing,
8dedupe, threading, where the cursor goes, what to download -- is
9object_mail's, which is pure and tested on its own. This object is the I/O:
10it opens the connection, asks, and writes rows.
11
12READ-ONLY TOWARDS THE MAILBOX. Folders are opened with EXAMINE (imaplib's
13``select(readonly=True)``) and messages fetched with BODY.PEEK[], so the
14sync can never set \\Seen, and ``is_read`` on each copied message is the
15server's own \\Seen flag at the moment of copying. The predecessor searched
16UNSEEN and fetched RFC822, which marked every synced message read in the
17person's own mail program and never copied anything they had read
18elsewhere first. Messages are chosen by UID, never by flag. The sync sends
19nothing, stores no link it does not open, and opens none.
20
21BOUNDED, because it runs inside the daemon's loop, between the other
22passes: at most BATCH_LIMIT messages and TICK_BYTE_BUDGET bytes per account
23per pass, socket timeouts on every connection, messages over
24MAX_MESSAGE_BYTES copied as headers only (the body then says so), a first
25sync that takes the newest FIRST_WINDOW messages of the last
26FIRST_WINDOW_DAYS days rather than the whole history, and no new account
27started after TICK_SECONDS. A backlog drains over several passes.
28
29AT LEAST ONCE, MADE SAFE. The folder cursor (mail_sync_state.last_uid) moves
30only after the rows for those messages are written. A pass that dies
31between the two repeats that work next time, and the repeat writes nothing
32new: every row id is object_mail.stable_id(owner, account, dedupe key), so
33the second create of the same message is refused as a duplicate.
34
35QUIET WHEN NOTHING HAPPENS. An account is checked read-only first (EXAMINE
36answers UIDNEXT); only when there is something to copy does the pass claim
37the account -- a compare-and-set on its row with expected_rev, so a manual
38run and the scheduled one cannot both write -- and only then are rows
39written. An idle account refreshes last_synced_at at most hourly. Without
40that, two account writes every two minutes would grow mail_accounts' change
41log by ~1,400 entries a day per account for nothing (AGENTS.md: a writer
42says how it is bounded).
43
44FAILURES ARE RECORDED, NEVER SWALLOWED. Two kinds of silence are told
45apart, the way system_tracking_poll does:
46
47 not set up the feature is off, the account has no password in
48 the vault, no owner, or an id that cannot name a
49 vault entry. Nothing is counted against the account;
50 the pass says why in its report.
51 this account failed a refused login, an unreachable or refused host, a
52 missing folder, a message that would not store.
53 sync_attempts goes up, sync_error says what happened,
54 and the account waits longer between tries
55 (object_mail.account_due) so a wrong password is not
56 tried every two minutes until the provider locks the
57 mailbox. At object_mail.ATTENTION_AFTER failures
58 in a row it is
59 counted on the home page.
60
61The password is read from the owner's identity vault under
62``imap-<account id>``, resolved by the ACCOUNT ROW's owner_id and never by
63anything a caller passed, used for the login, and never written, logged or
64returned; error text from the server is scrubbed of it before it is stored.
65"""
66
67import imaplib
68import os
69import socket
70import ssl
71import time
72from datetime import datetime, timedelta, timezone
73
74import object_feature_flags
75import object_mail
76import object_records
77import object_schemas
78import object_service_keys
79
80ACTOR = "system_mail_sync"
81
82ACCOUNTS = "mail_accounts"
83STATE = "mail_sync_state"
84SYNCED = "mail_synced_messages"
85MESSAGES = "messages"
86THREADS = "message_threads"
87RECIPIENTS = "message_recipients"
88
89BATCH_LIMIT = 50
90FIRST_WINDOW = 200
91FIRST_WINDOW_DAYS = 30
92MAX_MESSAGE_BYTES = 10 * 1024 * 1024
93TICK_BYTE_BUDGET = 25 * 1024 * 1024
94SOCKET_TIMEOUT_SECONDS = 30
95TICK_SECONDS = 60
96CLAIM_TTL_SECONDS = 15 * 60
97RETRY_BASE_SECONDS = 120
98RETRY_MAX_SECONDS = 30 * 60
99HEARTBEAT_SECONDS = 60 * 60
100MAX_ERROR = 500
101
102_DEFAULT_PORTS = {"ssl": 993, "starttls": 143, "plain": 143}
103
104
105class SyncError(Exception):
106 """A failure this account's owner needs to read; the text is stored."""
107
108
109def _base_dir():
110 return os.environ.get("DBBASIC_DATA_DIR", "data")
111
112
113def _text(value):
114 return str(value if value is not None else "").strip()
115
116
117def _int(value, default=0):
118 try:
119 return int(_text(value) or default)
120 except (TypeError, ValueError):
121 return default
122
123
124def _now():
125 return datetime.now(timezone.utc)
126
127
128def _stamp(moment):
129 return object_mail.format_timestamp(moment)
130
131
132def _collection_exists(base, name):
133 if object_records.collection_has_records(name, base_dir=base):
134 return True
135 try:
136 object_schemas.get_schema(name, base_dir=base)
137 return True
138 except (object_schemas.SchemaNotFoundError, OSError, ValueError):
139 return False
140
141
142def _get(base, collection, record_id):
143 try:
144 return object_records.get_collection_record(collection, record_id, base_dir=base)
145 except object_records.RecordNotFoundError:
146 return None
147
148
149def _create(base, collection, row):
150 """True when written, False when a row with this id already exists --
151 the replay of a batch a previous pass did not finish."""
152 try:
153 object_records.create_collection_record(collection, row, base_dir=base, actor=ACTOR)
154 return True
155 except object_records.DuplicateRecordIdError:
156 return False
157
158
159def _scrub(text, secrets):
160 text = _text(text)
161 for secret in secrets:
162 if secret and len(secret) >= 3:
163 text = text.replace(secret, "***")
164 return text[:MAX_ERROR]
165
166
167# --- the connection ------------------------------------------------------------
168
169def _connect(account, password):
170 host = _text(account.get("host"))
171 security = _text(account.get("security")) or "ssl"
172 if security not in _DEFAULT_PORTS:
173 raise SyncError(f"security must be ssl, starttls or plain, not {security!r}")
174 port = _int(account.get("port")) or _DEFAULT_PORTS[security]
175 username = _text(account.get("username")) or _text(account.get("email_address"))
176 if not host:
177 raise SyncError("the account has no IMAP server")
178 if any(char in username for char in "\r\n\x00"):
179 # imaplib quotes a login but does not refuse a line break, which
180 # would end the command and start another on the mailbox.
181 raise SyncError("the user name cannot contain a line break")
182
183 try:
184 infos = socket.getaddrinfo(host, port, type=socket.SOCK_STREAM)
185 except (socket.gaierror, UnicodeError, OSError) as exc:
186 raise SyncError(f"could not find the server {host!r}: {exc}") from exc
187 refusal = object_mail.host_refusal(
188 host, sorted({info[4][0] for info in infos}), security=security,
189 allow_private=object_feature_flags.env_opt_in(object_mail.ALLOW_PRIVATE_HOSTS_ENV))
190 if refusal:
191 raise SyncError(refusal)
192
193 conn = None
194 try:
195 if security == "ssl":
196 conn = imaplib.IMAP4_SSL(host, port, ssl_context=ssl.create_default_context(),
197 timeout=SOCKET_TIMEOUT_SECONDS)
198 else:
199 conn = imaplib.IMAP4(host, port, timeout=SOCKET_TIMEOUT_SECONDS)
200 if security == "starttls":
201 conn.starttls(ssl_context=ssl.create_default_context())
202 except (OSError, imaplib.IMAP4.error, ssl.SSLError) as exc:
203 if conn is not None:
204 _logout(conn)
205 raise SyncError(f"could not connect to {host}:{port} ({security}): {exc}") from exc
206
207 try:
208 conn.login(username, password)
209 except UnicodeError as exc:
210 _logout(conn)
211 raise SyncError("the user name or password has characters an IMAP login "
212 "cannot send (only plain ASCII is allowed)") from exc
213 except imaplib.IMAP4.error as exc:
214 _logout(conn)
215 raise SyncError(f"the server refused the login: {_scrub(exc, [password])}") from exc
216 return conn
217
218
219def _logout(conn):
220 try:
221 conn.logout()
222 except Exception: # noqa: BLE001 -- the work is done or already failed; closing is best effort
223 return
224
225
226def _ok(typ, data, what):
227 if typ != "OK":
228 detail = b" ".join(d for d in (data or []) if isinstance(d, bytes)).decode("utf-8", "replace")
229 raise SyncError(f"{what} failed: {detail or typ}")
230 return data
231
232
233def _examine(conn, folder):
234 """Open a folder read-only; (uidvalidity, uidnext or 0)."""
235 try:
236 quoted = object_mail.quote_mailbox(folder)
237 except ValueError as exc:
238 raise SyncError(f"the folder name {folder!r} cannot be used: {exc}") from exc
239 typ, data = conn.select(quoted, readonly=True)
240 _ok(typ, data, f"opening the folder {folder!r}")
241 uidvalidity = object_mail.parse_search(conn.response("UIDVALIDITY")[1])
242 uidnext = object_mail.parse_search(conn.response("UIDNEXT")[1])
243 if not uidvalidity:
244 raise SyncError(f"the server gave no UIDVALIDITY for {folder!r}")
245 return str(uidvalidity[0]), (uidnext[0] if uidnext else 0)
246
247
248# --- one account -----------------------------------------------------------------
249
250class _Pass:
251 """One account's sync: what to check, then (only if needed) the writes."""
252
253 def __init__(self, base, account, password, now):
254 self.base = base
255 self.account = account
256 self.password = password
257 self.now = now
258 self.account_id = account["id"]
259 self.owner_id = _text(account.get("owner_id"))
260 self.address = _text(account.get("email_address"))
261 self.imported = 0
262 self.duplicates = 0
263 self.errors = []
264 self.discovered = None
265 self._seen = None
266 self._index = None
267
268 # -- read-only check -------------------------------------------------------
269
270 def check(self, conn):
271 """Folders with something to do, found without writing anything."""
272 names = object_mail.split_folders(self.account.get("folders"))
273 entries = None
274 if not names:
275 entries = object_mail.parse_list(_ok(*conn.list(), "LIST"))
276 chosen = object_mail.choose_folders(entries)
277 self.discovered = ", ".join(name for name, _ in chosen)
278 names = [name for name, _ in chosen]
279
280 work = []
281 for name in names:
282 state_id = object_mail.stable_id("folder", self.account_id, name)
283 state = _get(self.base, STATE, state_id)
284 try:
285 uidvalidity, uidnext = _examine(conn, name)
286 except SyncError as exc:
287 self.errors.append(str(exc))
288 continue
289 decision = object_mail.plan_folder(state, uidvalidity=uidvalidity)
290 last_uid = _int((state or {}).get("last_uid"))
291 if decision == "continue" and uidnext and uidnext - 1 <= last_uid:
292 continue
293 role = (state or {}).get("role")
294 if not role:
295 if entries is None:
296 entries = object_mail.parse_list(_ok(*conn.list(), "LIST"))
297 role = object_mail.folder_role(name, entries)
298 work.append({"folder": name, "state_id": state_id, "state": state,
299 "decision": decision, "uidvalidity": uidvalidity,
300 "uidnext": uidnext, "role": role})
301 return work
302
303 # -- the writes --------------------------------------------------------------
304
305 def run(self, conn, work):
306 budget = BATCH_LIMIT
307 bytes_left = TICK_BYTE_BUDGET
308 for item in work:
309 if budget <= 0 or bytes_left <= 0:
310 break
311 try:
312 done, spent = self._folder(conn, item, budget, bytes_left)
313 except SyncError as exc:
314 self.errors.append(str(exc))
315 continue
316 except (imaplib.IMAP4.error, OSError) as exc:
317 # The connection itself failed; later folders cannot work.
318 self.errors.append(f"{item['folder']}: {_scrub(exc, [self.password])}")
319 break
320 budget -= done
321 bytes_left -= spent
322
323 def _folder(self, conn, item, budget, bytes_left):
324 folder = item["folder"]
325 uidvalidity, uidnext = _examine(conn, folder)
326 if uidvalidity != item["uidvalidity"]:
327 raise SyncError(f"{folder!r} changed UIDVALIDITY while it was being read; "
328 f"the next pass starts it again")
329 state = item["state"]
330 last_uid = _int((state or {}).get("last_uid"))
331
332 if item["decision"] in ("fresh", "reset"):
333 since = object_mail.imap_date(self.now - timedelta(days=FIRST_WINDOW_DAYS))
334 window = object_mail.parse_search(_ok(*conn.uid("SEARCH", "SINCE", since), "SEARCH"))
335 highest = uidnext - 1 if uidnext else max(
336 object_mail.parse_search(_ok(*conn.uid("SEARCH", "ALL"), "SEARCH")) or [0])
337 last_uid = object_mail.window_baseline(window, first_window=FIRST_WINDOW,
338 highest_uid=highest)
339 row = {"account_id": self.account_id, "folder": folder, "role": item["role"],
340 "uidvalidity": uidvalidity, "last_uid": str(last_uid),
341 "owner_id": self.owner_id}
342 if state is None:
343 _create(self.base, STATE, {"id": item["state_id"], **row})
344 else:
345 object_records.update_collection_record(
346 STATE, item["state_id"], row, base_dir=self.base, actor=ACTOR)
347
348 found = object_mail.parse_search(
349 _ok(*conn.uid("SEARCH", "UID", f"{last_uid + 1}:*"), "SEARCH"))
350 uids = object_mail.next_uids(found, last_uid=last_uid, limit=budget)
351 if not uids:
352 return 0, 0
353 metas = object_mail.parse_fetch_meta(_ok(*conn.uid(
354 "FETCH", ",".join(str(uid) for uid in uids),
355 "(UID FLAGS RFC822.SIZE INTERNALDATE)"), "FETCH"))
356 by_uid = {meta["uid"]: meta for meta in metas}
357 plan = object_mail.plan_downloads(metas, max_message_bytes=MAX_MESSAGE_BYTES,
358 byte_budget=bytes_left)
359
360 advanced = last_uid
361 done = spent = 0
362 try:
363 for uid, mode in plan:
364 meta = by_uid[uid]
365 part = "BODY.PEEK[]" if mode == "full" else "BODY.PEEK[HEADER]"
366 raw = object_mail.fetch_literal(_ok(*conn.uid("FETCH", str(uid), f"(UID {part})"),
367 "FETCH"))
368 if raw is None:
369 raise SyncError(f"{folder!r}: the server sent no message for UID {uid}")
370 parsed = object_mail.parse_message(
371 raw, flags=meta["flags"], internaldate=meta["internaldate"],
372 header_only_size=meta["size"] if mode == "header" else None)
373 try:
374 self._store(parsed, folder=folder, uid=uid, uidvalidity=uidvalidity,
375 role=item["role"])
376 except (object_records.InvalidRecordPayloadError, ValueError, LookupError) as exc:
377 raise SyncError(f"{folder!r}: UID {uid} could not be stored: "
378 f"{str(exc)[:200]}") from exc
379 advanced = uid
380 done += 1
381 spent += meta["size"] if mode == "full" else 0
382 finally:
383 # The cursor moves to the last message whose rows are written,
384 # and no further, whatever stopped the loop.
385 if advanced > last_uid:
386 object_records.update_collection_record(
387 STATE, item["state_id"], {"last_uid": str(advanced)},
388 base_dir=self.base, actor=ACTOR)
389 return done, spent
390
391 def _prepare(self):
392 """The dedupe keys and thread index, built once per pass and only
393 when there is something to import -- they are the expensive reads."""
394 if self._seen is not None:
395 return
396 synced = object_records.read_collection_records(SYNCED, base_dir=self.base)
397 own = [row for row in synced if row.get("owner_id") == self.owner_id]
398 self._seen = {row.get("dedupe_key") for row in own
399 if row.get("account_id") == self.account_id}
400 messages = object_records.read_collection_records(MESSAGES, base_dir=self.base)
401 self._index = object_mail.ThreadIndex.from_rows(self.owner_id, messages, own)
402
403 def _store(self, parsed, *, folder, uid, uidvalidity, role):
404 self._prepare()
405 key = object_mail.dedupe_key(parsed)
406 if key in self._seen:
407 self.duplicates += 1
408 return
409 owner, account_id = self.owner_id, self.account_id
410 direction = object_mail.direction_for(parsed, account_address=self.address,
411 folder_role=role)
412 message_id = object_mail.stable_id("message", owner, account_id, key)
413 thread_id = object_mail.match_thread(parsed, self._index)
414 joined = thread_id is not None
415 if not joined:
416 thread_id = object_mail.stable_id("thread", owner, account_id, key)
417 _create(self.base, THREADS, object_mail.new_thread_row(
418 parsed, owner_id=owner, thread_id=thread_id, direction=direction,
419 account_address=self.address))
420
421 row, recipients = object_mail.message_rows(
422 parsed, owner_id=owner, message_id=message_id, thread_id=thread_id,
423 direction=direction)
424 created = _create(self.base, MESSAGES, row)
425 for recipient in recipients:
426 _create(self.base, RECIPIENTS, recipient)
427 _create(self.base, SYNCED, object_mail.synced_row(
428 parsed, row_id=object_mail.stable_id("synced", owner, account_id, key),
429 owner_id=owner, account_id=account_id, folder=folder, uid=uid,
430 uidvalidity=uidvalidity, message_id=message_id, key=key))
431 if created and joined:
432 thread = _get(self.base, THREADS, thread_id)
433 if thread is not None:
434 changes = object_mail.thread_changes(thread, parsed, direction=direction,
435 account_address=self.address)
436 object_records.update_collection_record(
437 THREADS, thread_id, changes, base_dir=self.base, actor=ACTOR)
438 self._index.add(parsed, thread_id)
439 self._seen.add(key)
440 if created:
441 self.imported += 1
442 else:
443 self.duplicates += 1
444
445
446# --- the pass ------------------------------------------------------------------------
447
448def _finish(base, account, changes, *, expected_rev=None):
449 try:
450 object_records.update_collection_record(
451 ACCOUNTS, account["id"], changes, base_dir=base, actor=ACTOR,
452 preserve_read_only=True, expected_rev=expected_rev)
453 return True
454 except object_records.VersionConflictError:
455 return False
456
457
458def _failed(base, account, message, now):
459 attempts = _int(account.get("sync_attempts")) + 1
460 _finish(base, account, {"sync_attempts": str(attempts), "sync_error": message[:MAX_ERROR],
461 "sync_finished_at": _stamp(now)})
462 return attempts
463
464
465def _sync_one(base, account, now):
466 """One account's outcome, as a report entry."""
467 entry = {"account": account["id"]}
468 owner_id = _text(account.get("owner_id"))
469 if not owner_id:
470 entry["declined"] = "the account has no owner, so there is no vault to read a password from"
471 return entry
472 service = object_mail.password_service(account["id"])
473 if not object_service_keys.validate_service_name(service):
474 entry["declined"] = (f"the account id cannot name a vault entry ({service!r} is not a "
475 f"valid service name); recreate the account from /mail/accounts")
476 return entry
477 password = object_service_keys.get_service_key(owner_id, service, base_dir=base)
478 if not password:
479 entry["declined"] = f"no password set (the owner's vault has no {service!r})"
480 return entry
481
482 rev = object_records.compute_record_rev(account)
483 conn = None
484 try:
485 conn = _connect(account, password)
486 sync = _Pass(base, account, password, now)
487 work = sync.check(conn)
488 if not work and sync.discovered is None:
489 # Nothing to copy. Write only what changed: a recovery clears the
490 # failure count; otherwise last_synced_at is refreshed hourly.
491 if sync.errors:
492 raise SyncError("; ".join(sync.errors))
493 last = object_mail.parse_timestamp(account.get("last_synced_at"))
494 recovering = _int(account.get("sync_attempts")) or _text(account.get("sync_error"))
495 if recovering or last is None or (now - last).total_seconds() >= HEARTBEAT_SECONDS:
496 _finish(base, account, {"last_synced_at": _stamp(now), "sync_finished_at": _stamp(now),
497 "sync_attempts": "0", "sync_error": ""}, expected_rev=rev)
498 entry["up_to_date"] = True
499 return entry
500
501 # Something to write: claim the account first. expected_rev is the
502 # row as this pass read it, so a second pass (or an edit) that got
503 # there first makes this one stand down instead of writing beside it.
504 if not _finish(base, account, {"sync_started_at": _stamp(now)}, expected_rev=rev):
505 entry["declined"] = "the account changed while this pass read it; the next pass will try"
506 return entry
507 try:
508 sync.run(conn, work)
509 except Exception as exc: # noqa: BLE001 -- recorded on the account below, not hidden
510 sync.errors.append(_scrub(f"{type(exc).__name__}: {exc}", [password]))
511 entry["imported"] = sync.imported
512 if sync.duplicates:
513 entry["already_copied"] = sync.duplicates
514 changes = {"sync_finished_at": _stamp(_now())}
515 if sync.discovered is not None:
516 changes["folders"] = sync.discovered
517 if sync.errors:
518 message = _scrub("; ".join(sync.errors), [password])
519 changes.update({"sync_attempts": str(_int(account.get("sync_attempts")) + 1),
520 "sync_error": message})
521 entry["error"] = message
522 else:
523 changes.update({"last_synced_at": _stamp(now), "sync_attempts": "0", "sync_error": ""})
524 _finish(base, account, changes)
525 return entry
526 except SyncError as exc:
527 message = _scrub(exc, [password])
528 entry["error"] = message
529 entry["attempts"] = _failed(base, account, message, now)
530 return entry
531 except Exception as exc: # noqa: BLE001 -- written on the account below, never hidden
532 # Anything else (an answer imaplib could not parse, a bug here) is
533 # still this account's failure: recorded like the rest, and the
534 # other accounts still get their turn this pass.
535 message = _scrub(f"{type(exc).__name__}: {exc}", [password])
536 entry["error"] = message
537 entry["attempts"] = _failed(base, account, message, now)
538 return entry
539 finally:
540 if conn is not None:
541 _logout(conn)
542
543
544def EVENT(request):
545 base = _base_dir()
546 refusal = object_feature_flags.refusal(
547 env_name=object_mail.ENABLE_ENV, flag=object_mail.KILL_SWITCH_FLAG, base_dir=base)
548 if refusal:
549 return {"ok": True, "skipped": f"mail sync is {refusal}"}
550 if not _collection_exists(base, ACCOUNTS):
551 return {"ok": True, "skipped": "app-mail-sync is not installed (no mail_accounts)"}
552 missing = [name for name in (MESSAGES, THREADS, RECIPIENTS, STATE, SYNCED)
553 if not _collection_exists(base, name)]
554 if missing:
555 return {"ok": True, "skipped": f"missing collections {missing}: install app-messaging "
556 f"and app-mail-sync"}
557
558 accounts = object_records.read_collection_records(ACCOUNTS, base_dir=base)
559 epoch = datetime(1970, 1, 1, tzinfo=timezone.utc)
560 accounts.sort(key=lambda row: object_mail.parse_timestamp(row.get("last_synced_at")) or epoch)
561
562 report = {"ok": True, "accounts": 0, "imported": 0, "failed": 0, "declined": 0,
563 "waiting": 0, "results": []}
564 deadline = time.monotonic() + TICK_SECONDS
565 for account in accounts:
566 if not _text(account.get("id")):
567 continue
568 now = _now()
569 due, why = object_mail.account_due(
570 account, now, claim_ttl_seconds=CLAIM_TTL_SECONDS,
571 retry_base_seconds=RETRY_BASE_SECONDS, retry_max_seconds=RETRY_MAX_SECONDS)
572 if not due:
573 if why != "disabled":
574 report["waiting"] += 1
575 report["results"].append({"account": account["id"], "waiting": why})
576 continue
577 if time.monotonic() > deadline:
578 report["results"].append({"account": account["id"],
579 "waiting": "this pass ran out of time; the next one starts here"})
580 report["waiting"] += 1
581 continue
582 report["accounts"] += 1
583 entry = _sync_one(base, account, now)
584 report["imported"] += entry.get("imported", 0)
585 if "error" in entry:
586 report["failed"] += 1
587 if "declined" in entry:
588 report["declined"] += 1
589 report["results"].append(entry)
590 return report
591
592
593POST = EVENT