docs: update Design Principles section with enhanced lock strategy and event handling guidelines
This commit is contained in:
@@ -69,35 +69,55 @@ Webhook (HTTP) → Queue Event → Event Handler → External APIs
|
|||||||
1. **Fire events liberally** - 10 redundant events are cheaper than complex coordination
|
1. **Fire events liberally** - 10 redundant events are cheaper than complex coordination
|
||||||
2. **Make handlers idempotent** - Early returns when nothing to do
|
2. **Make handlers idempotent** - Early returns when nothing to do
|
||||||
3. **Sequential per entity, parallel across entities** - Lock prevents collisions, not updates
|
3. **Sequential per entity, parallel across entities** - Lock prevents collisions, not updates
|
||||||
4. **Accept event storms** - Handlers queue up, process one by one
|
4. **Accept event storms** - Handlers queue up, Motia retries automatically
|
||||||
|
|
||||||
**Lock Strategy: Sequential Processing per Entity**
|
**Lock Strategy: Step + Input Values**
|
||||||
|
|
||||||
```
|
```
|
||||||
Event Storm für Document A:
|
Lock Key Format: {step_name}:{entity_type}:{entity_id}
|
||||||
├─ Event 1: update → Handler starts → Lock acquired
|
|
||||||
├─ Event 2: update → Queued (waits for lock)
|
|
||||||
├─ Event 3: update → Queued (after Event 2)
|
|
||||||
└─ Event 4: update → Queued (after Event 3)
|
|
||||||
|
|
||||||
Document B (parallel):
|
Examples:
|
||||||
└─ Event 1: update → Own lock, processes in parallel!
|
- document_sync:CDokumente:doc123
|
||||||
|
- beteiligte_sync:CBeteiligte:bet456
|
||||||
|
- collection_create:Account:acc789
|
||||||
|
|
||||||
Result:
|
Rules:
|
||||||
- Same entity: Sequential (prevents file upload collisions)
|
✅ Parallel: sync document:A + sync document:B (different IDs)
|
||||||
- Different entities: Parallel (independent locks)
|
❌ Blocked: sync document:A + sync document:A (same ID)
|
||||||
- Lost events: Zero (all queued and processed)
|
|
||||||
- Duplicate work: Prevented by idempotency checks
|
Lock Acquisition:
|
||||||
|
├─ Try acquire lock
|
||||||
|
├─ Success → Process → Release in finally block
|
||||||
|
└─ Fail (already locked) → Return early → Motia retries later
|
||||||
```
|
```
|
||||||
|
|
||||||
**Example Flow:**
|
**Handler Pattern:**
|
||||||
|
```python
|
||||||
|
async def handler(event_data, ctx):
|
||||||
|
entity_id = event_data['entity_id']
|
||||||
|
|
||||||
|
# 1. Try acquire lock
|
||||||
|
lock_key = f"document_sync:{entity_type}:{entity_id}"
|
||||||
|
lock_acquired = await acquire_lock(lock_key)
|
||||||
|
|
||||||
|
if not lock_acquired:
|
||||||
|
ctx.logger.info(f"⏸️ Lock busy, skipping (Motia will retry)")
|
||||||
|
return # Early return - no error, just skip
|
||||||
|
|
||||||
|
try:
|
||||||
|
# 2. Do work (only if lock acquired)
|
||||||
|
await sync_document(entity_id)
|
||||||
|
|
||||||
|
finally:
|
||||||
|
# 3. ALWAYS release lock (robust cleanup)
|
||||||
|
await release_lock(lock_key)
|
||||||
```
|
```
|
||||||
t=0: User ändert Document A (fileStatus → "changed")
|
|
||||||
t=1: Event 1 fired → Lock acquired → Sync starts
|
**Retry Behavior:**
|
||||||
t=2: User ändert Document A again (fileStatus → "changed")
|
- Lock fail → Handler returns without error
|
||||||
t=3: Event 2 fired → Queued (lock busy)
|
- Motia Queue sees: No error, no completion marker
|
||||||
t=4: Event 1 completes → fileStatus="unchanged", xaiSyncStatus="clean"
|
- Motia automatically retries after configured delay
|
||||||
t=5: Event 2 starts → Lock acquired
|
- Eventually lock available → Handler processes
|
||||||
Check: fileStatus="unchanged", xaiSyncStatus="clean"
|
Check: fileStatus="unchanged", xaiSyncStatus="clean"
|
||||||
→ Early return (nothing to do) ✅
|
→ Early return (nothing to do) ✅
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user