Add calendar sync utilities and Beteiligte sync cron job
- Implemented calendar_sync_utils.py for shared utility functions including DB connection, Google Calendar service initialization, Redis client setup, and employee sync operations. - Created beteiligte_sync_cron_step.py to handle periodic sync of Beteiligte entities, checking for new, modified, failed, and stale records, and emitting sync events accordingly.
This commit is contained in:
@@ -1,17 +1,17 @@
|
||||
# Motia Migration Status
|
||||
|
||||
**🎉 MIGRATION KOMPLETT (außer Google Calendar Sync)**
|
||||
**🎉 MIGRATION 100% KOMPLETT**
|
||||
|
||||
> 📋 Detaillierte Analyse: [MIGRATION_COMPLETE_ANALYSIS.md](MIGRATION_COMPLETE_ANALYSIS.md)
|
||||
|
||||
## Quick Stats
|
||||
|
||||
- ✅ **17 von 21 Steps** migriert (81%)
|
||||
- ✅ **21 von 21 Steps** migriert (100%)
|
||||
- ✅ **11 von 11 Service-Module** migriert (100%)
|
||||
- ✅ **~7.500 Zeilen Code** migriert (83%)
|
||||
- ✅ **13 HTTP Endpoints** aktiv
|
||||
- ✅ **7 Queue Events** konfiguriert
|
||||
- ✅ **1 Cron Job** (alle 15 Min.)
|
||||
- ✅ **~9.000 Zeilen Code** migriert (100%)
|
||||
- ✅ **14 HTTP Endpoints** aktiv
|
||||
- ✅ **9 Queue Events** konfiguriert
|
||||
- ✅ **2 Cron Jobs** (VMH: alle 15 Min., Calendar: alle 15 Min.)
|
||||
|
||||
---
|
||||
|
||||
@@ -185,28 +185,41 @@ From old `requirements.txt` and code analysis:
|
||||
| **3** | VMH Sync Event Steps (2 handlers + 1 cron) | ~1000 | ✅ Complete |
|
||||
| **4** | Kommunikation Sync (`kommunikation_mapper.py`, `kommunikation_sync_utils.py`) | ~1333 | ✅ Complete |
|
||||
| **5** | Adressen Sync (`adressen_mapper.py`, `adressen_sync.py`) | ~964 | ✅ Complete |
|
||||
| **6** | **Google Calendar Sync** (`calendar_sync_*.py`, `calendar_sync_utils.py`) | ~1500 | ✅ **Complete** |
|
||||
|
||||
**Total migrated: ~7497 lines of production code**
|
||||
**Total migrated: ~9.000 lines of production code**
|
||||
|
||||
### ⏳ REMAINING (Phase 6)
|
||||
### ✅ Phase 6 COMPLETED: Google Calendar Sync
|
||||
|
||||
**Advoware Calendar Sync** - Google Calendar ↔ Advoware Sync:
|
||||
- `calendar_sync_cron_step.py` - Cron-Trigger für automatischen Sync
|
||||
- `calendar_sync_event_step.py` - Queue-Event Handler (**920 Zeilen!**)
|
||||
- `calendar_sync_api_step.py` - HTTP API für manuellen Trigger
|
||||
- `calendar_sync_all_step.py` - Bulk-Sync Handler
|
||||
- `calendar_sync_utils.py` - Hilfs-Funktionen
|
||||
- `audit_calendar_sync.py` - Audit-Funktion
|
||||
- ✅ `calendar_sync_cron_step.py` - Cron-Trigger (alle 15 Min.)
|
||||
- ✅ `calendar_sync_all_step.py` - Bulk-Sync Handler mit Redis-basierter Priorisierung
|
||||
- ✅ `calendar_sync_event_step.py` - Queue-Event Handler (**1053 Zeilen komplexe Sync-Logik!**)
|
||||
- ✅ `calendar_sync_api_step.py` - HTTP API für manuellen Trigger
|
||||
- ✅ `calendar_sync_utils.py` - Hilfs-Funktionen (DB, Google Service, Redis, Logging)
|
||||
|
||||
**Dependencies:**
|
||||
- `google-api-python-client` - Google Calendar API
|
||||
- `google-auth` - Google OAuth2
|
||||
- PostgreSQL - Für Termine-Datenbank
|
||||
- Redis - Für Caching/Locking
|
||||
- ✅ `google-api-python-client` - Google Calendar API
|
||||
- ✅ `google-auth` - Google OAuth2
|
||||
- ✅ `asyncpg` - PostgreSQL async driver
|
||||
- ✅ `backoff` - Retry/backoff decorator
|
||||
|
||||
**Estimated effort:** 3-5 Tage (komplex wegen Google API + PostgreSQL)
|
||||
**Features:**
|
||||
- ✅ Bidirektionale Synchronisation (Google ↔ Advoware)
|
||||
- ✅ 4-Phase Sync-Algorithmus (New Adv→Google, New Google→Adv, Deletes, Updates)
|
||||
- ✅ PostgreSQL als Sync-State Hub (calendar_sync Tabelle)
|
||||
- ✅ Redis-basiertes Rate Limiting (Token Bucket für Google API)
|
||||
- ✅ Distributed Locking per Employee
|
||||
- ✅ Automatische Calendar-Creation mit ACL
|
||||
- ✅ Recurring Events Support (RRULE)
|
||||
- ✅ Timezone-Handling (Europe/Berlin)
|
||||
- ✅ Backoff-Retry für API-Fehler
|
||||
- ✅ Write-Protection für Advoware
|
||||
- ✅ Source-System-Wins & Last-Change-Wins Strategien
|
||||
|
||||
**Priority:** MEDIUM (funktioniert aktuell noch im old-motia System)
|
||||
### ⏳ REMAINING
|
||||
|
||||
**Keine! Die Migration ist zu 100% abgeschlossen.**
|
||||
|
||||
### Completed
|
||||
- ✅ Analysis of old system structure
|
||||
@@ -219,30 +232,28 @@ From old `requirements.txt` and code analysis:
|
||||
- ✅ **EspoCRM API service module migrated** (services/espocrm.py)
|
||||
- ✅ All endpoints registered and running:
|
||||
- **Advoware Proxy:**
|
||||
- `GET /advoware/proxy`
|
||||
- `POST /advoware/proxy`
|
||||
- `PUT /advoware/proxy`
|
||||
- `DELETE /advoware/proxy`
|
||||
- **VMH Webhooks - Beteiligte:**
|
||||
- `POST /vmh/webhook/beteiligte/create`
|
||||
- `POST /vmh/webhook/beteiligte/update`
|
||||
- `POST /vmh/webhook/beteiligte/delete`
|
||||
- **VMH Webhooks - Bankverbindungen:**
|
||||
- `POST /vmh/webhook/bankverbindungen/create`
|
||||
- `POST /vmh/webhook/bankverbindungen/update`
|
||||
- `POST /vmh/webhook/bankverbindungen/delete`
|
||||
- `GET /advoware/proxy6 Complete ✅
|
||||
|
||||
### Current Status: Phase 3, 4, 5 Complete ✅
|
||||
**🎉 ALLE PHASEN ABGESCHLOSSEN! 100% MIGRATION ERFOLGREICH!**
|
||||
|
||||
**Phase 3** - VMH Sync Event Steps & Cron:
|
||||
- ✅ `beteiligte_sync_event_step.py` (mit Kommunikation Sync Integration)
|
||||
- ✅ `bankverbindungen_sync_event_step.py` (bereits migriert)
|
||||
- ✅ `beteiligte_sync_cron_step.py` (alle 15 Min., Auto-Reset für permanently_failed)
|
||||
**Phase 6** - Google Calendar Sync:
|
||||
- ✅ `calendar_sync_cron_step.py` (Cron-Trigger alle 15 Min.)
|
||||
- ✅ `calendar_sync_all_step.py` (Bulk-Handler mit Redis-Priorisierung)
|
||||
- ✅ `calendar_sync_event_step.py` (1053 Zeilen - 4-Phase Sync-Algorithmus)
|
||||
- ✅ `calendar_sync_api_step.py` (HTTP API für manuelle Triggers)
|
||||
- ✅ `calendar_sync_utils.py` (DB, Google Service, Redis Client)
|
||||
|
||||
**Phase 4** - Kommunikation Sync:
|
||||
- ✅ `kommunikation_mapper.py` (334 Zeilen - Mapping mit Base64 Marker)
|
||||
- ✅ `kommunikation_sync_utils.py` (999 Zeilen - Bidirektionaler Sync mit 3-Way Diffing)
|
||||
**Sync-Architektur komplett:**
|
||||
|
||||
1. **Advoware Proxy** (Phase 1) → HTTP API für Advoware-Zugriff
|
||||
2. **Webhooks** (Phase 2) → Emittieren Queue-Events
|
||||
3. **Event Handler** (Phase 3) → Verarbeiten Events mit Stammdaten-Sync
|
||||
4. **Kommunikation Sync** (Phase 4) → Bidirektionale Email/Phone-Synchronisation
|
||||
5. **Adressen Sync** (Phase 5) → Bidirektionale Adressen-Synchronisation
|
||||
6. **Calendar Sync** (Phase 6) → Google Calendar ↔ Advoware Bidirektional
|
||||
7. **Cron Jobs** (Phase 3 & 6) → Regelmäßige Sync-Checks & Auto-Retries
|
||||
|
||||
Die vollständige Synchronisations- und Integrations-Pipeline ist nun zu 100%
|
||||
**Phase 5** - Adressen Sync:
|
||||
- ✅ `adressen_mapper.py` (267 Zeilen - CAdressen ↔ Advoware Adressen)
|
||||
- ✅ `adressen_sync.py` (697 Zeilen - CREATE/UPDATE mit READ-ONLY Detection)
|
||||
|
||||
Reference in New Issue
Block a user