Commit Graph

73 Commits

Author SHA1 Message Date
8550107b89 feat: Enhance Advoware API integration with backward compatibility for data payloads and improve logging for sync events 2026-02-07 15:44:56 +00:00
b5abe6cf00 Implement EspoCRM-based sync strategy for Beteiligte entities
- Add SYNC_STRATEGY_ESPOCRM_BASED.md detailing the sync flows and status management.
- Create utilities for sync operations in services/beteiligte_sync_utils.py, including locking, timestamp comparison, conflict resolution, and notification handling.
- Implement entity mapping between EspoCRM and Advoware in services/espocrm_mapper.py.
- Develop a cron job for periodic sync checks in steps/vmh/beteiligte_sync_cron_step.py, emitting events for entities needing synchronization.
2026-02-07 15:21:16 +00:00
e6ab22d5f4 feat: Add EspoCRM and Advoware integration for Beteiligte comparison
- Implemented `compare_beteiligte.py` script for comparing Beteiligte structures between EspoCRM and Advoware.
- Created `beteiligte_comparison_result.json` to store comparison results.
- Developed `EspoCRMAPI` service for handling API interactions with EspoCRM.
- Added comprehensive documentation for the EspoCRM API service.
- Included error handling and logging for API operations.
- Enhanced entity management with CRUD operations and search capabilities.
2026-02-07 14:42:58 +00:00
36552903e7 cleanup 2026-02-07 09:23:49 +00:00
root
96eabe3db6 Refactor calendar sync to prioritize oldest synced employees with human-readable timestamps 2025-10-26 08:58:48 +00:00
root
b18e770f12 refactor: extract common functions to utils
- Add get_redis_client() to calendar_sync_utils.py
- Add get_advoware_employees() to calendar_sync_utils.py
- Add set_employee_lock() and clear_employee_lock() to calendar_sync_utils.py
- Update all step files to use shared utility functions
- Remove duplicate code across calendar_sync_*.py files
2025-10-25 09:21:45 +00:00
root
e4bf21e676 Refactor: Extract common utilities to shared module
- Created calendar_sync_utils.py with shared functions:
  - log_operation: Centralized logging with context support
  - connect_db: Database connection wrapper
  - get_google_service: Google Calendar API initialization
- Updated imports in calendar_sync_event_step.py and audit_calendar_sync.py
- Removed duplicated function definitions
- Maintained function logic without changes
- Improved code maintainability and reduced duplications
2025-10-25 08:54:54 +00:00
root
c5600b42ec feat: Enhance audit_calendar_sync tool with comprehensive management features
- Add detailed documentation in README.md for all 10 audit commands
- Fix Advoware API parameter from 'frNr' to 'frnr' for proper filtering
- Fix subject field mapping from 'betreff' to 'text' in API responses
- Add verify-sync command for bidirectional sync verification
- Add query-frnr and query-event commands for individual record lookup
- Add management commands: find-duplicates, delete-duplicates, find-orphaned, cleanup-orphaned
- Improve error handling and output formatting
- Remove temporary test script
2025-10-25 08:18:48 +00:00
root
c5ddd02307 Add query and verification features for individual appointments by frNr or Google Event ID 2025-10-25 07:53:43 +00:00
root
1539c26be6 Expand audit_calendar_sync.py to comprehensive management tool with pagination, duplicate detection/deletion, and orphaned calendar cleanup 2025-10-25 07:48:35 +00:00
root
62f57bb035 Fix ensure_google_calendar to fetch all calendars with pagination to avoid missing calendars on later pages 2025-10-25 07:42:05 +00:00
root
b4c4bf0a9e Improve calendar sync: Add Token Bucket rate limiting, security check to prevent syncing Advoware-sourced events back, fix random import 2025-10-24 23:56:00 +00:00
root
6ab7b4a376 Rate Limiting: Atomisches Lua Script zur Vermeidung von Race Conditions 2025-10-24 21:54:47 +00:00
root
774ed3fa0e Dokumentation: Globales Rate Limiting hinzugefügt 2025-10-24 20:44:20 +00:00
root
3c6d7e13c6 Globales Rate Limiting: 600 Anfragen/Minute über gleitendes Fenster mit Redis 2025-10-24 20:43:46 +00:00
root
dcb2dba50f API-Schutz: 100ms Verzögerung bei allen Google API Calls (create, update, delete) 2025-10-24 19:58:18 +00:00
root
2bf37b8616 Fix: Korrekte Endzeiten für recurring Events
- parse_times erkennt jetzt recurring Events (dauertermin=1 + turnus/turnusArt > 0)
- Für recurring Events wird das Ende aus dem gleichen Tag wie Start berechnet
- datumBis wird nur noch für die RRULE-Serie verwendet, nicht für jeden Termin
- Fix für Termin 84389: Statt 15-Tage-Terminen jetzt korrekte 30-Minuten-Termine
2025-10-24 19:48:12 +00:00
root
d154ba8172 Fix: ACL-Prüfung bei existierenden Google-Kalendern
- ensure_google_calendar prüft jetzt auch bei existierenden Kalendern die ACL-Regel
- Fügt fehlende ACL-Regel hinzu, falls sie nicht vorhanden ist
- Verhindert Sync-Abbruch bei unvollständigen Kalendern aus früheren Läufen
2025-10-24 19:27:59 +00:00
root
9d40f47e19 Refaktorierung Calendar Sync: Event-driven Design, Fixes für mehrtägige Termine, Logging und Locking
- Refaktorierung zu event-driven Ansatz ohne PostgreSQL Hub
- Fixes für mehrtägige Termine: korrekte Verwendung von datumBis, Entfernung 24h-Limit
- Per-Employee Locking mit Redis
- Logging via context.logger für Motia Workbench
- Neue Schritte: calendar_sync_all_step.py, calendar_sync_cron_step.py
- Aktualisiertes README.md mit aktueller Architektur
- Workbench-Gruppierung: advoware-calendar-sync
2025-10-24 19:13:41 +00:00
root
f4490f21cb Fix logging to appear in Motia workbench
- Updated log_operation to use context.logger.warn for warnings
- Added context parameter to all functions with logging
- Replaced all direct logger calls with log_operation calls
- Ensured all logging goes through context.logger for workbench visibility
- Adjusted backoff base from 4 to 3 for faster retries
- Added debug kuerzel list support in cron step
2025-10-24 07:04:57 +00:00
root
72ee01b74b Add recreated count to sync statistics 2025-10-24 06:32:33 +00:00
root
858d3d4fb3 Add final statistics logging to event step showing operations performed 2025-10-24 06:16:02 +00:00
root
c14273cac9 Integrate delete calendar functionality into audit script 2025-10-24 01:11:06 +00:00
root
25429edd76 Extend debug mode in cron step to accept list of kuerzel (SB, AI, RO) 2025-10-24 01:08:09 +00:00
root
2eb8330b1d Implement atomic locking in API and cron steps for per-employee parallel sync 2025-10-24 00:48:25 +00:00
root
9a1eb5bf0b Remove lock checking and setting from event step, keep only deletion in finally 2025-10-24 00:44:31 +00:00
root
f6bcbe664c Fix event handler signature to async def handler(event_data, context) 2025-10-24 00:38:12 +00:00
root
d18187f3aa Change event topic to calendar_sync_employee without dots to fix INVALID EMIT 2025-10-24 00:29:09 +00:00
root
6d2089ec69 Ensure Redis lock is always released with finally block 2025-10-24 00:26:18 +00:00
root
9ca3191542 Add flows to API and cron steps to fix INVALID EMIT error 2025-10-24 00:22:51 +00:00
root
b70619ab2d Update API step to trigger sync for specific employee with kuerzel 2025-10-24 00:20:41 +00:00
root
8e9dc87b2a Refactor calendar sync for parallel processing: cron emits per-employee events, event-step processes one employee with per-employee locks 2025-10-24 00:19:34 +00:00
root
409bea3615 Update logging to use Motia workbench context for visibility 2025-10-24 00:12:53 +00:00
root
ddad58faa3 Refactor calendar sync phases into separate functions for better modularity 2025-10-24 00:10:09 +00:00
root
c91d3fc76d Fix 'adv_map' not defined error by correcting state dict references 2025-10-24 00:08:18 +00:00
root
11ca7a5e75 Add missing phase functions to event step 2025-10-23 23:58:53 +00:00
root
3a1ed1ee55 Refactor calendar sync: move employee loop to cron step, parallel sync per employee 2025-10-23 23:57:51 +00:00
root
d15bd04167 Fix variable references in calendar sync after refactoring 2025-10-23 23:53:38 +00:00
root
a6d061a708 Refactor calendar sync: reduce redundancies, remove dead code, improve structure
- Remove unused share_google_calendar function
- Add centralized log_operation for consistent logging with context
- Add get_timestamps helper for Phase 4 timestamp handling
- Consolidate safe_* functions into generic safe_advoware_operation
- Optimize reload functions to handle initial fetches
- Split standardize_appointment_data into parse_times, adjust_times, build_notiz helpers
- Use state dict for better encapsulation in handler
2025-10-23 23:48:16 +00:00
root
42949c6871 Fix systemd service for automatic Motia startup 2025-10-23 23:37:52 +00:00
root
5ef77c91d5 Fix Advoware time filtering pitfall: extend ranges (Advoware -1 to +9 years, Google -2 to +10 years) and document issue in README 2025-10-23 23:28:49 +00:00
root
db1206f91c Fix recurring event duplication - handle recurringEventId properly in all phases 2025-10-23 18:46:49 +00:00
root
da2b9960b0 Fix recurring event duplication - handle recurringEventId properly in all phases 2025-10-23 18:30:46 +00:00
root
582c9422dc Fix recurring event duration bug - use datum instead of datumBis for event end time 2025-10-23 18:15:58 +00:00
root
ddd64a5e0f Fix critical bug: End date for recurring events should use 'datum' not 'datumBis'
- Fixed: For recurring appointments (dauertermin=1), end date now correctly uses 'datum' instead of 'datumBis'
- 'datumBis' is only for recurrence series end, not individual event duration
- Events now have correct duration instead of being capped at 24h unnecessarily
- Updated README with detailed explanation of the bug and fix
- Time breakdowns in descriptions are now accurate
2025-10-23 18:12:02 +00:00
root
08b21ef268 Fix cron handler signature: remove event parameter from cron step
- Fix calendar_sync_cron_step.py handler signature from (event, context) to (context)
- Cron steps only receive context parameter, not event
2025-10-23 17:33:58 +00:00
root
080eebd5ea Fix calendar sync issues: correct True/False, limit recurrence to 2 years, add share_google_calendar function
- Fix Python syntax error: change 'true' to 'True' in config.py
- Limit recurring events to max 2 years to avoid Google Calendar API limits
- Add share_google_calendar function for manual calendar sharing
- Update README with calendar sync documentation
- Debug mode: limit sync to user AI only
2025-10-23 13:53:51 +00:00
root
974116c024 Sync all employees in loop, remove employee_kuerzel requirement, update README 2025-10-23 13:02:52 +00:00
root
7b6e287b66 Remove AI limit, update README, fix update loops by setting last_sync to post-update time 2025-10-23 12:57:33 +00:00
root
48ce8d8e73 Add exponential backoff for Google API calls
- Use backoff library with 5s base, max 5 tries for HttpError 429/5xx
- Decorate Google functions: ensure_calendar, fetch_events, create/update/delete events
- Add backoff to requirements.txt
2025-10-23 12:49:45 +00:00