Commit Graph

83 Commits

Author SHA1 Message Date
d10554ea9d feat: Update dateOfBirth mapping and enhance rowId handling in sync process for Beteiligte 2026-02-07 22:27:29 +00:00
d5bc17e454 feat: Remove redundant conflict resolution logging in handle_update function 2026-02-07 21:53:37 +00:00
bed3c09bb1 feat: Improve logging and rowId handling in sync process for Advoware and EspoCRM 2026-02-07 21:52:58 +00:00
99fb2e22c7 feat: Enhance mapping logic in BeteiligteMapper for additional fields and improved comments 2026-02-07 21:36:50 +00:00
f3d41dbb7f feat: Add comprehensive documentation for Advoware Beteiligte API field support and best practices 2026-02-07 21:36:11 +00:00
32c3dc1c37 feat: Enhance sync process with rowId-based change detection and update handling for Advoware and EspoCRM 2026-02-07 21:26:03 +00:00
3d3014750f feat: Enhance sync conflict detection and resolution logic in BeteiligteSync class 2026-02-07 20:04:58 +00:00
101f290293 feat: Implement entity comparison logic for improved sync detection between EspoCRM and Advoware 2026-02-07 19:41:20 +00:00
9076688f58 fix: Replace warning with warn method in logger for consistency in Beteiligte sync steps 2026-02-07 19:18:38 +00:00
46f9301a17 feat: Update advowareLastSync to use EspoCRM datetime format for consistency in sync operations 2026-02-07 19:14:57 +00:00
3354aef936 feat: Refactor webhook handlers for Beteiligte to improve ID processing and logging, and enhance update filtering logic 2026-02-07 18:53:54 +00:00
ae1d96f767 Add sync strategy documentation and templates for bidirectional sync between EspoCRM and Advoware
- Introduced SYNC_STRATEGY_ARCHIVE.md detailing the sync process, status values, and flow for updating entities from EspoCRM to Advoware and vice versa.
- Created SYNC_TEMPLATE.md as a guide for implementing new syncs, including field definitions, mapper examples, sync utilities, event handlers, and cron jobs.
- Added README_SYNC.md for the Beteiligte sync event handler, outlining its functionality, event subscriptions, optimizations, error handling, and performance metrics.
2026-02-07 15:54:13 +00:00
8550107b89 feat: Enhance Advoware API integration with backward compatibility for data payloads and improve logging for sync events 2026-02-07 15:44:56 +00:00
b5abe6cf00 Implement EspoCRM-based sync strategy for Beteiligte entities
- Add SYNC_STRATEGY_ESPOCRM_BASED.md detailing the sync flows and status management.
- Create utilities for sync operations in services/beteiligte_sync_utils.py, including locking, timestamp comparison, conflict resolution, and notification handling.
- Implement entity mapping between EspoCRM and Advoware in services/espocrm_mapper.py.
- Develop a cron job for periodic sync checks in steps/vmh/beteiligte_sync_cron_step.py, emitting events for entities needing synchronization.
2026-02-07 15:21:16 +00:00
e6ab22d5f4 feat: Add EspoCRM and Advoware integration for Beteiligte comparison
- Implemented `compare_beteiligte.py` script for comparing Beteiligte structures between EspoCRM and Advoware.
- Created `beteiligte_comparison_result.json` to store comparison results.
- Developed `EspoCRMAPI` service for handling API interactions with EspoCRM.
- Added comprehensive documentation for the EspoCRM API service.
- Included error handling and logging for API operations.
- Enhanced entity management with CRUD operations and search capabilities.
2026-02-07 14:42:58 +00:00
36552903e7 cleanup 2026-02-07 09:23:49 +00:00
root
96eabe3db6 Refactor calendar sync to prioritize oldest synced employees with human-readable timestamps 2025-10-26 08:58:48 +00:00
root
b18e770f12 refactor: extract common functions to utils
- Add get_redis_client() to calendar_sync_utils.py
- Add get_advoware_employees() to calendar_sync_utils.py
- Add set_employee_lock() and clear_employee_lock() to calendar_sync_utils.py
- Update all step files to use shared utility functions
- Remove duplicate code across calendar_sync_*.py files
2025-10-25 09:21:45 +00:00
root
e4bf21e676 Refactor: Extract common utilities to shared module
- Created calendar_sync_utils.py with shared functions:
  - log_operation: Centralized logging with context support
  - connect_db: Database connection wrapper
  - get_google_service: Google Calendar API initialization
- Updated imports in calendar_sync_event_step.py and audit_calendar_sync.py
- Removed duplicated function definitions
- Maintained function logic without changes
- Improved code maintainability and reduced duplications
2025-10-25 08:54:54 +00:00
root
c5600b42ec feat: Enhance audit_calendar_sync tool with comprehensive management features
- Add detailed documentation in README.md for all 10 audit commands
- Fix Advoware API parameter from 'frNr' to 'frnr' for proper filtering
- Fix subject field mapping from 'betreff' to 'text' in API responses
- Add verify-sync command for bidirectional sync verification
- Add query-frnr and query-event commands for individual record lookup
- Add management commands: find-duplicates, delete-duplicates, find-orphaned, cleanup-orphaned
- Improve error handling and output formatting
- Remove temporary test script
2025-10-25 08:18:48 +00:00
root
c5ddd02307 Add query and verification features for individual appointments by frNr or Google Event ID 2025-10-25 07:53:43 +00:00
root
1539c26be6 Expand audit_calendar_sync.py to comprehensive management tool with pagination, duplicate detection/deletion, and orphaned calendar cleanup 2025-10-25 07:48:35 +00:00
root
62f57bb035 Fix ensure_google_calendar to fetch all calendars with pagination to avoid missing calendars on later pages 2025-10-25 07:42:05 +00:00
root
b4c4bf0a9e Improve calendar sync: Add Token Bucket rate limiting, security check to prevent syncing Advoware-sourced events back, fix random import 2025-10-24 23:56:00 +00:00
root
6ab7b4a376 Rate Limiting: Atomisches Lua Script zur Vermeidung von Race Conditions 2025-10-24 21:54:47 +00:00
root
774ed3fa0e Dokumentation: Globales Rate Limiting hinzugefügt 2025-10-24 20:44:20 +00:00
root
3c6d7e13c6 Globales Rate Limiting: 600 Anfragen/Minute über gleitendes Fenster mit Redis 2025-10-24 20:43:46 +00:00
root
dcb2dba50f API-Schutz: 100ms Verzögerung bei allen Google API Calls (create, update, delete) 2025-10-24 19:58:18 +00:00
root
2bf37b8616 Fix: Korrekte Endzeiten für recurring Events
- parse_times erkennt jetzt recurring Events (dauertermin=1 + turnus/turnusArt > 0)
- Für recurring Events wird das Ende aus dem gleichen Tag wie Start berechnet
- datumBis wird nur noch für die RRULE-Serie verwendet, nicht für jeden Termin
- Fix für Termin 84389: Statt 15-Tage-Terminen jetzt korrekte 30-Minuten-Termine
2025-10-24 19:48:12 +00:00
root
d154ba8172 Fix: ACL-Prüfung bei existierenden Google-Kalendern
- ensure_google_calendar prüft jetzt auch bei existierenden Kalendern die ACL-Regel
- Fügt fehlende ACL-Regel hinzu, falls sie nicht vorhanden ist
- Verhindert Sync-Abbruch bei unvollständigen Kalendern aus früheren Läufen
2025-10-24 19:27:59 +00:00
root
9d40f47e19 Refaktorierung Calendar Sync: Event-driven Design, Fixes für mehrtägige Termine, Logging und Locking
- Refaktorierung zu event-driven Ansatz ohne PostgreSQL Hub
- Fixes für mehrtägige Termine: korrekte Verwendung von datumBis, Entfernung 24h-Limit
- Per-Employee Locking mit Redis
- Logging via context.logger für Motia Workbench
- Neue Schritte: calendar_sync_all_step.py, calendar_sync_cron_step.py
- Aktualisiertes README.md mit aktueller Architektur
- Workbench-Gruppierung: advoware-calendar-sync
2025-10-24 19:13:41 +00:00
root
f4490f21cb Fix logging to appear in Motia workbench
- Updated log_operation to use context.logger.warn for warnings
- Added context parameter to all functions with logging
- Replaced all direct logger calls with log_operation calls
- Ensured all logging goes through context.logger for workbench visibility
- Adjusted backoff base from 4 to 3 for faster retries
- Added debug kuerzel list support in cron step
2025-10-24 07:04:57 +00:00
root
72ee01b74b Add recreated count to sync statistics 2025-10-24 06:32:33 +00:00
root
858d3d4fb3 Add final statistics logging to event step showing operations performed 2025-10-24 06:16:02 +00:00
root
c14273cac9 Integrate delete calendar functionality into audit script 2025-10-24 01:11:06 +00:00
root
25429edd76 Extend debug mode in cron step to accept list of kuerzel (SB, AI, RO) 2025-10-24 01:08:09 +00:00
root
2eb8330b1d Implement atomic locking in API and cron steps for per-employee parallel sync 2025-10-24 00:48:25 +00:00
root
9a1eb5bf0b Remove lock checking and setting from event step, keep only deletion in finally 2025-10-24 00:44:31 +00:00
root
f6bcbe664c Fix event handler signature to async def handler(event_data, context) 2025-10-24 00:38:12 +00:00
root
d18187f3aa Change event topic to calendar_sync_employee without dots to fix INVALID EMIT 2025-10-24 00:29:09 +00:00
root
6d2089ec69 Ensure Redis lock is always released with finally block 2025-10-24 00:26:18 +00:00
root
9ca3191542 Add flows to API and cron steps to fix INVALID EMIT error 2025-10-24 00:22:51 +00:00
root
b70619ab2d Update API step to trigger sync for specific employee with kuerzel 2025-10-24 00:20:41 +00:00
root
8e9dc87b2a Refactor calendar sync for parallel processing: cron emits per-employee events, event-step processes one employee with per-employee locks 2025-10-24 00:19:34 +00:00
root
409bea3615 Update logging to use Motia workbench context for visibility 2025-10-24 00:12:53 +00:00
root
ddad58faa3 Refactor calendar sync phases into separate functions for better modularity 2025-10-24 00:10:09 +00:00
root
c91d3fc76d Fix 'adv_map' not defined error by correcting state dict references 2025-10-24 00:08:18 +00:00
root
11ca7a5e75 Add missing phase functions to event step 2025-10-23 23:58:53 +00:00
root
3a1ed1ee55 Refactor calendar sync: move employee loop to cron step, parallel sync per employee 2025-10-23 23:57:51 +00:00
root
d15bd04167 Fix variable references in calendar sync after refactoring 2025-10-23 23:53:38 +00:00