Commit Graph

96 Commits

Author SHA1 Message Date
8de2654d74 feat(sync): Fix Var6 revert logic for direction='to_advoware' and enhance conflict handling 2026-02-08 22:07:55 +00:00
79e097be6f feat(sync): Implement auto-reset for permanently_failed entities and add retry backoff logic
- Added logic to reset permanently_failed entities that have reached their auto-reset threshold in `beteiligte_sync_cron_step.py`.
- Enhanced event handling in `beteiligte_sync_event_step.py` to skip retries if the next retry time has not been reached.
- Introduced validation checks after sync operations to ensure data consistency and integrity.
- Created detailed documentation outlining the fixes and their impacts on the sync process.
- Added scripts for analyzing sync issues and comparing entities to facilitate debugging and validation.
2026-02-08 21:12:00 +00:00
6e0e9a9730 feat: Enhance logging in sync utilities and add code validation script 2026-02-08 20:10:47 +00:00
bfe2f4f7e3 feat: Add Kommunikation-Sync documentation for bidirectional synchronization between EspoCRM and Advoware 2026-02-08 19:58:43 +00:00
ebbbf419ee feat: Implement bidirectional synchronization utilities for Advoware and EspoCRM communications
- Added KommunikationSyncManager class to handle synchronization logic.
- Implemented methods for loading data, computing diffs, and applying changes between Advoware and EspoCRM.
- Introduced 3-way diffing mechanism to intelligently resolve conflicts.
- Added helper methods for creating empty slots and detecting changes in communications.
- Enhanced logging for better traceability during synchronization processes.
2026-02-08 19:53:40 +00:00
da9a962858 feat: Integrate NotificationManager for handling notifications in sync operations 2026-02-08 14:42:33 +00:00
b4e41e7381 feat: Add test notification system for EspoCRM integration 2026-02-08 14:40:02 +00:00
c770f2c8ee feat: Implement address synchronization between EspoCRM and Advoware
- Add AdressenMapper for transforming addresses between EspoCRM and Advoware formats.
- Create AdressenSync class to handle address creation, update, and deletion synchronization.
- Introduce NotificationManager for managing manual intervention notifications in case of sync issues.
- Implement detailed logging for address sync operations and error handling.
- Ensure READ-ONLY field changes are detected and notified for manual resolution.
2026-02-08 14:29:29 +00:00
68c8b398aa feat: Implement VMH Bankverbindungen sync handlers and API steps for create, update, and delete operations 2026-02-08 12:49:14 +00:00
709456301c feat: Optimize initial sync logic and remove redundant rowId updates in sync process 2026-02-07 22:38:53 +00:00
7a7a322389 feat: Remove redundant update filtering logic in VMH Webhook handler 2026-02-07 22:29:33 +00:00
d10554ea9d feat: Update dateOfBirth mapping and enhance rowId handling in sync process for Beteiligte 2026-02-07 22:27:29 +00:00
d5bc17e454 feat: Remove redundant conflict resolution logging in handle_update function 2026-02-07 21:53:37 +00:00
bed3c09bb1 feat: Improve logging and rowId handling in sync process for Advoware and EspoCRM 2026-02-07 21:52:58 +00:00
99fb2e22c7 feat: Enhance mapping logic in BeteiligteMapper for additional fields and improved comments 2026-02-07 21:36:50 +00:00
f3d41dbb7f feat: Add comprehensive documentation for Advoware Beteiligte API field support and best practices 2026-02-07 21:36:11 +00:00
32c3dc1c37 feat: Enhance sync process with rowId-based change detection and update handling for Advoware and EspoCRM 2026-02-07 21:26:03 +00:00
3d3014750f feat: Enhance sync conflict detection and resolution logic in BeteiligteSync class 2026-02-07 20:04:58 +00:00
101f290293 feat: Implement entity comparison logic for improved sync detection between EspoCRM and Advoware 2026-02-07 19:41:20 +00:00
9076688f58 fix: Replace warning with warn method in logger for consistency in Beteiligte sync steps 2026-02-07 19:18:38 +00:00
46f9301a17 feat: Update advowareLastSync to use EspoCRM datetime format for consistency in sync operations 2026-02-07 19:14:57 +00:00
3354aef936 feat: Refactor webhook handlers for Beteiligte to improve ID processing and logging, and enhance update filtering logic 2026-02-07 18:53:54 +00:00
ae1d96f767 Add sync strategy documentation and templates for bidirectional sync between EspoCRM and Advoware
- Introduced SYNC_STRATEGY_ARCHIVE.md detailing the sync process, status values, and flow for updating entities from EspoCRM to Advoware and vice versa.
- Created SYNC_TEMPLATE.md as a guide for implementing new syncs, including field definitions, mapper examples, sync utilities, event handlers, and cron jobs.
- Added README_SYNC.md for the Beteiligte sync event handler, outlining its functionality, event subscriptions, optimizations, error handling, and performance metrics.
2026-02-07 15:54:13 +00:00
8550107b89 feat: Enhance Advoware API integration with backward compatibility for data payloads and improve logging for sync events 2026-02-07 15:44:56 +00:00
b5abe6cf00 Implement EspoCRM-based sync strategy for Beteiligte entities
- Add SYNC_STRATEGY_ESPOCRM_BASED.md detailing the sync flows and status management.
- Create utilities for sync operations in services/beteiligte_sync_utils.py, including locking, timestamp comparison, conflict resolution, and notification handling.
- Implement entity mapping between EspoCRM and Advoware in services/espocrm_mapper.py.
- Develop a cron job for periodic sync checks in steps/vmh/beteiligte_sync_cron_step.py, emitting events for entities needing synchronization.
2026-02-07 15:21:16 +00:00
e6ab22d5f4 feat: Add EspoCRM and Advoware integration for Beteiligte comparison
- Implemented `compare_beteiligte.py` script for comparing Beteiligte structures between EspoCRM and Advoware.
- Created `beteiligte_comparison_result.json` to store comparison results.
- Developed `EspoCRMAPI` service for handling API interactions with EspoCRM.
- Added comprehensive documentation for the EspoCRM API service.
- Included error handling and logging for API operations.
- Enhanced entity management with CRUD operations and search capabilities.
2026-02-07 14:42:58 +00:00
36552903e7 cleanup 2026-02-07 09:23:49 +00:00
root
96eabe3db6 Refactor calendar sync to prioritize oldest synced employees with human-readable timestamps 2025-10-26 08:58:48 +00:00
root
b18e770f12 refactor: extract common functions to utils
- Add get_redis_client() to calendar_sync_utils.py
- Add get_advoware_employees() to calendar_sync_utils.py
- Add set_employee_lock() and clear_employee_lock() to calendar_sync_utils.py
- Update all step files to use shared utility functions
- Remove duplicate code across calendar_sync_*.py files
2025-10-25 09:21:45 +00:00
root
e4bf21e676 Refactor: Extract common utilities to shared module
- Created calendar_sync_utils.py with shared functions:
  - log_operation: Centralized logging with context support
  - connect_db: Database connection wrapper
  - get_google_service: Google Calendar API initialization
- Updated imports in calendar_sync_event_step.py and audit_calendar_sync.py
- Removed duplicated function definitions
- Maintained function logic without changes
- Improved code maintainability and reduced duplications
2025-10-25 08:54:54 +00:00
root
c5600b42ec feat: Enhance audit_calendar_sync tool with comprehensive management features
- Add detailed documentation in README.md for all 10 audit commands
- Fix Advoware API parameter from 'frNr' to 'frnr' for proper filtering
- Fix subject field mapping from 'betreff' to 'text' in API responses
- Add verify-sync command for bidirectional sync verification
- Add query-frnr and query-event commands for individual record lookup
- Add management commands: find-duplicates, delete-duplicates, find-orphaned, cleanup-orphaned
- Improve error handling and output formatting
- Remove temporary test script
2025-10-25 08:18:48 +00:00
root
c5ddd02307 Add query and verification features for individual appointments by frNr or Google Event ID 2025-10-25 07:53:43 +00:00
root
1539c26be6 Expand audit_calendar_sync.py to comprehensive management tool with pagination, duplicate detection/deletion, and orphaned calendar cleanup 2025-10-25 07:48:35 +00:00
root
62f57bb035 Fix ensure_google_calendar to fetch all calendars with pagination to avoid missing calendars on later pages 2025-10-25 07:42:05 +00:00
root
b4c4bf0a9e Improve calendar sync: Add Token Bucket rate limiting, security check to prevent syncing Advoware-sourced events back, fix random import 2025-10-24 23:56:00 +00:00
root
6ab7b4a376 Rate Limiting: Atomisches Lua Script zur Vermeidung von Race Conditions 2025-10-24 21:54:47 +00:00
root
774ed3fa0e Dokumentation: Globales Rate Limiting hinzugefügt 2025-10-24 20:44:20 +00:00
root
3c6d7e13c6 Globales Rate Limiting: 600 Anfragen/Minute über gleitendes Fenster mit Redis 2025-10-24 20:43:46 +00:00
root
dcb2dba50f API-Schutz: 100ms Verzögerung bei allen Google API Calls (create, update, delete) 2025-10-24 19:58:18 +00:00
root
2bf37b8616 Fix: Korrekte Endzeiten für recurring Events
- parse_times erkennt jetzt recurring Events (dauertermin=1 + turnus/turnusArt > 0)
- Für recurring Events wird das Ende aus dem gleichen Tag wie Start berechnet
- datumBis wird nur noch für die RRULE-Serie verwendet, nicht für jeden Termin
- Fix für Termin 84389: Statt 15-Tage-Terminen jetzt korrekte 30-Minuten-Termine
2025-10-24 19:48:12 +00:00
root
d154ba8172 Fix: ACL-Prüfung bei existierenden Google-Kalendern
- ensure_google_calendar prüft jetzt auch bei existierenden Kalendern die ACL-Regel
- Fügt fehlende ACL-Regel hinzu, falls sie nicht vorhanden ist
- Verhindert Sync-Abbruch bei unvollständigen Kalendern aus früheren Läufen
2025-10-24 19:27:59 +00:00
root
9d40f47e19 Refaktorierung Calendar Sync: Event-driven Design, Fixes für mehrtägige Termine, Logging und Locking
- Refaktorierung zu event-driven Ansatz ohne PostgreSQL Hub
- Fixes für mehrtägige Termine: korrekte Verwendung von datumBis, Entfernung 24h-Limit
- Per-Employee Locking mit Redis
- Logging via context.logger für Motia Workbench
- Neue Schritte: calendar_sync_all_step.py, calendar_sync_cron_step.py
- Aktualisiertes README.md mit aktueller Architektur
- Workbench-Gruppierung: advoware-calendar-sync
2025-10-24 19:13:41 +00:00
root
f4490f21cb Fix logging to appear in Motia workbench
- Updated log_operation to use context.logger.warn for warnings
- Added context parameter to all functions with logging
- Replaced all direct logger calls with log_operation calls
- Ensured all logging goes through context.logger for workbench visibility
- Adjusted backoff base from 4 to 3 for faster retries
- Added debug kuerzel list support in cron step
2025-10-24 07:04:57 +00:00
root
72ee01b74b Add recreated count to sync statistics 2025-10-24 06:32:33 +00:00
root
858d3d4fb3 Add final statistics logging to event step showing operations performed 2025-10-24 06:16:02 +00:00
root
c14273cac9 Integrate delete calendar functionality into audit script 2025-10-24 01:11:06 +00:00
root
25429edd76 Extend debug mode in cron step to accept list of kuerzel (SB, AI, RO) 2025-10-24 01:08:09 +00:00
root
2eb8330b1d Implement atomic locking in API and cron steps for per-employee parallel sync 2025-10-24 00:48:25 +00:00
root
9a1eb5bf0b Remove lock checking and setting from event step, keep only deletion in finally 2025-10-24 00:44:31 +00:00
root
f6bcbe664c Fix event handler signature to async def handler(event_data, context) 2025-10-24 00:38:12 +00:00