Commit Graph

17 Commits

Author SHA1 Message Date
bsiggel
bcb6454b2a Add comprehensive test scripts for thumbnail generation and xAI collections API
- Implemented `test_thumbnail_generation.py` to validate the complete flow of document thumbnail generation in EspoCRM, including document creation, file upload, webhook triggering, and preview verification.
- Created `test_xai_collections_api.py` to test critical operations of the xAI Collections API, covering file uploads, collection CRUD operations, document management, and response validation.
- Both scripts include detailed logging for success and error states, ensuring robust testing and easier debugging.
2026-03-03 17:03:08 +00:00
bsiggel
c45bfb7233 Enhance EspoCRM API and Webhook Handling
- Improved logging for file uploads in EspoCRMAPI to include upload parameters and error details.
- Updated cron job configurations for calendar sync and participant sync to trigger every 15 minutes on the first minute of the hour.
- Enhanced document create, delete, and update webhook handlers to determine and log the entity type.
- Refactored document sync event handler to include entity type in sync operations and logging.
- Added a new test script for uploading preview images to EspoCRM and verifying the upload process.
- Created a test script for document thumbnail generation, including document creation, file upload, webhook triggering, and preview verification.
2026-03-03 16:53:55 +00:00
bsiggel
0e521f22f8 feat(preview-generation): implement thumbnail generation for documents; add preview upload to EspoCRM 2026-03-03 09:28:49 +00:00
bsiggel
ee9aab049f feat(document-sync): add Document Sync Utilities and VMH Document Sync Handler for xAI integration 2026-03-03 06:55:54 +00:00
bsiggel
cb0e170ee9 feat(webhooks): update EspoCRM webhook configurations to use VMH naming and endpoints 2026-03-02 17:34:17 +00:00
bsiggel
bc917bd885 feat(webhooks): add EspoCRM Document Delete and Update webhook handlers with detailed logging
- Implemented document delete webhook handler to process delete notifications from EspoCRM.
- Added detailed logging for incoming payloads, including headers and entity information.
- Extracted and emitted document IDs for further processing.
- Implemented document update webhook handler to handle update notifications from EspoCRM.
- Enhanced logging for update payloads, including changed fields and metadata.
- Both handlers emit events for queue processing to facilitate further actions.

test: add comprehensive xAI Collections API test suite

- Created a test suite for xAI Collections API covering critical operations.
- Included tests for file uploads, CRUD operations on collections and documents, and response structures.
- Verified shared file behavior across multiple collections.
- Implemented cleanup procedures to remove test resources after execution.
2026-03-02 17:16:07 +00:00
bsiggel
0282149613 Fix cron expression in Calendar Sync Cron Job to trigger at second 0 and update entity retrieval methods in Beteiligte Sync steps for consistency 2026-03-02 09:36:09 +00:00
bsiggel
0740952063 Fix cron expression in Calendar Sync Cron Job to trigger every 15 minutes 2026-03-01 23:47:36 +00:00
bsiggel
f1ac5ffc7e Update cron description in Calendar Sync Cron Job for clarity 2026-03-01 23:46:51 +00:00
bsiggel
721339ca9b Update flow names for Advoware and VMH steps to improve clarity and organization 2026-03-01 23:28:29 +00:00
bsiggel
dc586385df Refactor logging in log_operation to support context-based logging and fallback to standard logger 2026-03-01 23:15:14 +00:00
bsiggel
0b8da01b71 Update README and add detailed documentation for Advoware integration and calendar sync 2026-03-01 23:03:01 +00:00
bsiggel
52356e634e Add calendar sync utilities and Beteiligte sync cron job
- Implemented calendar_sync_utils.py for shared utility functions
  including DB connection, Google Calendar service initialization,
  Redis client setup, and employee sync operations.

- Created beteiligte_sync_cron_step.py to handle periodic sync
  of Beteiligte entities, checking for new, modified, failed,
  and stale records, and emitting sync events accordingly.
2026-03-01 22:55:17 +00:00
bsiggel
014947e9e0 Migrate VMH Integration - Phase 3: Core sync handlers & utilities
- Added 5 core service modules:
  * services/notification_utils.py: NotificationManager for manual actions (412 lines)
  * services/advoware_service.py: Extended Advoware operations wrapper
  * services/espocrm_mapper.py: BeteiligteMapper for data transformation (198 lines)
  * services/bankverbindungen_mapper.py: BankverbindungenMapper (174 lines)
  * services/beteiligte_sync_utils.py: BeteiligteSync with complex logic (663 lines)
    - Distributed locking via Redis + syncStatus
    - rowId-based change detection (primary) + timestamp fallback
    - Retry with exponential backoff (1min, 5min, 15min, 1h, 4h)
    - Conflict resolution (EspoCRM wins)
    - Soft-delete handling & round-trip validation

- Added 2 event handler steps:
  * steps/vmh/beteiligte_sync_event_step.py: Handles vmh.beteiligte.* queue events
    - CREATE: New Beteiligte → POST to Advoware
    - UPDATE: Bi-directional sync with conflict resolution
    - DELETE: Not yet implemented
    - SYNC_CHECK: Periodic sync check via cron
  * steps/vmh/bankverbindungen_sync_event_step.py: Handles vmh.bankverbindungen.* events
    - CREATE: New Bankverbindung → POST to Advoware
    - UPDATE/DELETE: Send notification (API limitation - no PUT/DELETE support)

- Added pytz dependency to pyproject.toml (required for timezone handling)

System Status:
- 27 steps registered (14 VMH-specific)
- 8 queue subscriptions active (4 Beteiligte + 4 Bankverbindungen)
- All webhook endpoints operational and emitting queue events
- Sync handlers ready for processing

Note: Kommunikation sync (kommunikation_sync_utils.py) not yet migrated.
Complex bi-directional sync for Telefon/Email/Fax will be added in Phase 4.

Updated MIGRATION_STATUS.md
2026-03-01 22:19:36 +00:00
bsiggel
0216c4c3ae Migrate VMH Integration - Phase 2: Webhook endpoints
- Added services/espocrm.py: EspoCRM API client with Redis support
- Added 6 VMH webhook steps for EspoCRM integration:

  Beteiligte webhooks:
  - POST /vmh/webhook/beteiligte/create
  - POST /vmh/webhook/beteiligte/update
  - POST /vmh/webhook/beteiligte/delete

  Bankverbindungen webhooks:
  - POST /vmh/webhook/bankverbindungen/create
  - POST /vmh/webhook/bankverbindungen/update
  - POST /vmh/webhook/bankverbindungen/delete

All webhook endpoints receive batch/single entity notifications from
EspoCRM and emit queue events for downstream processing.

Note: Complex sync handlers (event processors) not yet migrated -
they require additional utility modules (beteiligte_sync_utils.py,
mappers, notification_utils) which will be migrated in Phase 3.

Updated MIGRATION_STATUS.md with Phase 2 completion.
2026-03-01 21:52:19 +00:00
bsiggel
a24282c346 Migrate Advoware Proxy steps from old-motia
- Added services/advoware.py: Advoware API client with Redis token caching
- Added advoware_proxy steps (GET, POST, PUT, DELETE)
- Updated pyproject.toml: added aiohttp, redis, python-dotenv
- Updated iii-config.yaml: fixed ExecModule command to use full path
- Created MIGRATION_STATUS.md: documentation of migration progress

All 4 proxy endpoints registered successfully:
- GET /advoware/proxy
- POST /advoware/proxy
- PUT /advoware/proxy
- DELETE /advoware/proxy
2026-03-01 21:47:43 +00:00
bsiggel
b3bdb56753 Initial commit: Motia III Backend Setup
- iii-config.yaml mit Production-Settings (CORS, all interfaces)
- Ticketing-System Steps (create, triage, escalate, notify, SLA monitoring)
- Python dependencies via uv
- Systemd services für Motia Engine und iii Console
- README mit Deployment-Info
2026-03-01 21:38:07 +00:00