Commit Graph

135 Commits

Author SHA1 Message Date
bsiggel
e30f55b4e5 fix: handle unsupported document types in RAGflow sync and improve logging 2026-04-01 12:06:50 +00:00
bsiggel
4026194fc5 feat: enhance document update webhook to trigger akte.sync and improve logging 2026-04-01 10:45:43 +00:00
bsiggel
9639a375a6 fix: ensure syncStatus is reset to 'synced' for unchanged files in Advoware sync process 2026-03-31 10:40:14 +00:00
bsiggel
f9c8aa7c6d fix: update DocumentSync initialization to pass EspoCRMAPI as first argument 2026-03-31 08:04:56 +00:00
bsiggel
3086afa855 feat: implement Akte Sync event handler with preview generation trigger 2026-03-31 07:19:08 +00:00
bsiggel
0bd397803e fix: reset syncStatus to 'synced' when file is confirmed in-sync in AdvowareDocumentSyncUtils 2026-03-31 07:14:52 +00:00
bsiggel
8df214afb0 fix: add section header separator (=*80) to calendar_sync_api handler entry per INDEX.md logging convention 2026-03-31 07:14:24 +00:00
bsiggel
96c1588a87 fix: move deferred inline imports (traceback, datetime, pytz) to module top-level in beteiligte and bankverbindungen sync steps 2026-03-31 07:13:45 +00:00
bsiggel
00e95b7526 fix: replace bare except clauses with typed except (ValueError, AttributeError) in calendar_sync_event_step 2026-03-31 07:11:39 +00:00
bsiggel
6595a50561 fix: add missing enqueues key to ragflow_parsing_status_cron_step config 2026-03-31 07:10:47 +00:00
bsiggel
f3f3eff796 fix: add missing enqueues key to ragflow_graph_build_cron_step config 2026-03-31 07:10:43 +00:00
bsiggel
c6996e04ea fix: rename handler parameter input_data → event_data in calendar_sync queue handlers per INDEX.md convention 2026-03-31 07:09:58 +00:00
bsiggel
7560953e33 fix: correct cron handler signature input_data: None in beteiligte_sync_cron_step (cron receives no input) 2026-03-31 07:09:02 +00:00
bsiggel
e1695e530a fix: move calendar_sync_utils to services/ and remove sys.path.insert hacks from all calendar sync steps 2026-03-31 07:08:21 +00:00
bsiggel
cc70189513 fix: replace log_operation() with ctx.logger in calendar_sync_all_step – comply with INDEX.md logging standard 2026-03-31 07:03:04 +00:00
bsiggel
1f302e0de8 fix: replace log_operation() with ctx.logger/context.logger in calendar_sync_event_step – comply with INDEX.md logging standard 2026-03-31 07:02:41 +00:00
bsiggel
c5299b2889 fix: replace step_logger with ctx.logger in beteiligte_sync_event_step – use consistent context-aware logging 2026-03-31 07:00:59 +00:00
bsiggel
3ade3fc3cc fix: add try/finally safety net for lock release in beteiligte_sync handler – guarantees Redis key cleanup on any exit path 2026-03-31 07:00:35 +00:00
bsiggel
814e869ca0 fix: add try/finally for lock release in bankverbindungen_sync handler – prevents 15-min lock leaks on exception 2026-03-31 06:59:03 +00:00
bsiggel
7a0ebb949d fix: replace return dicts with raise in calendar_sync_all queue handler – enables Motia retry on failure 2026-03-31 06:57:17 +00:00
bsiggel
4ed6259ed9 fix: remove duplicate akte_sync_event_step (src/steps/akte/) – same queue trigger as crm/akte version causes double processing 2026-03-31 06:55:54 +00:00
bsiggel
141eabae6d feat(sync): Enhance metadata synchronization for unchanged files in Advoware sync process 2026-03-31 06:26:33 +00:00
bsiggel
52c4f84b44 feat(ragflow): Remove unsupported MIME types from the upload filter 2026-03-31 06:01:54 +00:00
bsiggel
a5ca63ea52 feat(ragflow): Add MIME type filter to skip unsupported file uploads 2026-03-31 06:01:10 +00:00
bsiggel
b6a0bd8d80 Refactor code structure for improved readability and maintainability 2026-03-31 05:58:20 +00:00
bsiggel
0516648bc0 feat(config): Increase default timeout for RestApiModule to 300000 ms
feat(dependencies): Add langfuse for enhanced observability and tracing
feat(uv.lock): Include langfuse package details in lock file
2026-03-30 22:07:51 +00:00
bsiggel
b4ff370823 feat(graphiti): Integrate Langfuse Tracer for enhanced monitoring and tracing of Graphiti operations
feat(graphiti): Define entity and edge types in new graphiti_schema for structured data extraction
feat(graphiti): Enhance ingest_episode_event_step with schema integration for improved episode processing
2026-03-30 22:00:53 +00:00
bsiggel
e255ae1263 feat(graphiti): Implement Graphiti client and related steps for episode ingestion and querying
- Added `graphiti_client.py` to manage the Graphiti client as a singleton.
- Created `ingest_episode_event_step.py` to handle episode ingestion from HTTP webhook events.
- Implemented `ingest_episode_step.py` for validating and enqueuing episode payloads via a POST request.
- Developed `query_graph_step.py` for performing semantic searches in the Graphiti Knowledge-Graph.
- Introduced an `__init__.py` file for the graphiti steps module.
2026-03-30 08:25:49 +00:00
bsiggel
1271e38f2d feat(cron): Update graphParsingStatus documentation and refine query conditions for new Graph builds 2026-03-27 11:29:06 +00:00
bsiggel
88c9df5995 feat(cron): Add RAGflow Graph Build Cron for periodic status updates and new builds 2026-03-27 11:27:09 +00:00
bsiggel
a2181a25fc feat(sync): Implement RAGflow Parsing Status Poller for syncing document statuses with EspoCRM 2026-03-27 10:12:52 +00:00
bsiggel
c20baeb21a feat(sync): Add EML to TXT conversion for improved document handling in RAGflow sync 2026-03-27 01:23:52 +00:00
bsiggel
61113d8f3d feat(sync): Update RAGFlow dataset creation to use stable EspoCRM-ID and improve logging 2026-03-27 00:52:48 +00:00
bsiggel
9bd62fc5ab feat(sync): Enhance Akte Sync with RAGflow support and improve error handling 2026-03-26 23:09:42 +00:00
bsiggel
1cd8de8574 Refactor code structure for improved readability and maintainability 2026-03-26 22:24:07 +00:00
bsiggel
9b2fb5ae4a feat: Implement AI Knowledge Sync Utilities and RAGFlow Service
- Added `aiknowledge_sync_utils.py` for provider-agnostic synchronization logic for CAIKnowledge entities, supporting both xAI and RAGFlow.
- Introduced lifecycle management for CAIKnowledge entities including states: new, active, paused, and deactivated.
- Implemented change detection using Blake3 hash for efficient document synchronization.
- Created `ragflow_service.py` to handle dataset and document management with RAGFlow API.
- Added daily cron job in `aiknowledge_daily_cron_step.py` to synchronize active CAIKnowledge entities with unclean or failed statuses.
- Developed `aiknowledge_sync_event_step.py` to process synchronization events from webhooks and cron jobs.
2026-03-26 21:38:42 +00:00
bsiggel
439101f35d feat(sync): Update document preview trigger to use enqueue method and improve content change detection for xAI re-sync 2026-03-26 16:17:52 +00:00
bsiggel
5e9c791a1b feat(sync): Remove unused file renaming method from XAIService 2026-03-26 14:33:34 +00:00
bsiggel
6682b0bd1f feat(sync): Remove redundant file renaming logic after upload when hash matches 2026-03-26 14:32:04 +00:00
bsiggel
1d0bd9d568 feat(upload): Update document metadata handling to exclude empty fields during upload 2026-03-26 14:26:06 +00:00
bsiggel
c9bdd021e4 feat(sync): Implement orphan cleanup for xAI documents without EspoCRM equivalents 2026-03-26 14:20:33 +00:00
bsiggel
1e202a6233 feat(sync): Update xAI collection file addition endpoint and improve documentation 2026-03-26 13:22:14 +00:00
bsiggel
459fa41033 feat(sync): Refactor Akte sync status handling and remove deprecated event step 2026-03-26 13:06:32 +00:00
bsiggel
52cee5bd16 feat(upload): Enhance document metadata handling with additional fields for better context 2026-03-26 12:51:04 +00:00
bsiggel
b320f01255 feat(sync): Enhance xAI sync process with collection verification and creation logic 2026-03-26 12:42:35 +00:00
bsiggel
a6dc708954 feat(espocrm): Implement automatic pagination for related records and enforce API max page size 2026-03-26 12:41:45 +00:00
bsiggel
d9193f7993 feat(sync): Update Akte sync process to remove unused entity ID queue and streamline processing 2026-03-26 11:22:04 +00:00
bsiggel
ef32373dc9 feat(sync): Enhance Akte sync process with batch processing and retry logic for failed events 2026-03-26 11:13:37 +00:00
bsiggel
52114a3c95 feat(webhooks): Update Akte webhook handlers to trigger immediate synchronization 2026-03-26 10:16:33 +00:00
bsiggel
bf02b1a4e1 feat(webhooks): Implement Akte webhooks for create, delete, and update operations 2026-03-26 10:16:27 +00:00