Update README and add detailed documentation for Advoware integration and calendar sync

This commit is contained in:
bsiggel
2026-03-01 23:03:01 +00:00
parent 52356e634e
commit 0b8da01b71
4 changed files with 1076 additions and 15 deletions

263
README.md
View File

@@ -1,34 +1,207 @@
# Motia III - BitByLaw Backend # bitbylaw - Motia III Integration Platform
**Migration Complete: 21/21 Steps (100%)**
Event-driven Integration zwischen Advoware, EspoCRM und Google Calendar mit **Motia III v1.0-RC** (Pure Python).
## Quick Start
```bash
cd /opt/motia-iii/bitbylaw
# Start iii Engine
/opt/bin/iii -c iii-config.yaml
# Start iii Console (Web UI) - separate terminal
/opt/bin/iii-console --enable-flow --host 0.0.0.0 --port 3113 \
--engine-host localhost --engine-port 3111 --ws-port 3114
```
Open browser: `http://localhost:3113/`
## Migration Status
This project has been **fully migrated** from old Motia v0.17 (Node.js + Python) to **Motia III v1.0-RC** (Pure Python).
See:
- [MIGRATION_STATUS.md](MIGRATION_STATUS.md) - Progress overview
- [MIGRATION_COMPLETE_ANALYSIS.md](MIGRATION_COMPLETE_ANALYSIS.md) - Complete analysis
- [docs/INDEX.md](docs/INDEX.md) - Complete documentation index
## Komponenten
1. **Advoware API Proxy** (4 Steps) - REST-API-Proxy mit HMAC-512 Auth
- GET, POST, PUT, DELETE proxies
- [Details](steps/advoware_proxy/README.md)
2. **Calendar Sync** (4 Steps) - Bidirektionale Synchronisation Advoware ↔ Google
- Cron-triggered (every 15 min)
- Manual API trigger
- Per-employee sync
- [Details](steps/advoware_cal_sync/README.md)
3. **VMH Integration** (9 Steps) - EspoCRM Webhook-Receiver & Sync
- Beteiligte sync (bidirectional)
- Bankverbindungen sync
- Webhook handlers (create/update/delete)
- [Details](steps/vmh/README.md)
**Total: 17 Steps registered**
## Architektur
```
┌─────────────┐ ┌──────────┐ ┌────────────┐
│ EspoCRM │────▶│ Webhooks │────▶│ Redis │
│ (VMH) │ │ (HTTP) │ │ Locking │
└─────────────┘ └──────────┘ └────────────┘
┌─────────────┐ ┌──────────┐ │
│ Clients │────▶│ Proxy │────▶ │
│ │ │ API API │ │
└─────────────┘ └──────────┘ ▼
┌────────────┐
Cron (15min) │ Motia │
│ │ Steps │
└──────────────────────────▶│ (Python) │
└────────────┘
┌────────────┐
│ Advoware │
│ Google │
│ Calendar │
└────────────┘
```
## API Endpoints
**Advoware Proxy:**
- `GET/POST/PUT/DELETE /advoware/proxy?endpoint=...`
**Calendar Sync:**
- `POST /advoware/calendar/sync` - Manual trigger
```bash
# Sync single employee
curl -X POST "http://localhost:3111/advoware/calendar/sync" \
-H "Content-Type: application/json" \
-d '{"kuerzel": "PB"}'
# Sync all employees
curl -X POST "http://localhost:3111/advoware/calendar/sync" \
-H "Content-Type: application/json" \
-d '{"kuerzel": "ALL"}'
```
**VMH Webhooks:**
- `POST /vmh/webhook/beteiligte/create`
- `POST /vmh/webhook/beteiligte/update`
- `POST /vmh/webhook/beteiligte/delete`
- `POST /vmh/webhook/bankverbindungen/create`
- `POST /vmh/webhook/bankverbindungen/update`
- `POST /vmh/webhook/bankverbindungen/delete`
## Configuration
Environment variables loaded from systemd service or `.env` file:
```bash
# Advoware API
ADVOWARE_API_BASE_URL=https://www2.advo-net.net:90/
ADVOWARE_PRODUCT_ID=64
ADVOWARE_APP_ID=your_app_id
ADVOWARE_API_KEY=your_base64_hmac_key
ADVOWARE_KANZLEI=your_kanzlei
ADVOWARE_DATABASE=your_database
ADVOWARE_USER=api_user
ADVOWARE_ROLE=2
ADVOWARE_PASSWORD=your_password
ADVOWARE_TOKEN_LIFETIME_MINUTES=55
ADVOWARE_API_TIMEOUT_SECONDS=30
# Redis
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_DB_CALENDAR_SYNC=1
REDIS_TIMEOUT_SECONDS=5
# Google Calendar
GOOGLE_CALENDAR_SERVICE_ACCOUNT_PATH=/path/to/service-account.json
# PostgreSQL
POSTGRES_HOST=localhost
POSTGRES_PORT=5432
POSTGRES_USER=bitbylaw
POSTGRES_PASSWORD=your_password
POSTGRES_DATABASE=bitbylaw
```
See [docs/INDEX.md](docs/INDEX.md) for complete configuration guide.
Motia Backend-Anwendung powered by the iii engine.
## Project Structure ## Project Structure
``` ```
bitbylaw/ bitbylaw/
├── steps/ # Motia Steps (Business Logic) ├── docs/ # Documentation
├── iii-config.yaml # iii Engine Configuration
├── pyproject.toml # Python Dependencies
└── .venv/ # Python Virtual Environment
```
## Services ## Services & Ports
- **Motia API**: Port 3111 (https://api-motia.bitbylaw.com) - **iii Engine (API)**: Port 3111 (https://api-motia.bitbylaw.com)
- **iii Console**: Port 3113 (https://motia.bitbylaw.com) - **iii Console (Web UI)**: Port 3113 (https://motia.bitbylaw.com)
- **Streams/WebSocket**: Port 3114 - **Streams/WebSocket**: Port 3114
- **Redis**: Port 6379 (localhost)
- **PostgreSQL**: Port 5432 (localhost)eps)
│ ├── advoware_cal_sync/ # Calendar Sync (4 steps)
│ └── vmh/ # VMH Integration (9 steps)
├── services/ # Shared Services
│ ├── advoware_service.py # Advoware API Client
│ ├── espocrm.py # EspoCRM API Client
│ └── ... # Other services
├── iii-config.yaml # iii Engine Configuration
├── pyproject.toml # Python Dependencies (uv)
├── MIGRATION_STATUS.md # Migration progress
├── MIGRATION_COMPLETE_ANALYSIS.md # Migration analysis
└── .venv/ # Python Virtual Environment
```
## Systemd Services ## Systemd Services
- `motia.service` - Backend Engine ## Systemd Services
- `iii-console.service` - Observability Dashboard
- `motia.service` - iii Engine (Backend)
- `iii-console.service` - iii Console (Observability Dashboard)
### Service Management
```bash
# Status
systemctl status motia.service
systemctl status iii-console.service
# Restart
## Development ## Development
```bash ```bash
# Install dependencies # Install dependencies
cd /opt/motia-iii/bitbylaw
uv sync uv sync
# Start iii Engine (development)
iii -c iii-config.yaml
# Start iii Console (separate terminal)
iii-console --enable-flow --host 0.0.0.0 --port 3113 \
--engine-host localhost --engine-port 3111 --ws-port 3114
# Test step import
uv run python -c "from steps.advoware_proxy import advoware_api_proxy_get_step"
# Check registered steps
curl http://localhost:3111/_console/functions
# Start locally # Start locally
iii -c iii-config.yaml iii -c iii-config.yaml
``` ```
@@ -37,10 +210,70 @@ iii -c iii-config.yaml
Services run automatically via systemd on boot. Services run automatically via systemd on boot.
```bash
# Restart services
systemctl restart motia.service iii-console.service
## Technology Stack
- **Framework**: Motia III v1.0-RC (Pure Python, iii Engine)
- **Language**: Python 3.13
- **Package Manager**: uv
- **Data Store**: Redis (Caching, Locking), PostgreSQL (Sync State)
- **External APIs**:
- Advoware REST API (HMAC-512 auth)
- Google Calendar API (Service Account)
- EspoCRM API (X-Api-Key auth)
## Documentation
### Getting Started
- [Documentation Index](docs/INDEX.md) - Complete index
- [Migration Status](MIGRATION_STATUS.md) - 100% complete
- [Migration Analysis](MIGRATION_COMPLETE_ANALYSIS.md) - Complete analysis
### Technical Details
- [Architecture](docs/ARCHITECTURE.md) - System design (Motia III)
- [Advoware Proxy](steps/advoware_proxy/README.md) - API Proxy details
- [Calendar Sync](steps/advoware_cal_sync/README.md) - Sync logic
- [VMH Integration](steps/vmh/README.md) - Webhook handlers
### Migration
- [Migration Guide](../MIGRATION_GUIDE.md) - Old Motia → Motia III patterns
## Testing
```bash
# Test HTTP endpoints
curl http://localhost:3111/advoware/proxy?endpoint=employees
# Trigger calendar sync manually
curl -X POST "http://localhost:3111/advoware/calendar/sync" \
-H "Content-Type: application/json" \
-d '{"kuerzel": "ALL"}'
# Check registered functions
curl http://localhost:3111/_console/functions | grep "Calendar"
# View logs in Console
open http://localhost:3113/
```
## Production
Services run automatically via systemd on boot.
**Deployed on**: motia.bitbylaw.com
**Deployment Date**: März 2026
```bash
# Restart production services
sudo systemctl restart motia.service iii-console.service
# View production logs
journalctl -u motia.service -f
journalctl -u iii-console.service -f
# Check service status
systemctl status motia.service iii-console.service
```
# View logs # View logs
journalctl -u motia.service -f journalctl -u motia.service -f
journalctl -u iii-console.service -f journalctl -u iii-console.service -f

246
docs/INDEX.md Normal file
View File

@@ -0,0 +1,246 @@
# Documentation Index - Motia III
## Getting Started
**New to the project?** Start here:
1. [README.md](../README.md) - Project Overview & Quick Start
2. [MIGRATION_STATUS.md](../MIGRATION_STATUS.md) - Migration Progress (100% Complete!)
3. [MIGRATION_COMPLETE_ANALYSIS.md](../MIGRATION_COMPLETE_ANALYSIS.md) - Complete Migration Analysis
## Migration to Motia III
**Status: ✅ 100% Complete (21/21 Steps migrated)**
- **[MIGRATION_GUIDE.md](../MIGRATION_GUIDE.md)** - Complete migration patterns
- Old Motia v0.17 → Motia III v1.0-RC
- TypeScript + Python Hybrid → Pure Python
- Configuration changes, trigger patterns, API differences
- **[MIGRATION_STATUS.md](../MIGRATION_STATUS.md)** - Current migration status
- **[MIGRATION_COMPLETE_ANALYSIS.md](../MIGRATION_COMPLETE_ANALYSIS.md)** - Complete analysis
## Core Documentation
### For Developers
- **[ARCHITECTURE.md](ARCHITECTURE.md)** - System design and architecture
- Components, Data Flow, Event-Driven Design
- Updated for Motia III patterns
### For Operations
- **[DEPLOYMENT.md](DEPLOYMENT.md)** - Production deployment (if available)
- Installation, systemd, nginx, Monitoring
- **[CONFIGURATION.md](CONFIGURATION.md)** - Environment configuration (if available)
- All environment variables, secrets management
## Component Documentation
### Steps (Business Logic)
**Advoware Proxy** ([Module README](../steps/advoware_proxy/README.md)):
Universal HTTP proxy for Advoware API with automatic authentication.
- GET, POST, PUT, DELETE proxies
- HMAC-512 authentication
- Redis token caching
**Calendar Sync** ([Module README](../steps/advoware_cal_sync/README.md)):
Bidirectional sync between Advoware appointments and Google Calendar.
- `calendar_sync_cron_step.py` - Auto trigger (every 15 min)
- `calendar_sync_api_step.py` - Manual trigger endpoint
- `calendar_sync_all_step.py` - Employee cascade handler
- `calendar_sync_event_step.py` - Per-employee sync logic (1053 lines)
**VMH Integration** ([Module README](../steps/vmh/README.md)):
Webhooks and bidirectional sync between EspoCRM and Advoware.
- **Beteiligte Sync** (Bidirectional EspoCRM ↔ Advoware)
- Cron job (every 15 min)
- Event handlers for create/update/delete
- **Webhooks** (6 endpoints)
- Beteiligte: create, update, delete
- Bankverbindungen: create, update, delete
### Services
Service modules providing API clients and business logic:
- `advoware_service.py` - Advoware API client (HMAC-512 auth, token caching)
- `espocrm.py` - EspoCRM API client
- `advoware.py` - Legacy Advoware service (deprecated)
- Sync utilities and mappers
## Motia III Patterns
### Step Configuration
**Old (Motia v0.17):**
```python
config = {
'type': 'api', # or 'event', 'cron'
'name': 'MyStep',
'method': 'POST',
'path': '/my-step',
'emits': ['my-event'],
'subscribes': ['other-event']
}
```
**New (Motia III):**
```python
from motia import http, queue, cron
config = {
'name': 'MyStep',
'flows': ['my-flow'],
'triggers': [
http('POST', '/my-step') # or queue('topic') or cron('0 */15 * * * *')
],
'enqueues': ['my-event']
}
```
### Handler Signatures
**HTTP Trigger:**
```python
from motia import ApiRequest, ApiResponse, FlowContext
async def handler(request: ApiRequest, ctx: FlowContext) -> ApiResponse:
# Access: request.body, request.query_params, request.path_params
# Enqueue: await ctx.enqueue(topic='...', data={...})
# Log: ctx.logger.info('...')
return ApiResponse(status=200, body={...})
```
**Queue Trigger:**
```python
async def handler(input_data: dict, ctx: FlowContext) -> None:
# Process queue data
await ctx.enqueue(topic='next-step', data={...})
```
**Cron Trigger:**
```python
async def handler(input_data: None, ctx: FlowContext) -> None:
# Cron jobs receive no input
ctx.logger.info('Cron triggered')
```
### Key Differences from Old Motia
| Old Motia v0.17 | Motia III v1.0-RC |
|-----------------|-------------------|
| `type: 'api'` | `triggers: [http()]` |
| `type: 'event'` | `triggers: [queue()]` |
| `type: 'cron'` | `triggers: [cron()]` |
| `context.emit()` | `ctx.enqueue()` |
| `emits: [...]` | `enqueues: [...]` |
| `subscribes: [...]` | Moved to trigger: `queue('topic')` |
| 5-field cron | 6-field cron (prepend seconds) |
| `context.logger` | `ctx.logger` |
| Motia Workbench | iii Console |
| Node.js + Python | Pure Python |
## Documentation Structure
```
docs/
├── INDEX.md # This file
├── ARCHITECTURE.md # System design (Motia III)
└── advoware/
└── (optional API specs)
steps/{module}/
├── README.md # Module overview
└── {step_name}_step.py # Step implementation
services/
└── {service_name}.py # Service implementations
MIGRATION_GUIDE.md # Complete migration guide
MIGRATION_STATUS.md # Migration progress
MIGRATION_COMPLETE_ANALYSIS.md # Final analysis
```
## Quick Reference
### Common Tasks
| Task | Documentation |
|------|---------------|
| Understand migration | [MIGRATION_GUIDE.md](../MIGRATION_GUIDE.md) |
| Check migration status | [MIGRATION_STATUS.md](../MIGRATION_STATUS.md) |
| Understand architecture | [ARCHITECTURE.md](ARCHITECTURE.md) |
| Calendar sync overview | [steps/advoware_cal_sync/README.md](../steps/advoware_cal_sync/README.md) |
| Proxy API usage | [steps/advoware_proxy/README.md](../steps/advoware_proxy/README.md) |
| VMH sync details | [steps/vmh/README.md](../steps/vmh/README.md) |
### Code Locations
| Component | Location | Documentation |
|-----------|----------|---------------|
| API Proxy Steps | `steps/advoware_proxy/` | [README](../steps/advoware_proxy/README.md) |
| Calendar Sync Steps | `steps/advoware_cal_sync/` | [README](../steps/advoware_cal_sync/README.md) |
| VMH Steps | `steps/vmh/` | [README](../steps/vmh/README.md) |
| Advoware Service | `services/advoware_service.py` | (in-code docs) |
| Configuration | `iii-config.yaml` | System config |
| Environment | `.env` or systemd | Environment variables |
## Running the System
### Start iii Engine
```bash
cd /opt/motia-iii/bitbylaw
/opt/bin/iii -c iii-config.yaml
```
### Start iii Console (Web UI)
```bash
/opt/bin/iii-console --enable-flow --host 0.0.0.0 --port 3113 \
--engine-host 192.168.67.233 --engine-port 3111 --ws-port 3114
```
### Access Web Console
Open browser: `http://localhost:3113/`
### Check Registered Steps
```bash
curl http://localhost:3111/_console/functions | python3 -m json.tool
```
## Testing
### Test HTTP Step
```bash
# Calendar sync API
curl -X POST "http://localhost:3111/advoware/calendar/sync" \
-H "Content-Type: application/json" \
-d '{"kuerzel": "PB"}'
# Advoware proxy
curl "http://localhost:3111/advoware/proxy?endpoint=employees"
```
### Trigger Cron Manually
```bash
curl -X POST "http://localhost:3111/_console/cron/trigger" \
-H "Content-Type: application/json" \
-d '{"function_id": "steps::Calendar Sync Cron Job::trigger::0"}'
```
### Check Logs
View logs in iii Console or via API:
```bash
curl "http://localhost:3111/_console/logs"
```
## External Resources
- [Motia III Documentation](https://iii.dev)
- [Python SDK](https://pypi.org/project/motia/)
- [Google Calendar API](https://developers.google.com/calendar)
- [Redis Documentation](https://redis.io/documentation)
## Support
- **Migration Questions**: Check [MIGRATION_GUIDE.md](../MIGRATION_GUIDE.md)
- **Runtime Issues**: Check iii Console logs
- **Step Not Showing**: Verify import errors in logs
- **Redis Issues**: Check Redis connection in `services/`

View File

@@ -0,0 +1,268 @@
# Advoware Calendar Sync - Event-Driven Design
Dieser Abschnitt implementiert die bidirektionale Synchronisation zwischen Advoware-Terminen und Google Calendar. Das System nutzt einen event-driven Ansatz mit **Motia III v1.0-RC**, der auf direkten API-Calls basiert, mit Redis für Locking und Deduplikation. Es stellt sicher, dass Termine konsistent gehalten werden, mit Fokus auf Robustheit, Fehlerbehandlung und korrekte Handhabung von mehrtägigen Terminen.
## Übersicht
Das System synchronisiert Termine zwischen:
- **Advoware**: Zentrale Terminverwaltung mit detaillierten Informationen.
- **Google Calendar**: Benutzerfreundliche Kalenderansicht für jeden Mitarbeiter.
## Architektur
### Event-Driven Design
- **Direkte API-Synchronisation**: Kein zentraler Hub; Sync läuft direkt zwischen APIs.
- **Redis Locking**: Per-Employee Locking verhindert Race-Conditions.
- **Event Emission**: Cron → All-Step → Employee-Step für skalierbare Verarbeitung.
- **Fehlerresistenz**: Einzelne Fehler stoppen nicht den gesamten Sync.
- **Logging**: Alle Logs erscheinen in der iii Console via `ctx.logger`.
### Sync-Phasen
1. **Cron-Step**: Automatische Auslösung alle 15 Minuten.
2. **All-Step**: Fetcht alle Mitarbeiter und emittiert Events pro Employee.
3. **Employee-Step**: Synchronisiert Termine für einen einzelnen Mitarbeiter.
### Datenmapping und Standardisierung
Beide Systeme werden auf gemeinsames Format normalisiert (Berlin TZ):
```python
{
'start': datetime, # Berlin TZ
'end': datetime,
'text': str,
'notiz': str,
'ort': str,
'dauertermin': int, # 0/1
'turnus': int, # 0/1
'turnusArt': int,
'recurrence': str # RRULE oder None
}
```
#### Advoware → Standard
- Start: `datum` + `uhrzeitVon` (Fallback 09:00), oder `datum` als datetime.
- End: `datumBis` + `uhrzeitBis` (Fallback 10:00), oder `datum` + 1h.
- All-Day: `dauertermin=1` oder Dauer >1 Tag.
- Recurring: `turnus`/`turnusArt` (vereinfacht, keine RRULE).
#### Google → Standard
- Start/End: `dateTime` oder `date` (All-Day).
- All-Day: `dauertermin=1` wenn All-Day oder Dauer >1 Tag.
- Recurring: RRULE aus `recurrence`.
#### Standard → Advoware
- POST/PUT: `datum`/`uhrzeitBis`/`datumBis` aus start/end.
- Defaults: `vorbereitungsDauer='00:00:00'`, `sb`/`anwalt`=employee_kuerzel.
#### Standard → Google
- All-Day: `date` statt `dateTime`, end +1 Tag.
- Recurring: RRULE aus `recurrence`.
## Funktionalität
### Automatische Kalender-Erstellung
- Für jeden Advoware-Mitarbeiter wird ein Google Calendar mit dem Namen `AW-{Kuerzel}` erstellt.
- Beispiel: Mitarbeiter mit Kürzel "SB" → Calendar "AW-SB".
- Kalender wird mit dem Haupt-Google-Account (`lehmannundpartner@gmail.com`) als Owner geteilt.
### Sync-Details
#### Cron-Step (calendar_sync_cron_step.py)
- Läuft alle 15 Minuten und emittiert "calendar_sync_all".
- **Trigger**: `cron("0 */15 * * * *")` (6-field: Sekunde Minute Stunde Tag Monat Wochentag)
#### All-Step (calendar_sync_all_step.py)
- Fetcht alle Mitarbeiter aus Advoware.
- Filtert Debug-Liste (falls konfiguriert).
- Setzt Redis-Lock pro Employee.
- Emittiert "calendar_sync" Event pro Employee.
- **Trigger**: `queue('calendar_sync_all')`
#### Employee-Step (calendar_sync_event_step.py)
- Fetcht Advoware-Termine für den Employee.
- Fetcht Google-Events für den Employee.
- Synchronisiert: Neue erstellen, Updates anwenden, Deletes handhaben.
- Verwendet Locking, um parallele Syncs zu verhindern.
- **Trigger**: `queue('calendar_sync')`
#### API-Step (calendar_sync_api_step.py)
- Manueller Trigger für einzelnen Employee oder "ALL".
- Bei "ALL": Emittiert "calendar_sync_all".
- Bei Employee: Setzt Lock und emittiert "calendar_sync".
- **Trigger**: `http('POST', '/advoware/calendar/sync')`
## API-Schwächen und Fixes
### Advoware API
- **Mehrtägige Termine**: `datumBis` wird korrekt für Enddatum verwendet; '00:00:00' als '23:59:59' interpretiert.
- **Zeitformate**: Robuste Parsing mit Fallbacks.
- **Keine 24h-Limit**: Termine können länger als 24h sein; Google Calendar unterstützt das.
### Google Calendar API
- **Zeitbereiche**: Akzeptiert Events >24h ohne Probleme.
- **Rate Limits**: Backoff-Retry implementiert.
## Step-Konfiguration (Motia III)
### calendar_sync_cron_step.py
```python
config = {
'name': 'Calendar Sync Cron Job',
'flows': ['advoware'],
'triggers': [
cron("0 */15 * * * *") # Alle 15 Minuten (6-field format)
],
'enqueues': ['calendar_sync_all']
}
```
### calendar_sync_all_step.py
```python
config = {
'name': 'Calendar Sync All Step',
'flows': ['advoware'],
'triggers': [
queue('calendar_sync_all')
],
'enqueues': ['calendar_sync']
}
```
### calendar_sync_event_step.py
```python
config = {
'name': 'Calendar Sync Event Step',
'flows': ['advoware'],
'triggers': [
queue('calendar_sync')
],
'enqueues': []
}
```
### calendar_sync_api_step.py
```python
config = {
'name': 'Calendar Sync API Trigger',
'flows': ['advoware'],
'triggers': [
http('POST', '/advoware/calendar/sync')
],
'enqueues': ['calendar_sync', 'calendar_sync_all']
}
```
## Setup
### Umgebungsvariablen
```env
# Google Calendar
GOOGLE_CALENDAR_SERVICE_ACCOUNT_PATH=service-account.json
# Advoware API
ADVOWARE_API_BASE_URL=https://www2.advo-net.net:90/
ADVOWARE_PRODUCT_ID=64
ADVOWARE_APP_ID=your_app_id
ADVOWARE_API_KEY=your_api_key
ADVOWARE_KANZLEI=your_kanzlei
ADVOWARE_DATABASE=your_database
ADVOWARE_USER=your_user
ADVOWARE_ROLE=2
ADVOWARE_PASSWORD=your_password
ADVOWARE_TOKEN_LIFETIME_MINUTES=55
ADVOWARE_API_TIMEOUT_SECONDS=30
# Redis
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_DB_CALENDAR_SYNC=1
REDIS_TIMEOUT_SECONDS=5
# Debug
CALENDAR_SYNC_DEBUG_EMPLOYEES=PB,AI # Optional, filter employees
```
## Verwendung
### Manueller Sync
```bash
# Sync für einen bestimmten Mitarbeiter
curl -X POST "http://localhost:3111/advoware/calendar/sync" \
-H "Content-Type: application/json" \
-d '{"kuerzel": "PB"}'
# Sync für alle Mitarbeiter
curl -X POST "http://localhost:3111/advoware/calendar/sync" \
-H "Content-Type: application/json" \
-d '{"kuerzel": "ALL"}'
```
### Automatischer Sync
Cron-Step läuft automatisch alle 15 Minuten.
## Fehlerbehandlung und Logging
- **Locking**: Redis NX/EX verhindert parallele Syncs.
- **Logging**: `ctx.logger` für iii Console-Sichtbarkeit.
- **API-Fehler**: Retry mit Backoff.
- **Parsing-Fehler**: Robuste Fallbacks.
## Sicherheit
- Service Account für Google Calendar API.
- HMAC-512 Authentifizierung für Advoware API.
- Redis für Concurrency-Control.
## Bekannte Probleme
- **Recurring-Events**: Begrenzte Unterstützung für komplexe Wiederholungen.
- **Performance**: Bei vielen Terminen Paginierung beachten.
- **Timezone-Handling**: Alle Operationen in Europe/Berlin TZ.
## Datenfluss
```
Cron (alle 15min)
→ calendar_sync_cron_step
→ ctx.enqueue(topic: "calendar_sync_all")
→ calendar_sync_all_step
→ Fetch Employees from Advoware
→ For each Employee:
→ Set Redis Lock (key: calendar_sync:employee:{kuerzel})
→ ctx.enqueue(topic: "calendar_sync", data: {kuerzel, ...})
→ calendar_sync_event_step
→ Fetch Advoware Termine (frNr, datum, text, etc.)
→ Fetch Google Calendar Events
→ 4-Phase Sync:
1. New from Advoware → Google
2. New from Google → Advoware
3. Process Deletes
4. Process Updates
→ Clear Redis Lock
```
## Weitere Dokumentation
- **Individual Step Docs**: Siehe `docs/` Ordner in diesem Verzeichnis
- **Architecture Overview**: [../../docs/ARCHITECTURE.md](../../docs/ARCHITECTURE.md)
- **Google Setup Guide**: [../../docs/GOOGLE_SETUP.md](../../docs/GOOGLE_SETUP.md)
- **Troubleshooting**: [../../docs/TROUBLESHOOTING.md](../../docs/TROUBLESHOOTING.md)
## Migration Notes
Dieses System wurde von **Motia v0.17** nach **Motia III v1.0-RC** migriert:
### Wichtige Änderungen:
-`type: 'event'``triggers: [queue('topic')]`
-`type: 'cron'``triggers: [cron('expression')]` (6-field format)
-`type: 'api'``triggers: [http('METHOD', 'path')]`
-`context.emit()``ctx.enqueue()`
-`emits: [...]``enqueues: [...]`
- ✅ Relative Imports → Absolute Imports mit `sys.path.insert()`
- ✅ Motia Workbench → iii Console
### Kompatibilität:
- ✅ Alle 4 Steps vollständig migriert
- ✅ Google Calendar API Integration unverändert
- ✅ Advoware API Integration unverändert
- ✅ Redis Locking-Mechanismus unverändert
- ✅ Datenbank-Schema kompatibel

View File

@@ -0,0 +1,314 @@
# Advoware API Proxy Steps
Dieser Ordner enthält die API-Proxy-Steps für die Advoware-Integration. Jeder Step implementiert eine HTTP-Methode als universellen Proxy zur Advoware-API mit **Motia III v1.0-RC**.
## Übersicht
Die Proxy-Steps fungieren als transparente Schnittstelle zwischen Clients und der Advoware-API. Sie handhaben Authentifizierung, Fehlerbehandlung und Logging automatisch.
## Steps
### 1. GET Proxy (`advoware_api_proxy_get_step.py`)
**Zweck:** Universeller Proxy für GET-Requests an die Advoware-API.
**Konfiguration:**
```python
config = {
'name': 'Advoware Proxy GET',
'flows': ['advoware'],
'triggers': [
http('GET', '/advoware/proxy')
],
'enqueues': []
}
```
**Funktionalität:**
- Extrahiert den Ziel-Endpoint aus Query-Parametern (`endpoint`)
- Übergibt alle anderen Query-Parameter als API-Parameter
- Gibt das Ergebnis als JSON zurück
**Beispiel Request:**
```bash
GET /advoware/proxy?endpoint=employees&limit=10&offset=0
```
**Response:**
```json
{
"result": {
"data": [...],
"total": 100
}
}
```
### 2. POST Proxy (`advoware_api_proxy_post_step.py`)
**Zweck:** Universeller Proxy für POST-Requests an die Advoware-API.
**Konfiguration:**
```python
config = {
'name': 'Advoware Proxy POST',
'flows': ['advoware'],
'triggers': [
http('POST', '/advoware/proxy')
],
'enqueues': []
}
```
**Funktionalität:**
- Extrahiert den Ziel-Endpoint aus Query-Parametern (`endpoint`)
- Verwendet den Request-Body als JSON-Daten für die API
- Erstellt neue Ressourcen in Advoware
**Beispiel Request:**
```bash
POST /advoware/proxy?endpoint=employees
Content-Type: application/json
{
"name": "John Doe",
"email": "john@example.com"
}
```
### 3. PUT Proxy (`advoware_api_proxy_put_step.py`)
**Zweck:** Universeller Proxy für PUT-Requests an die Advoware-API.
**Konfiguration:**
```python
config = {
'name': 'Advoware Proxy PUT',
'flows': ['advoware'],
'triggers': [
http('PUT', '/advoware/proxy')
],
'enqueues': []
}
```
**Funktionalität:**
- Extrahiert den Ziel-Endpoint aus Query-Parametern (`endpoint`)
- Verwendet den Request-Body als JSON-Daten für Updates
- Aktualisiert bestehende Ressourcen in Advoware
**Beispiel Request:**
```bash
PUT /advoware/proxy?endpoint=employees/123
Content-Type: application/json
{
"name": "John Smith",
"email": "johnsmith@example.com"
}
```
### 4. DELETE Proxy (`advoware_api_proxy_delete_step.py`)
**Zweck:** Universeller Proxy für DELETE-Requests an die Advoware-API.
**Konfiguration:**
```python
config = {
'name': 'Advoware Proxy DELETE',
'flows': ['advoware'],
'triggers': [
http('DELETE', '/advoware/proxy')
],
'enqueues': []
}
```
**Funktionalität:**
- Extrahiert den Ziel-Endpoint aus Query-Parametern (`endpoint`)
- Löscht Ressourcen in Advoware
**Beispiel Request:**
```bash
DELETE /advoware/proxy?endpoint=employees/123
```
## Gemeinsame Features
### Authentifizierung
Alle Steps verwenden den `AdvowareService` für automatische Token-Verwaltung und Authentifizierung:
- HMAC-512 basierte Signatur
- Token-Caching in Redis (55 Minuten Lifetime)
- Automatischer Token-Refresh bei 401-Errors
### Fehlerbehandling
- **400 Bad Request:** Fehlender `endpoint` Parameter
- **500 Internal Server Error:** API-Fehler oder Exceptions
- **401 Unauthorized:** Automatischer Token-Refresh und Retry
### Logging
Detaillierte Logs via `ctx.logger` für:
- Eingehende Requests
- API-Calls an Advoware
- Fehler und Exceptions
- Token-Management
Alle Logs sind in der **iii Console** sichtbar.
### Sicherheit
- Keine direkte Weitergabe sensibler Daten
- Authentifizierung über Service-Layer
- Input-Validation für erforderliche Parameter
- HMAC-512 Signatur für alle API-Requests
## Handler-Struktur (Motia III)
Alle Steps folgen dem gleichen Pattern:
```python
from motia import http, ApiRequest, ApiResponse, FlowContext
from services.advoware_service import AdvowareService
config = {
'name': 'Advoware Proxy {METHOD}',
'flows': ['advoware'],
'triggers': [
http('{METHOD}', '/advoware/proxy')
],
'enqueues': []
}
async def handler(request: ApiRequest, ctx: FlowContext) -> ApiResponse:
# Extract endpoint from query params
endpoint = request.query_params.get('endpoint')
if not endpoint:
return ApiResponse(
status=400,
body={'error': 'Missing required query parameter: endpoint'}
)
# Call Advoware API
advoware_service = AdvowareService()
result = await advoware_service.{method}(endpoint, **params)
return ApiResponse(status=200, body={'result': result})
```
## Testing
### Unit Tests
```bash
# Test GET Proxy
curl -X GET "http://localhost:3111/advoware/proxy?endpoint=employees"
# Test POST Proxy
curl -X POST "http://localhost:3111/advoware/proxy?endpoint=employees" \
-H "Content-Type: application/json" \
-d '{"name": "Test Employee"}'
# Test PUT Proxy
curl -X PUT "http://localhost:3111/advoware/proxy?endpoint=employees/1" \
-H "Content-Type: application/json" \
-d '{"name": "Updated Employee"}'
# Test DELETE Proxy
curl -X DELETE "http://localhost:3111/advoware/proxy?endpoint=employees/1"
```
### Integration Tests
Überprüfen Sie die Logs in der iii Console:
```bash
# Check logs
curl http://localhost:3111/_console/logs
```
## Konfiguration
### Umgebungsvariablen
Stellen Sie sicher, dass folgende Variablen gesetzt sind:
```env
ADVOWARE_API_BASE_URL=https://www2.advo-net.net:90/
ADVOWARE_PRODUCT_ID=64
ADVOWARE_APP_ID=your_app_id
ADVOWARE_API_KEY=your_api_key
ADVOWARE_KANZLEI=your_kanzlei
ADVOWARE_DATABASE=your_database
ADVOWARE_USER=your_user
ADVOWARE_ROLE=2
ADVOWARE_PASSWORD=your_password
ADVOWARE_TOKEN_LIFETIME_MINUTES=55
ADVOWARE_API_TIMEOUT_SECONDS=30
# Redis (für Token-Caching)
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_DB=0
```
### Dependencies
- `services/advoware_service.py` - Advoware API Client mit HMAC Auth
- `config.py` - Konfigurationsmanagement
- `motia` - Motia III Python SDK
- `asyncpg` - PostgreSQL Client
- `redis` - Redis Client für Token-Caching
## Erweiterungen
### Geplante Features
- Request/Response Caching für häufige Queries
- Rate Limiting pro Client
- Request Validation Schemas mit Pydantic
- Batch-Operations Support
### Custom Endpoints
Für spezifische Endpoints können zusätzliche Steps erstellt werden, die direkt auf bestimmte Ressourcen zugreifen und erweiterte Validierung/Transformation bieten.
## Architektur
```
Client Request
HTTP Trigger (http('METHOD', '/advoware/proxy'))
Handler (ApiRequest → ApiResponse)
├─► Extract 'endpoint' from query params
├─► Extract other params/body
AdvowareService
├─► Check Redis for valid token
├─► If expired: Get new token (HMAC-512 auth)
├─► Build HTTP request
Advoware API
Response → Transform → Return ApiResponse
```
## Migration Notes
Dieses System wurde von **Motia v0.17** nach **Motia III v1.0-RC** migriert:
### Wichtige Änderungen:
-`type: 'api'``triggers: [http('METHOD', 'path')]`
-`ApiRouteConfig``StepConfig` mit `as const satisfies`
-`Handlers['StepName']``Handlers<typeof config>`
-`context``ctx`
-`req` dict → `ApiRequest` typed object
- ✅ Return dict → `ApiResponse` typed object
-`method`, `path` moved into trigger
- ✅ Motia Workbench → iii Console
### Kompatibilität:
- ✅ Alle 4 Proxy Steps vollständig migriert
- ✅ AdvowareService kompatibel (keine Änderungen)
- ✅ Redis Token-Caching unverändert
- ✅ HMAC-512 Auth unverändert
- ✅ API-Endpoints identisch