feat: Add Kommunikation-Sync documentation for bidirectional synchronization between EspoCRM and Advoware
This commit is contained in:
@@ -110,13 +110,15 @@ Acquire Lock (Redis)
|
||||
↓
|
||||
GET /api/v1/advonet/Beteiligte/{betnr}
|
||||
↓
|
||||
Timestamp-Vergleich:
|
||||
- espocrm_newer → Update Advoware (PUT)
|
||||
- advoware_newer → Update EspoCRM (PATCH)
|
||||
- conflict → EspoCRM wins (PUT) + Notification
|
||||
- no_change → Skip
|
||||
Timestamp-Vergleich (rowId + modifiedAt vs geaendertAm):
|
||||
- no_change → Nur Kommunikation sync (direction=both)
|
||||
- espocrm_newer → Update Advoware (PUT) + Kommunikation sync (direction=both)
|
||||
- advoware_newer → Update EspoCRM (PATCH) + Kommunikation sync (direction=both)
|
||||
- conflict → EspoCRM wins (PUT) + Notification + Kommunikation sync (direction=to_advoware ONLY!)
|
||||
↓
|
||||
Release Lock
|
||||
Kommunikation Sync (Hash-basiert, siehe unten)
|
||||
↓
|
||||
Release Lock (NACH Kommunikation-Sync!)
|
||||
```
|
||||
|
||||
### 3. Cron Check
|
||||
@@ -180,11 +182,63 @@ results = await asyncio.gather(*tasks, return_exceptions=True)
|
||||
|
||||
## Kommunikation-Sync Integration
|
||||
|
||||
### Base64-Marker Strategie ✅
|
||||
**WICHTIG**: Kommunikation-Sync läuft **IMMER** nach Stammdaten-Sync (auch bei `no_change`)!
|
||||
|
||||
Die Kommunikation-Synchronisation (Telefon, Email) ist in den Beteiligte-Sync integriert.
|
||||
### Hash-basierte Änderungserkennung ✅
|
||||
|
||||
**Marker-Format**:
|
||||
Die Kommunikation-Synchronisation verwendet **MD5-Hash** der `kommunikation` rowIds aus Advoware:
|
||||
- **Hash-Berechnung**: MD5 von sortierten rowIds (erste 16 Zeichen)
|
||||
- **Speicherort**: `kommunikationHash` in EspoCRM CBeteiligte
|
||||
- **Vorteil**: Erkennt Kommunikations-Änderungen ohne Beteiligte-rowId-Änderung
|
||||
|
||||
**Problem gelöst**: Beteiligte-rowId ändert sich **NICHT**, wenn nur Kommunikation geändert wird!
|
||||
|
||||
### 3-Way Diffing mit Konflikt-Erkennung
|
||||
|
||||
```python
|
||||
# Timestamp-basiert für EspoCRM
|
||||
espo_changed = espo_bet.modifiedAt > espo_bet.advowareLastSync
|
||||
|
||||
# Hash-basiert für Advoware
|
||||
stored_hash = espo_bet.kommunikationHash # z.B. "a3f5d2e8b1c4f6a9"
|
||||
current_hash = MD5(sorted(komm.rowId for komm in advo_kommunikationen))[:16]
|
||||
advo_changed = stored_hash != current_hash
|
||||
|
||||
# Konflikt-Erkennung
|
||||
if espo_changed AND advo_changed:
|
||||
espo_wins = True # EspoCRM gewinnt immer!
|
||||
```
|
||||
|
||||
### Konflikt-Behandlung: EspoCRM Wins
|
||||
|
||||
**Bei Konflikt** (beide Seiten geändert):
|
||||
1. **Stammdaten**: EspoCRM → Advoware (PUT)
|
||||
2. **Kommunikation**: `direction='to_advoware'` (NUR EspoCRM→Advoware, blockiert Advoware→EspoCRM)
|
||||
3. **Notification**: In-App Benachrichtigung
|
||||
4. **Hash-Update**: Neuer Hash wird gespeichert
|
||||
|
||||
**Ohne Konflikt**:
|
||||
- **Stammdaten**: Je nach Timestamp-Vergleich
|
||||
- **Kommunikation**: `direction='both'` (bidirektional)
|
||||
|
||||
### 6 Sync-Varianten (Var1-6)
|
||||
|
||||
**Var1**: Neu in EspoCRM → CREATE in Advoware
|
||||
**Var2**: Gelöscht in EspoCRM → DELETE in Advoware (Empty Slot)
|
||||
**Var3**: Gelöscht in Advoware → DELETE in EspoCRM
|
||||
**Var4**: Neu in Advoware → CREATE in EspoCRM
|
||||
**Var5**: Geändert in EspoCRM → UPDATE in Advoware
|
||||
**Var6**: Geändert in Advoware → UPDATE in EspoCRM
|
||||
|
||||
### Base64-Marker Strategie
|
||||
```
|
||||
[ESPOCRM:bWF4QGV4YW1wbGUuY29t:4] Geschäftlich
|
||||
[ESPOCRM-SLOT:4] # Leerer Slot nach Löschung
|
||||
```
|
||||
|
||||
### Base64-Marker Strategie
|
||||
|
||||
**Marker-Format** im Advoware `bemerkung` Feld:
|
||||
```
|
||||
[ESPOCRM:bWF4QGV4YW1wbGUuY29t:4] Geschäftlich
|
||||
[ESPOCRM-SLOT:4] # Leerer Slot nach Löschung
|
||||
@@ -201,28 +255,90 @@ Die Kommunikation-Synchronisation (Telefon, Email) ist in den Beteiligte-Sync in
|
||||
# Update: EspoCRM-Eintrag + Marker mit neuem Base64-Wert
|
||||
```
|
||||
|
||||
**Async/Await Architektur** ⚡:
|
||||
- Alle Sync-Methoden sind **async** für Non-Blocking I/O
|
||||
- AdvowareService: Native async (kein `asyncio.run()` mehr)
|
||||
- KommunikationSyncManager: Vollständig async mit proper await
|
||||
- Integration im Webhook-Handler: Seamless async/await flow
|
||||
### 4-Stufen kommKz-Erkennung (Type Detection)
|
||||
|
||||
**4-Stufen Typ-Erkennung**:
|
||||
1. **Marker** (höchste Priorität) → `[ESPOCRM:...:3]` = kommKz 3
|
||||
2. **Top-Level Felder** → `beteiligte.mobil` = kommKz 3
|
||||
3. **Wert-Pattern** → `@` in Wert = Email (kommKz 4)
|
||||
4. **Default** → Fallback (TelGesch=1, MailGesch=4)
|
||||
**Problem**: Advoware `kommKz` ist via GET immer 0, via PUT read-only!
|
||||
|
||||
**Bidirektionale Sync**:
|
||||
- **Advoware → EspoCRM**: Komplett (inkl. Marker-Update bei Wert-Änderung)
|
||||
- **EspoCRM → Advoware**: Vollständig (CREATE/UPDATE/DELETE via Slots)
|
||||
- **Slot-Wiederverwendung**: Gelöschte Einträge werden als `[ESPOCRM-SLOT:kommKz]` markiert
|
||||
**Lösung - Prioritäts-Kaskade**:
|
||||
1. **Marker** (höchste Priorität) → `[ESPOCRM:...:3]` = kommKz 3 (Mobil)
|
||||
2. **EspoCRM Type** (bei EspoCRM→Advoware) → `type: 'Mobile'` = kommKz 3
|
||||
3. **Top-Level Felder** → `beteiligte.mobil` = kommKz 3
|
||||
4. **Wert-Pattern** → `@` in Wert = Email (kommKz 4)
|
||||
5. **Default** → Fallback (TelGesch=1, MailGesch=4)
|
||||
|
||||
**Mapping EspoCRM phoneNumberData.type → kommKz**:
|
||||
```python
|
||||
PHONE_TYPE_TO_KOMMKZ = {
|
||||
'Office': 1, # TelGesch
|
||||
'Fax': 2, # FaxGesch
|
||||
'Mobile': 3, # Mobil
|
||||
'Home': 6, # TelPrivat
|
||||
'Other': 10 # Sonstige
|
||||
}
|
||||
```
|
||||
|
||||
### Slot-Wiederverwendung (Empty Slots)
|
||||
|
||||
**Problem**: Advoware DELETE gibt 403 Forbidden!
|
||||
|
||||
**Lösung**: Empty Slots mit Marker
|
||||
```python
|
||||
# Gelöscht in EspoCRM → Create Empty Slot in Advoware
|
||||
{
|
||||
"tlf": "",
|
||||
"bemerkung": "[ESPOCRM-SLOT:4]", # kommKz=4 (Email)
|
||||
"kommKz": 4,
|
||||
"online": True
|
||||
}
|
||||
```
|
||||
|
||||
**Wiederverwendung**:
|
||||
- Neue Einträge prüfen zuerst Empty Slots mit passendem kommKz
|
||||
- UPDATE statt CREATE spart API-Calls und IDs
|
||||
|
||||
### Lock-Management mit Redis
|
||||
|
||||
**WICHTIG**: Lock wird erst NACH Kommunikation-Sync freigegeben!
|
||||
|
||||
```python
|
||||
# Pattern in allen 4 Szenarien:
|
||||
await sync_utils.acquire_sync_lock(entity_id)
|
||||
try:
|
||||
# 1. Stammdaten sync
|
||||
# 2. Kommunikation sync (run_kommunikation_sync helper)
|
||||
# 3. Lock release
|
||||
await sync_utils.release_sync_lock(entity_id, 'clean')
|
||||
finally:
|
||||
# Failsafe: Lock wird auch bei Exception released
|
||||
pass
|
||||
```
|
||||
|
||||
**Vorher (BUG)**: Lock wurde teilweise VOR Kommunikation-Sync released!
|
||||
**Jetzt**: Konsistentes Pattern - Lock schützt gesamte Operation
|
||||
|
||||
### Implementation Details
|
||||
|
||||
**Implementation**:
|
||||
- [kommunikation_mapper.py](../services/kommunikation_mapper.py) - Base64 encoding/decoding
|
||||
- [kommunikation_sync_utils.py](../services/kommunikation_sync_utils.py) - Sync-Manager
|
||||
- [kommunikation_mapper.py](../services/kommunikation_mapper.py) - Base64 encoding/decoding, kommKz detection
|
||||
- [kommunikation_sync_utils.py](../services/kommunikation_sync_utils.py) - Sync-Manager mit 3-way diffing
|
||||
- [beteiligte_sync_event_step.py](../steps/vmh/beteiligte_sync_event_step.py) - Event handler mit helper function
|
||||
- Tests: [test_kommunikation_sync_implementation.py](../scripts/test_kommunikation_sync_implementation.py)
|
||||
|
||||
**Helper Function** (DRY-Prinzip):
|
||||
```python
|
||||
async def run_kommunikation_sync(entity_id, betnr, komm_sync, context, direction='both'):
|
||||
"""Führt Kommunikation-Sync aus mit Error-Handling und Logging"""
|
||||
context.logger.info(f"📞 Starte Kommunikation-Sync (direction={direction})...")
|
||||
komm_result = await komm_sync.sync_bidirectional(entity_id, betnr, direction=direction)
|
||||
return komm_result
|
||||
```
|
||||
|
||||
**Verwendet in**:
|
||||
- no_change: `direction='both'`
|
||||
- espocrm_newer: `direction='both'`
|
||||
- advoware_newer: `direction='both'`
|
||||
- **conflict**: `direction='to_advoware'` ← NUR EspoCRM→Advoware!
|
||||
|
||||
## Performance
|
||||
|
||||
| Operation | API Calls | Latency |
|
||||
@@ -357,6 +473,8 @@ Custom fields für Sync-Management:
|
||||
- `syncStatus` (enum) - Sync-Status
|
||||
- `advowareLastSync` (datetime) - Letzter erfolgreicher Sync
|
||||
- `advowareDeletedAt` (datetime) - Soft-Delete timestamp
|
||||
- `advowareRowId` (varchar, 50) - Cached Advoware rowId für Change Detection
|
||||
- **`kommunikationHash` (varchar, 16)** - MD5-Hash der Kommunikation rowIds (erste 16 Zeichen)
|
||||
- `syncErrorMessage` (text, 2000 chars) - Letzte Fehlermeldung
|
||||
- `syncRetryCount` (int) - Anzahl fehlgeschlagener Versuche
|
||||
|
||||
|
||||
711
bitbylaw/docs/KOMMUNIKATION_SYNC.md
Normal file
711
bitbylaw/docs/KOMMUNIKATION_SYNC.md
Normal file
@@ -0,0 +1,711 @@
|
||||
# Kommunikation-Sync - Bidirektionale Synchronisation EspoCRM ↔ Advoware
|
||||
|
||||
**Erstellt**: 8. Februar 2026
|
||||
**Status**: ✅ Implementiert und getestet
|
||||
|
||||
---
|
||||
|
||||
## Übersicht
|
||||
|
||||
Bidirektionale Synchronisation der **Kommunikationsdaten** (Telefon, Email, Fax) zwischen EspoCRM (CBeteiligte) und Advoware (Kommunikationen).
|
||||
|
||||
**Scope**: Telefonnummern, Email-Adressen, Fax-Nummern
|
||||
**Trigger**: Automatisch nach jedem Beteiligte-Stammdaten-Sync
|
||||
**Change Detection**: Hash-basiert (MD5 von kommunikation rowIds)
|
||||
|
||||
---
|
||||
|
||||
## Architektur
|
||||
|
||||
### Integration in Beteiligte-Sync
|
||||
|
||||
```
|
||||
┌─────────────────┐
|
||||
│ Beteiligte Sync │ (Stammdaten)
|
||||
│ Event Handler │
|
||||
└────────┬────────┘
|
||||
│ ✅ Stammdaten synced
|
||||
↓
|
||||
┌─────────────────────────────┐
|
||||
│ Kommunikation Sync Manager │
|
||||
│ sync_bidirectional() │
|
||||
│ │
|
||||
│ 1. Load Data (1x) │
|
||||
│ 2. Compute Diff (3-Way) │
|
||||
│ 3. Apply Changes │
|
||||
│ 4. Update Hash │
|
||||
└─────────────────────────────┘
|
||||
│
|
||||
↓
|
||||
┌─────────────────┐
|
||||
│ Lock Release │
|
||||
└─────────────────┘
|
||||
```
|
||||
|
||||
**WICHTIG**: Lock wird erst NACH Kommunikation-Sync freigegeben!
|
||||
|
||||
### Komponenten
|
||||
|
||||
1. **KommunikationSyncManager** ([kommunikation_sync_utils.py](../services/kommunikation_sync_utils.py))
|
||||
- Bidirektionale Sync-Logik
|
||||
- 3-Way Diffing
|
||||
- Hash-basierte Änderungserkennung
|
||||
- Konflikt-Behandlung
|
||||
|
||||
2. **KommunikationMapper** ([kommunikation_mapper.py](../services/kommunikation_mapper.py))
|
||||
- Base64-Marker Encoding/Decoding
|
||||
- kommKz Detection (4-Stufen)
|
||||
- Type Mapping (EspoCRM ↔ Advoware)
|
||||
|
||||
3. **Helper Function** ([beteiligte_sync_event_step.py](../steps/vmh/beteiligte_sync_event_step.py))
|
||||
- `run_kommunikation_sync()` mit Error Handling
|
||||
- Direction-Parameter für Konflikt-Handling
|
||||
|
||||
---
|
||||
|
||||
## Change Detection: Hash-basiert
|
||||
|
||||
### Problem
|
||||
|
||||
Advoware Beteiligte-rowId ändert sich **NICHT**, wenn nur Kommunikation geändert wird!
|
||||
|
||||
**Beispiel**:
|
||||
```
|
||||
Beteiligte: rowId = "ABCD1234..."
|
||||
Kommunikation 1: "max@example.com"
|
||||
|
||||
→ Email zu "new@example.com" ändern
|
||||
|
||||
Beteiligte: rowId = "ABCD1234..." ← UNCHANGED!
|
||||
Kommunikation 1: "new@example.com"
|
||||
```
|
||||
|
||||
### Lösung: MD5-Hash der Kommunikation-rowIds
|
||||
|
||||
```python
|
||||
# Hash-Berechnung
|
||||
komm_rowids = sorted([k['rowId'] for k in kommunikationen])
|
||||
komm_hash = hashlib.md5(''.join(komm_rowids).encode()).hexdigest()[:16]
|
||||
|
||||
# Beispiel:
|
||||
komm_rowids = [
|
||||
"FBABAAAAEMDGACAAFLBGNLBAAAAAEDNHOEAFAAAA",
|
||||
"GBABAAAAEMDGACAAFLBGNLBAAAAAEDNHOEAFAAAA"
|
||||
]
|
||||
→ Hash: "a3f5d2e8b1c4f6a9"
|
||||
```
|
||||
|
||||
**Speicherort**: `kommunikationHash` in EspoCRM CBeteiligte (varchar, 16)
|
||||
|
||||
**Vergleich**:
|
||||
```python
|
||||
stored_hash = espo_bet.get('kommunikationHash')
|
||||
current_hash = calculate_hash(advo_kommunikationen)
|
||||
|
||||
if stored_hash != current_hash:
|
||||
# Kommunikation hat sich geändert!
|
||||
advo_changed = True
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3-Way Diffing
|
||||
|
||||
### Konflikt-Erkennung
|
||||
|
||||
```python
|
||||
# EspoCRM: Timestamp-basiert
|
||||
espo_modified = espo_bet.get('modifiedAt')
|
||||
last_sync = espo_bet.get('advowareLastSync')
|
||||
espo_changed = espo_modified > last_sync
|
||||
|
||||
# Advoware: Hash-basiert
|
||||
stored_hash = espo_bet.get('kommunikationHash')
|
||||
current_hash = calculate_hash(advo_kommunikationen)
|
||||
advo_changed = stored_hash != current_hash
|
||||
|
||||
# Konflikt?
|
||||
if espo_changed AND advo_changed:
|
||||
espo_wins = True # EspoCRM gewinnt IMMER!
|
||||
```
|
||||
|
||||
### Direction-Parameter
|
||||
|
||||
```python
|
||||
async def sync_bidirectional(entity_id, betnr, direction='both'):
|
||||
"""
|
||||
direction:
|
||||
- 'both': Bidirektional (normal)
|
||||
- 'to_espocrm': Nur Advoware→EspoCRM
|
||||
- 'to_advoware': Nur EspoCRM→Advoware (bei Konflikt!)
|
||||
"""
|
||||
```
|
||||
|
||||
**Bei Konflikt**:
|
||||
```python
|
||||
# Beteiligte Sync Event Handler
|
||||
if comparison == 'conflict':
|
||||
# Stammdaten: EspoCRM → Advoware
|
||||
await advoware.put_beteiligte(...)
|
||||
|
||||
# Kommunikation: NUR EspoCRM → Advoware
|
||||
await run_kommunikation_sync(
|
||||
entity_id, betnr, komm_sync, context,
|
||||
direction='to_advoware' # ← Blockiert Advoware→EspoCRM!
|
||||
)
|
||||
```
|
||||
|
||||
**Ohne Konflikt**:
|
||||
```python
|
||||
# Normal: Bidirektional
|
||||
await run_kommunikation_sync(
|
||||
entity_id, betnr, komm_sync, context,
|
||||
direction='both' # ← Default
|
||||
)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6 Sync-Varianten (Var1-6)
|
||||
|
||||
### Var1: Neu in EspoCRM → CREATE in Advoware
|
||||
|
||||
**Trigger**: EspoCRM Entry ohne Marker-Match in Advoware
|
||||
|
||||
```python
|
||||
# EspoCRM
|
||||
phoneNumberData: [{
|
||||
phoneNumber: "+49 511 123456",
|
||||
type: "Mobile",
|
||||
primary: true
|
||||
}]
|
||||
|
||||
# → Advoware
|
||||
POST /Beteiligte/{betnr}/Kommunikationen
|
||||
{
|
||||
"tlf": "+49 511 123456",
|
||||
"kommKz": 3, # Mobile
|
||||
"bemerkung": "[ESPOCRM:KzQ5IDUxMSAxMjM0NTY=:3] ",
|
||||
"online": false
|
||||
}
|
||||
```
|
||||
|
||||
**Empty Slot Reuse**: Prüft zuerst leere Slots mit passendem kommKz!
|
||||
|
||||
### Var2: Gelöscht in EspoCRM → Empty Slot in Advoware
|
||||
|
||||
**Problem**: Advoware DELETE gibt 403 Forbidden!
|
||||
|
||||
**Lösung**: Update zu Empty Slot
|
||||
```python
|
||||
# Advoware
|
||||
PUT /Beteiligte/{betnr}/Kommunikationen/{id}
|
||||
{
|
||||
"tlf": "",
|
||||
"bemerkung": "[ESPOCRM-SLOT:3]", # kommKz=3 gespeichert
|
||||
"online": false
|
||||
}
|
||||
```
|
||||
|
||||
**Wiederverwendung**: Var1 prüft Empty Slots vor neuem CREATE
|
||||
|
||||
### Var3: Gelöscht in Advoware → DELETE in EspoCRM
|
||||
|
||||
**Trigger**: Marker in Advoware vorhanden, aber keine Sync-relevante Kommunikation
|
||||
|
||||
```python
|
||||
# Marker vorhanden: [ESPOCRM:...:4]
|
||||
# Aber: tlf="" oder should_sync_to_espocrm() = False
|
||||
|
||||
# → EspoCRM
|
||||
# Entferne aus emailAddressData[] oder phoneNumberData[]
|
||||
```
|
||||
|
||||
### Var4: Neu in Advoware → CREATE in EspoCRM
|
||||
|
||||
**Trigger**: Advoware Entry ohne [ESPOCRM:...] Marker
|
||||
|
||||
```python
|
||||
# Advoware
|
||||
{
|
||||
"tlf": "info@firma.de",
|
||||
"kommKz": 4, # MailGesch
|
||||
"bemerkung": "Allgemeine Anfragen"
|
||||
}
|
||||
|
||||
# → EspoCRM
|
||||
emailAddressData: [{
|
||||
emailAddress: "info@firma.de",
|
||||
primary: false,
|
||||
optOut: false
|
||||
}]
|
||||
|
||||
# → Advoware Marker Update
|
||||
"bemerkung": "[ESPOCRM:aW5mb0BmaXJtYS5kZQ==:4] Allgemeine Anfragen"
|
||||
```
|
||||
|
||||
### Var5: Geändert in EspoCRM → UPDATE in Advoware
|
||||
|
||||
**Trigger**: Marker-dekodierter Wert ≠ EspoCRM Wert, aber Marker vorhanden
|
||||
|
||||
```python
|
||||
# Marker: [ESPOCRM:b2xkQGV4YW1wbGUuY29t:4]
|
||||
# Dekodiert: "old@example.com"
|
||||
# EspoCRM: "new@example.com"
|
||||
|
||||
# → Advoware
|
||||
PUT /Beteiligte/{betnr}/Kommunikationen/{id}
|
||||
{
|
||||
"tlf": "new@example.com",
|
||||
"bemerkung": "[ESPOCRM:bmV3QGV4YW1wbGUuY29t:4] ",
|
||||
"online": true
|
||||
}
|
||||
```
|
||||
|
||||
**Primary-Änderungen**: Auch `online` Flag wird aktualisiert
|
||||
|
||||
### Var6: Geändert in Advoware → UPDATE in EspoCRM
|
||||
|
||||
**Trigger**: Marker vorhanden, aber Advoware tlf ≠ Marker-Wert
|
||||
|
||||
```python
|
||||
# Marker: [ESPOCRM:b2xkQGV4YW1wbGUuY29t:4]
|
||||
# Advoware: "new@example.com"
|
||||
|
||||
# → EspoCRM
|
||||
# Update emailAddressData[]
|
||||
# Update Marker mit neuem Base64
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Base64-Marker Strategie
|
||||
|
||||
### Marker-Format
|
||||
|
||||
```
|
||||
[ESPOCRM:base64_encoded_value:kommKz] user_text
|
||||
[ESPOCRM-SLOT:kommKz]
|
||||
```
|
||||
|
||||
**Beispiele**:
|
||||
```
|
||||
[ESPOCRM:bWF4QGV4YW1wbGUuY29t:4] Geschäftliche Email
|
||||
[ESPOCRM:KzQ5IDUxMSAxMjM0NTY=:3] Mobil Herr Müller
|
||||
[ESPOCRM-SLOT:1]
|
||||
```
|
||||
|
||||
### Encoding/Decoding
|
||||
|
||||
```python
|
||||
import base64
|
||||
|
||||
def encode_value(value: str) -> str:
|
||||
return base64.b64encode(value.encode()).decode()
|
||||
|
||||
def decode_value(encoded: str) -> str:
|
||||
return base64.b64decode(encoded.encode()).decode()
|
||||
```
|
||||
|
||||
### Vorteile
|
||||
|
||||
1. **Bidirektionales Matching**: Alter Wert im Marker → Findet Match auch bei Änderung
|
||||
2. **Konflikt-freies Merge**: User-Text bleibt erhalten
|
||||
3. **Type Information**: kommKz im Marker gespeichert
|
||||
|
||||
### Parsing
|
||||
|
||||
```python
|
||||
def parse_marker(bemerkung: str) -> Optional[Dict]:
|
||||
"""
|
||||
Pattern: [ESPOCRM:base64:kommKz] user_text
|
||||
"""
|
||||
import re
|
||||
pattern = r'\[ESPOCRM:([A-Za-z0-9+/=]+):(\d+)\](.*)'
|
||||
match = re.match(pattern, bemerkung)
|
||||
|
||||
if match:
|
||||
return {
|
||||
'synced_value': decode_value(match.group(1)),
|
||||
'kommKz': int(match.group(2)),
|
||||
'user_text': match.group(3).strip()
|
||||
}
|
||||
|
||||
# Empty Slot?
|
||||
slot_pattern = r'\[ESPOCRM-SLOT:(\d+)\]'
|
||||
slot_match = re.match(slot_pattern, bemerkung)
|
||||
if slot_match:
|
||||
return {
|
||||
'is_empty_slot': True,
|
||||
'kommKz': int(slot_match.group(1))
|
||||
}
|
||||
|
||||
return None
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## kommKz Detection (4-Stufen)
|
||||
|
||||
### Problem: Advoware API-Limitierungen
|
||||
|
||||
1. **GET Response**: kommKz ist IMMER 0 (Bug oder Permission)
|
||||
2. **PUT Request**: kommKz ist READ-ONLY (wird ignoriert)
|
||||
|
||||
→ **Lösung**: Multi-Level Detection mit EspoCRM als Source of Truth
|
||||
|
||||
### Prioritäts-Kaskade
|
||||
|
||||
```python
|
||||
def detect_kommkz(value, beteiligte=None, bemerkung=None, espo_type=None):
|
||||
"""
|
||||
1. Marker (höchste Priorität)
|
||||
2. EspoCRM Type (bei EspoCRM→Advoware)
|
||||
3. Top-Level Fields
|
||||
4. Value Pattern
|
||||
5. Default
|
||||
"""
|
||||
|
||||
# 1. Marker
|
||||
if bemerkung:
|
||||
marker = parse_marker(bemerkung)
|
||||
if marker and marker.get('kommKz'):
|
||||
return marker['kommKz']
|
||||
|
||||
# 2. EspoCRM Type (NEU!)
|
||||
if espo_type:
|
||||
mapping = {
|
||||
'Office': 1, # TelGesch
|
||||
'Fax': 2, # FaxGesch
|
||||
'Mobile': 3, # Mobil
|
||||
'Home': 6, # TelPrivat
|
||||
'Other': 10 # Sonstige
|
||||
}
|
||||
if espo_type in mapping:
|
||||
return mapping[espo_type]
|
||||
|
||||
# 3. Top-Level Fields
|
||||
if beteiligte:
|
||||
if value == beteiligte.get('mobil'):
|
||||
return 3 # Mobil
|
||||
if value == beteiligte.get('tel'):
|
||||
return 1 # TelGesch
|
||||
if value == beteiligte.get('fax'):
|
||||
return 2 # FaxGesch
|
||||
# ... weitere Felder
|
||||
|
||||
# 4. Value Pattern
|
||||
if '@' in value:
|
||||
return 4 # MailGesch (Email)
|
||||
|
||||
# 5. Default
|
||||
if '@' in value:
|
||||
return 4 # MailGesch
|
||||
else:
|
||||
return 1 # TelGesch
|
||||
```
|
||||
|
||||
### Type Mapping: EspoCRM ↔ Advoware
|
||||
|
||||
**EspoCRM phoneNumberData.type**:
|
||||
- `Office` → kommKz 1 (TelGesch)
|
||||
- `Fax` → kommKz 2 (FaxGesch)
|
||||
- `Mobile` → kommKz 3 (Mobil)
|
||||
- `Home` → kommKz 6 (TelPrivat)
|
||||
- `Other` → kommKz 10 (Sonstige)
|
||||
|
||||
**kommKz Enum** (vollständig):
|
||||
```python
|
||||
KOMMKZ_TEL_GESCH = 1 # Geschäftstelefon
|
||||
KOMMKZ_FAX_GESCH = 2 # Geschäftsfax
|
||||
KOMMKZ_MOBIL = 3 # Mobiltelefon
|
||||
KOMMKZ_MAIL_GESCH = 4 # Geschäfts-Email
|
||||
KOMMKZ_INTERNET = 5 # Website/URL
|
||||
KOMMKZ_TEL_PRIVAT = 6 # Privattelefon
|
||||
KOMMKZ_FAX_PRIVAT = 7 # Privatfax
|
||||
KOMMKZ_MAIL_PRIVAT = 8 # Private Email
|
||||
KOMMKZ_AUTO_TEL = 9 # Autotelefon
|
||||
KOMMKZ_SONSTIGE = 10 # Sonstige
|
||||
KOMMKZ_EPOST = 11 # E-Post (DE-Mail)
|
||||
KOMMKZ_BEA = 12 # BeA
|
||||
```
|
||||
|
||||
**Email vs Phone**:
|
||||
```python
|
||||
def is_email_type(kommkz: int) -> bool:
|
||||
return kommkz in [4, 8, 11, 12] # Emails
|
||||
|
||||
def is_phone_type(kommkz: int) -> bool:
|
||||
return kommkz in [1, 2, 3, 6, 7, 9, 10] # Phones
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Empty Slot Management
|
||||
|
||||
### Problem: DELETE gibt 403 Forbidden
|
||||
|
||||
Advoware API erlaubt kein DELETE auf Kommunikationen!
|
||||
|
||||
### Lösung: Empty Slots
|
||||
|
||||
**Create Empty Slot**:
|
||||
```python
|
||||
async def _create_empty_slot(komm_id: int, kommkz: int):
|
||||
"""Var2: Gelöscht in EspoCRM → Empty Slot in Advoware"""
|
||||
|
||||
slot_marker = f"[ESPOCRM-SLOT:{kommkz}]"
|
||||
|
||||
await advoware.update_kommunikation(betnr, komm_id, {
|
||||
'tlf': '',
|
||||
'bemerkung': slot_marker,
|
||||
'online': False if is_phone_type(kommkz) else True
|
||||
})
|
||||
```
|
||||
|
||||
**Reuse Empty Slot**:
|
||||
```python
|
||||
def find_empty_slot(advo_kommunikationen, kommkz):
|
||||
"""Findet leeren Slot mit passendem kommKz"""
|
||||
|
||||
for komm in advo_kommunikationen:
|
||||
marker = parse_marker(komm.get('bemerkung', ''))
|
||||
if marker and marker.get('is_empty_slot'):
|
||||
if marker.get('kommKz') == kommkz:
|
||||
return komm
|
||||
|
||||
return None
|
||||
```
|
||||
|
||||
**Var1 mit Slot-Reuse**:
|
||||
```python
|
||||
# Neu in EspoCRM
|
||||
empty_slot = find_empty_slot(advo_kommunikationen, kommkz)
|
||||
|
||||
if empty_slot:
|
||||
# UPDATE statt CREATE
|
||||
await advoware.update_kommunikation(betnr, empty_slot['id'], {
|
||||
'tlf': value,
|
||||
'bemerkung': create_marker(value, kommkz, ''),
|
||||
'online': online
|
||||
})
|
||||
else:
|
||||
# CREATE new
|
||||
await advoware.create_kommunikation(betnr, {...})
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Performance
|
||||
|
||||
### Single Data Load
|
||||
|
||||
```python
|
||||
# Optimiert: Lade Daten nur 1x
|
||||
advo_bet = await advoware.get_beteiligter(betnr)
|
||||
espo_bet = await espocrm.get_entity('CBeteiligte', entity_id)
|
||||
|
||||
# Enthalten bereits alle Kommunikationen:
|
||||
advo_kommunikationen = advo_bet.get('kommunikation', [])
|
||||
espo_emails = espo_bet.get('emailAddressData', [])
|
||||
espo_phones = espo_bet.get('phoneNumberData', [])
|
||||
```
|
||||
|
||||
**Vorteil**: Keine separaten API-Calls für Kommunikationen nötig
|
||||
|
||||
### Hash-Update Strategie
|
||||
|
||||
```python
|
||||
# Update Hash nur bei Änderungen
|
||||
if total_changes > 0 or is_initial_sync:
|
||||
# Re-load Advoware (rowIds könnten sich geändert haben)
|
||||
advo_result_final = await advoware.get_beteiligter(betnr)
|
||||
new_hash = calculate_hash(advo_result_final['kommunikation'])
|
||||
|
||||
await espocrm.update_entity('CBeteiligte', entity_id, {
|
||||
'kommunikationHash': new_hash
|
||||
})
|
||||
```
|
||||
|
||||
### Latency
|
||||
|
||||
| Operation | API Calls | Latency |
|
||||
|-----------|-----------|---------|
|
||||
| Bidirectional Sync | 2-4 | ~300-500ms |
|
||||
| - Load Data | 2 | ~200ms |
|
||||
| - Apply Changes | 0-N | ~50ms/change |
|
||||
| - Update Hash | 0-1 | ~100ms |
|
||||
|
||||
---
|
||||
|
||||
## Error Handling
|
||||
|
||||
### Logging mit context.logger
|
||||
|
||||
```python
|
||||
class KommunikationSyncManager:
|
||||
def __init__(self, advoware, espocrm, context=None):
|
||||
self.logger = context.logger if context else logger
|
||||
```
|
||||
|
||||
**Wichtig**: `context.logger` statt module `logger` für Workbench-sichtbare Logs!
|
||||
|
||||
### Log-Prefix Convention
|
||||
|
||||
```python
|
||||
self.logger.info(f"[KOMM] ===== DIFF RESULTS =====")
|
||||
self.logger.info(f"[KOMM] Diff: {len(diff['advo_changed'])} Advoware changed...")
|
||||
self.logger.info(f"[KOMM] ⚠️ CONFLICT: EspoCRM wins - skipping Advoware→EspoCRM sync")
|
||||
```
|
||||
|
||||
**Prefix `[KOMM]`**: Identifiziert Kommunikation-Sync Logs
|
||||
|
||||
### Varianten-Logging
|
||||
|
||||
```python
|
||||
# Var1
|
||||
self.logger.info(f"[KOMM] ➕ Var1: New in EspoCRM '{value[:30]}...', type={espo_type}")
|
||||
|
||||
# Var2
|
||||
self.logger.info(f"[KOMM] 🗑️ Var2: Deleted in EspoCRM - komm_id={komm_id}, value='{tlf[:30]}...'")
|
||||
|
||||
# Var3
|
||||
self.logger.info(f"[KOMM] Var3: Deleted in Advoware '{value}', removing from EspoCRM")
|
||||
|
||||
# Var4
|
||||
self.logger.info(f"[KOMM] ➕ Var4: New in Advoware '{tlf}', syncing to EspoCRM")
|
||||
|
||||
# Var5
|
||||
self.logger.info(f"[KOMM] ✏️ Var5: EspoCRM changed '{value[:30]}...', primary={espo_primary}")
|
||||
|
||||
# Var6
|
||||
self.logger.info(f"[KOMM] ✏️ Var6: Advoware changed '{old_value}' → '{new_value}'")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
|
||||
### Unit Tests
|
||||
|
||||
```bash
|
||||
cd /opt/motia-app/bitbylaw
|
||||
source python_modules/bin/activate
|
||||
python scripts/test_kommunikation_sync_implementation.py
|
||||
```
|
||||
|
||||
### Manual Test
|
||||
|
||||
```python
|
||||
# Test Bidirectional Sync
|
||||
from services.kommunikation_sync_utils import KommunikationSyncManager
|
||||
|
||||
komm_sync = KommunikationSyncManager(advoware, espocrm, context)
|
||||
|
||||
result = await komm_sync.sync_bidirectional(
|
||||
beteiligte_id='68e3e7eab49f09adb',
|
||||
betnr=104860,
|
||||
direction='both'
|
||||
)
|
||||
|
||||
print(f"Advoware→EspoCRM: {result['advoware_to_espocrm']}")
|
||||
print(f"EspoCRM→Advoware: {result['espocrm_to_advoware']}")
|
||||
print(f"Total Changes: {result['summary']['total_changes']}")
|
||||
```
|
||||
|
||||
### Expected Log Output
|
||||
|
||||
```
|
||||
📞 Starte Kommunikation-Sync (direction=both)...
|
||||
[KOMM] Bidirectional Sync: betnr=104860, bet_id=68e3e7eab49f09adb
|
||||
[KOMM] Geladen: 5 Advoware, 2 EspoCRM emails, 3 EspoCRM phones
|
||||
[KOMM] ===== DIFF RESULTS =====
|
||||
[KOMM] Diff: 1 Advoware changed, 0 EspoCRM changed, 0 Advoware new, 1 EspoCRM new, 0 Advoware deleted, 0 EspoCRM deleted
|
||||
[KOMM] ===== CONFLICT STATUS: espo_wins=False =====
|
||||
[KOMM] ✅ Applying Advoware→EspoCRM changes...
|
||||
[KOMM] ✏️ Var6: Advoware changed 'old@example.com' → 'new@example.com'
|
||||
[KOMM] ✅ Updated EspoCRM: 1 emails, 0 phones
|
||||
[KOMM] ➕ Var1: New in EspoCRM '+49 511 123456', type=Mobile
|
||||
[KOMM] 🔍 kommKz detected: espo_type=Mobile, kommKz=3
|
||||
[KOMM] ✅ Created new kommunikation with kommKz=3
|
||||
[KOMM] ✅ Updated kommunikationHash: a3f5d2e8b1c4f6a9
|
||||
[KOMM] ✅ Bidirectional Sync complete: 2 total changes
|
||||
✅ Kommunikation synced: {'advoware_to_espocrm': {'emails_synced': 1, 'phones_synced': 0, 'markers_updated': 1, 'errors': []}, 'espocrm_to_advoware': {'created': 1, 'updated': 0, 'deleted': 0, 'errors': []}, 'summary': {'total_changes': 2}}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Hash bleibt unverändert trotz Änderungen
|
||||
|
||||
**Problem**: `kommunikationHash` wird nicht aktualisiert
|
||||
|
||||
**Ursachen**:
|
||||
1. `total_changes = 0` (keine Änderungen erkannt)
|
||||
2. Exception beim Hash-Update
|
||||
|
||||
**Lösung**:
|
||||
```python
|
||||
# Debug-Logging aktivieren
|
||||
self.logger.info(f"[KOMM] Total changes: {total_changes}, Initial sync: {is_initial_sync}")
|
||||
```
|
||||
|
||||
### kommKz-Erkennung fehlerhaft
|
||||
|
||||
**Problem**: Falscher Typ zugewiesen (z.B. Office statt Mobile)
|
||||
|
||||
**Ursachen**:
|
||||
1. `espo_type` nicht übergeben
|
||||
2. Marker fehlt oder fehlerhaft
|
||||
3. Top-Level Field mismatch
|
||||
|
||||
**Lösung**:
|
||||
```python
|
||||
# Bei EspoCRM→Advoware: espo_type explizit übergeben
|
||||
kommkz = detect_kommkz(
|
||||
value=phone_number,
|
||||
espo_type=espo_item.get('type'), # ← WICHTIG!
|
||||
bemerkung=existing_marker
|
||||
)
|
||||
```
|
||||
|
||||
### Empty Slots nicht wiederverwendet
|
||||
|
||||
**Problem**: Neue CREATEs statt UPDATE von Empty Slots
|
||||
|
||||
**Ursache**: `find_empty_slot()` findet keinen passenden kommKz
|
||||
|
||||
**Lösung**:
|
||||
```python
|
||||
# Debug
|
||||
self.logger.info(f"[KOMM] Looking for empty slot with kommKz={kommkz}")
|
||||
empty_slot = find_empty_slot(advo_kommunikationen, kommkz)
|
||||
if empty_slot:
|
||||
self.logger.info(f"[KOMM] ♻️ Found empty slot: {empty_slot['id']}")
|
||||
```
|
||||
|
||||
### Konflikt nicht erkannt
|
||||
|
||||
**Problem**: Bei gleichzeitigen Änderungen wird kein Konflikt gemeldet
|
||||
|
||||
**Ursachen**:
|
||||
1. Hash-Vergleich fehlerhaft
|
||||
2. Timestamp-Vergleich fehlerhaft
|
||||
|
||||
**Debug**:
|
||||
```python
|
||||
self.logger.info(f"[KOMM] 🔍 Konflikt-Check:")
|
||||
self.logger.info(f"[KOMM] - EspoCRM changed: {espo_changed}")
|
||||
self.logger.info(f"[KOMM] - Advoware changed: {advo_changed}")
|
||||
self.logger.info(f"[KOMM] - stored_hash={stored_hash}, current_hash={current_hash}")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Siehe auch
|
||||
|
||||
- [BETEILIGTE_SYNC.md](BETEILIGTE_SYNC.md) - Integration in Stammdaten-Sync
|
||||
- [KOMMUNIKATION_SYNC_ANALYSE.md](KOMMUNIKATION_SYNC_ANALYSE.md) - Detaillierte API-Tests
|
||||
- [kommunikation_mapper.py](../services/kommunikation_mapper.py) - Implementation Details
|
||||
- [kommunikation_sync_utils.py](../services/kommunikation_sync_utils.py) - Sync Manager
|
||||
Reference in New Issue
Block a user