feat: Add EspoCRM and Advoware integration for Beteiligte comparison
- Implemented `compare_beteiligte.py` script for comparing Beteiligte structures between EspoCRM and Advoware. - Created `beteiligte_comparison_result.json` to store comparison results. - Developed `EspoCRMAPI` service for handling API interactions with EspoCRM. - Added comprehensive documentation for the EspoCRM API service. - Included error handling and logging for API operations. - Enhanced entity management with CRUD operations and search capabilities.
This commit is contained in:
296
bitbylaw/scripts/README.md
Normal file
296
bitbylaw/scripts/README.md
Normal file
@@ -0,0 +1,296 @@
|
||||
# Beteiligte Structure Comparison Tool
|
||||
|
||||
## Purpose
|
||||
|
||||
This helper script fetches entity data from both **EspoCRM** and **Advoware** to compare their data structures. This helps understand:
|
||||
|
||||
- What fields exist in each system
|
||||
- How field names differ
|
||||
- Potential field mappings for synchronization
|
||||
- Data type differences
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
cd /opt/motia-app
|
||||
|
||||
# Basic usage: Compare by EspoCRM ID (will auto-search in Advoware)
|
||||
python bitbylaw/scripts/compare_beteiligte.py <espocrm_entity_id>
|
||||
|
||||
# Advanced: Specify both IDs
|
||||
python bitbylaw/scripts/compare_beteiligte.py <espocrm_entity_id> <advoware_id>
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
```bash
|
||||
# Example 1: Fetch from EspoCRM and search in Advoware by name
|
||||
python bitbylaw/scripts/compare_beteiligte.py 64a3f2b8c9e1234567890abc
|
||||
|
||||
# Example 2: Fetch from both systems by ID
|
||||
python bitbylaw/scripts/compare_beteiligte.py 64a3f2b8c9e1234567890abc 12345
|
||||
|
||||
# Example 3: Using the virtual environment
|
||||
source python_modules/bin/activate
|
||||
python bitbylaw/scripts/compare_beteiligte.py 64a3f2b8c9e1234567890abc
|
||||
```
|
||||
|
||||
## Requirements
|
||||
|
||||
### Environment Variables
|
||||
|
||||
Make sure these are set in `.env` or environment:
|
||||
|
||||
```bash
|
||||
# EspoCRM
|
||||
ESPOCRM_API_BASE_URL=https://crm.bitbylaw.com/api/v1
|
||||
ESPOCRM_MARVIN_API_KEY=your_api_key_here
|
||||
|
||||
# Advoware
|
||||
ADVOWARE_API_BASE_URL=https://www2.advo-net.net:90/
|
||||
ADVOWARE_API_KEY=your_base64_encoded_key
|
||||
ADVOWARE_USER=your_user
|
||||
ADVOWARE_PASSWORD=your_password
|
||||
ADVOWARE_KANZLEI=your_kanzlei
|
||||
ADVOWARE_DATABASE=your_database
|
||||
# ... (see config.py for all required vars)
|
||||
```
|
||||
|
||||
### Dependencies
|
||||
|
||||
```bash
|
||||
pip install aiohttp redis python-dotenv requests
|
||||
```
|
||||
|
||||
## Output
|
||||
|
||||
The script produces:
|
||||
|
||||
### 1. Console Output
|
||||
|
||||
```
|
||||
================================================================================
|
||||
BETEILIGTE STRUCTURE COMPARISON TOOL
|
||||
================================================================================
|
||||
|
||||
EspoCRM Entity ID: 64a3f2b8c9e1234567890abc
|
||||
|
||||
Environment Check:
|
||||
----------------------------------------
|
||||
ESPOCRM_API_BASE_URL: https://crm.bitbylaw.com/api/v1
|
||||
ESPOCRM_API_KEY: ✓ Set
|
||||
ADVOWARE_API_BASE_URL: https://www2.advo-net.net:90/
|
||||
ADVOWARE_API_KEY: ✓ Set
|
||||
|
||||
================================================================================
|
||||
ESPOCRM - Fetching Beteiligter
|
||||
================================================================================
|
||||
|
||||
Trying entity type: Beteiligte
|
||||
|
||||
✓ Success! Found in Beteiligte
|
||||
|
||||
Entity Structure:
|
||||
--------------------------------------------------------------------------------
|
||||
{
|
||||
"id": "64a3f2b8c9e1234567890abc",
|
||||
"name": "Max Mustermann",
|
||||
"firstName": "Max",
|
||||
"lastName": "Mustermann",
|
||||
"email": "max@example.com",
|
||||
"phone": "+49123456789",
|
||||
...
|
||||
}
|
||||
|
||||
================================================================================
|
||||
ADVOWARE - Fetching Beteiligter
|
||||
================================================================================
|
||||
|
||||
Searching by name: Max Mustermann
|
||||
Trying endpoint: /contacts
|
||||
|
||||
✓ Found 2 results
|
||||
|
||||
Search Results:
|
||||
--------------------------------------------------------------------------------
|
||||
[
|
||||
{
|
||||
"id": 12345,
|
||||
"full_name": "Max Mustermann",
|
||||
"email": "max@example.com",
|
||||
...
|
||||
}
|
||||
]
|
||||
|
||||
================================================================================
|
||||
STRUCTURE COMPARISON
|
||||
================================================================================
|
||||
|
||||
EspoCRM Fields (25):
|
||||
----------------------------------------
|
||||
id (str)
|
||||
name (str)
|
||||
firstName (str)
|
||||
lastName (str)
|
||||
email (str)
|
||||
phone (str)
|
||||
...
|
||||
|
||||
Advoware Fields (30):
|
||||
----------------------------------------
|
||||
id (int)
|
||||
full_name (str)
|
||||
email (str)
|
||||
phone_number (str)
|
||||
...
|
||||
|
||||
Common Fields (5):
|
||||
----------------------------------------
|
||||
✓ id
|
||||
✓ email
|
||||
✗ phone
|
||||
EspoCRM: +49123456789
|
||||
Advoware: 0123456789
|
||||
|
||||
EspoCRM Only (20):
|
||||
----------------------------------------
|
||||
firstName
|
||||
lastName
|
||||
...
|
||||
|
||||
Advoware Only (25):
|
||||
----------------------------------------
|
||||
full_name
|
||||
phone_number
|
||||
...
|
||||
|
||||
Potential Field Mappings:
|
||||
----------------------------------------
|
||||
firstName → first_name
|
||||
lastName → last_name
|
||||
email → email
|
||||
phone → phone_number
|
||||
...
|
||||
|
||||
================================================================================
|
||||
Comparison saved to: /opt/motia-app/bitbylaw/scripts/beteiligte_comparison_result.json
|
||||
================================================================================
|
||||
```
|
||||
|
||||
### 2. JSON Output File
|
||||
|
||||
Saved to `bitbylaw/scripts/beteiligte_comparison_result.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"espocrm_data": {
|
||||
"id": "64a3f2b8c9e1234567890abc",
|
||||
"name": "Max Mustermann",
|
||||
...
|
||||
},
|
||||
"advoware_data": {
|
||||
"id": 12345,
|
||||
"full_name": "Max Mustermann",
|
||||
...
|
||||
},
|
||||
"comparison": {
|
||||
"espo_fields": ["id", "name", "firstName", ...],
|
||||
"advo_fields": ["id", "full_name", "email", ...],
|
||||
"common": ["id", "email"],
|
||||
"espo_only": ["firstName", "lastName", ...],
|
||||
"advo_only": ["full_name", "phone_number", ...],
|
||||
"suggested_mappings": [
|
||||
["firstName", "first_name"],
|
||||
["lastName", "last_name"],
|
||||
...
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## How It Works
|
||||
|
||||
### 1. EspoCRM Fetch
|
||||
|
||||
The script tries multiple entity types to find the data:
|
||||
- `Beteiligte` (custom VMH entity)
|
||||
- `Contact` (standard)
|
||||
- `Account` (standard)
|
||||
- `Lead` (standard)
|
||||
|
||||
### 2. Advoware Fetch
|
||||
|
||||
**By ID (if provided):**
|
||||
- Tries: `/contacts/{id}`, `/parties/{id}`, `/clients/{id}`
|
||||
|
||||
**By Name (if EspoCRM data available):**
|
||||
- Searches: `/contacts?search=...`, `/parties?search=...`, `/clients?search=...`
|
||||
|
||||
### 3. Comparison
|
||||
|
||||
- Lists all fields from both systems
|
||||
- Identifies common fields (same name)
|
||||
- Shows values for common fields
|
||||
- Suggests mappings based on naming patterns
|
||||
- Exports full comparison to JSON
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "ESPOCRM_API_KEY not set"
|
||||
|
||||
```bash
|
||||
# Check if .env exists and has the key
|
||||
cat .env | grep ESPOCRM_MARVIN_API_KEY
|
||||
|
||||
# Or set it manually
|
||||
export ESPOCRM_MARVIN_API_KEY=your_key_here
|
||||
```
|
||||
|
||||
### "Authentication failed - check API key"
|
||||
|
||||
1. Verify API key in EspoCRM admin panel
|
||||
2. Check API User permissions
|
||||
3. Ensure API User has access to entity type
|
||||
|
||||
### "Entity not found"
|
||||
|
||||
- Check if entity ID is correct
|
||||
- Verify entity type exists in EspoCRM
|
||||
- Check API User permissions for that entity
|
||||
|
||||
### "Advoware token error"
|
||||
|
||||
- Verify all Advoware credentials in `.env`
|
||||
- Check HMAC signature generation
|
||||
- Ensure API key is base64 encoded
|
||||
- Test token generation separately
|
||||
|
||||
## Next Steps
|
||||
|
||||
After running this script:
|
||||
|
||||
1. **Review JSON output** - Check `beteiligte_comparison_result.json`
|
||||
2. **Define mappings** - Create mapping table based on suggestions
|
||||
3. **Implement mapper** - Create transformation functions
|
||||
4. **Test sync** - Use mappings in sync event step
|
||||
|
||||
Example mapping implementation:
|
||||
|
||||
```python
|
||||
def map_espocrm_to_advoware(espo_entity: dict) -> dict:
|
||||
"""Transform EspoCRM Beteiligter to Advoware format"""
|
||||
return {
|
||||
'first_name': espo_entity.get('firstName'),
|
||||
'last_name': espo_entity.get('lastName'),
|
||||
'email': espo_entity.get('email'),
|
||||
'phone_number': espo_entity.get('phone'),
|
||||
# Add more mappings based on comparison...
|
||||
}
|
||||
```
|
||||
|
||||
## Related Files
|
||||
|
||||
- [services/espocrm.py](../services/espocrm.py) - EspoCRM API client
|
||||
- [services/advoware.py](../services/advoware.py) - Advoware API client
|
||||
- [services/ESPOCRM_SERVICE.md](../services/ESPOCRM_SERVICE.md) - EspoCRM docs
|
||||
- [config.py](../config.py) - Configuration
|
||||
Reference in New Issue
Block a user