motia/bitbylaw/scripts

Beteiligte Structure Comparison Tool

Purpose

This helper script fetches entity data from both EspoCRM and Advoware to compare their data structures. This helps understand:

• What fields exist in each system
• How field names differ
• Potential field mappings for synchronization
• Data type differences

Usage

cd /opt/motia-app

# Basic usage: Compare by EspoCRM ID (will auto-search in Advoware)
python bitbylaw/scripts/compare_beteiligte.py <espocrm_entity_id>

# Advanced: Specify both IDs
python bitbylaw/scripts/compare_beteiligte.py <espocrm_entity_id> <advoware_id>

Examples

# Example 1: Fetch from EspoCRM and search in Advoware by name
python bitbylaw/scripts/compare_beteiligte.py 64a3f2b8c9e1234567890abc

# Example 2: Fetch from both systems by ID
python bitbylaw/scripts/compare_beteiligte.py 64a3f2b8c9e1234567890abc 12345

# Example 3: Using the virtual environment
source python_modules/bin/activate
python bitbylaw/scripts/compare_beteiligte.py 64a3f2b8c9e1234567890abc

Requirements

Environment Variables

Make sure these are set in .env or environment:

# EspoCRM
ESPOCRM_API_BASE_URL=https://crm.bitbylaw.com/api/v1
ESPOCRM_MARVIN_API_KEY=your_api_key_here

# Advoware
ADVOWARE_API_BASE_URL=https://www2.advo-net.net:90/
ADVOWARE_API_KEY=your_base64_encoded_key
ADVOWARE_USER=your_user
ADVOWARE_PASSWORD=your_password
ADVOWARE_KANZLEI=your_kanzlei
ADVOWARE_DATABASE=your_database
# ... (see config.py for all required vars)

Dependencies

pip install aiohttp redis python-dotenv requests

Output

The script produces:

1. Console Output

================================================================================
BETEILIGTE STRUCTURE COMPARISON TOOL
================================================================================

EspoCRM Entity ID: 64a3f2b8c9e1234567890abc

Environment Check:
----------------------------------------
ESPOCRM_API_BASE_URL: https://crm.bitbylaw.com/api/v1
ESPOCRM_API_KEY: ✓ Set
ADVOWARE_API_BASE_URL: https://www2.advo-net.net:90/
ADVOWARE_API_KEY: ✓ Set

================================================================================
ESPOCRM - Fetching Beteiligter
================================================================================

Trying entity type: Beteiligte

✓ Success! Found in Beteiligte

Entity Structure:
--------------------------------------------------------------------------------
{
  "id": "64a3f2b8c9e1234567890abc",
  "name": "Max Mustermann",
  "firstName": "Max",
  "lastName": "Mustermann",
  "email": "max@example.com",
  "phone": "+49123456789",
  ...
}

================================================================================
ADVOWARE - Fetching Beteiligter
================================================================================

Searching by name: Max Mustermann
  Trying endpoint: /contacts

✓ Found 2 results

Search Results:
--------------------------------------------------------------------------------
[
  {
    "id": 12345,
    "full_name": "Max Mustermann",
    "email": "max@example.com",
    ...
  }
]

================================================================================
STRUCTURE COMPARISON
================================================================================

EspoCRM Fields (25):
----------------------------------------
  id                             (str)
  name                           (str)
  firstName                      (str)
  lastName                       (str)
  email                          (str)
  phone                          (str)
  ...

Advoware Fields (30):
----------------------------------------
  id                             (int)
  full_name                      (str)
  email                          (str)
  phone_number                   (str)
  ...

Common Fields (5):
----------------------------------------
  ✓ id
  ✓ email
  ✗ phone
      EspoCRM: +49123456789
      Advoware: 0123456789

EspoCRM Only (20):
----------------------------------------
  firstName
  lastName
  ...

Advoware Only (25):
----------------------------------------
  full_name
  phone_number
  ...

Potential Field Mappings:
----------------------------------------
  firstName                      → first_name
  lastName                       → last_name
  email                          → email
  phone                          → phone_number
  ...

================================================================================
Comparison saved to: /opt/motia-app/bitbylaw/scripts/beteiligte_comparison_result.json
================================================================================

2. JSON Output File

Saved to bitbylaw/scripts/beteiligte_comparison_result.json:

{
  "espocrm_data": {
    "id": "64a3f2b8c9e1234567890abc",
    "name": "Max Mustermann",
    ...
  },
  "advoware_data": {
    "id": 12345,
    "full_name": "Max Mustermann",
    ...
  },
  "comparison": {
    "espo_fields": ["id", "name", "firstName", ...],
    "advo_fields": ["id", "full_name", "email", ...],
    "common": ["id", "email"],
    "espo_only": ["firstName", "lastName", ...],
    "advo_only": ["full_name", "phone_number", ...],
    "suggested_mappings": [
      ["firstName", "first_name"],
      ["lastName", "last_name"],
      ...
    ]
  }
}

How It Works

1. EspoCRM Fetch

The script tries multiple entity types to find the data:

• Beteiligte (custom VMH entity)
• Contact (standard)
• Account (standard)
• Lead (standard)

2. Advoware Fetch

By ID (if provided):

• Tries: /contacts/{id}, /parties/{id}, /clients/{id}

By Name (if EspoCRM data available):

• Searches: /contacts?search=..., /parties?search=..., /clients?search=...

3. Comparison

• Lists all fields from both systems
• Identifies common fields (same name)
• Shows values for common fields
• Suggests mappings based on naming patterns
• Exports full comparison to JSON

Troubleshooting

"ESPOCRM_API_KEY not set"

# Check if .env exists and has the key
cat .env | grep ESPOCRM_MARVIN_API_KEY

# Or set it manually
export ESPOCRM_MARVIN_API_KEY=your_key_here

"Authentication failed - check API key"

1. Verify API key in EspoCRM admin panel
2. Check API User permissions
3. Ensure API User has access to entity type

"Entity not found"

• Check if entity ID is correct
• Verify entity type exists in EspoCRM
• Check API User permissions for that entity

"Advoware token error"

• Verify all Advoware credentials in .env
• Check HMAC signature generation
• Ensure API key is base64 encoded
• Test token generation separately

Next Steps

After running this script:

1. Review JSON output - Check beteiligte_comparison_result.json
2. Define mappings - Create mapping table based on suggestions
3. Implement mapper - Create transformation functions
4. Test sync - Use mappings in sync event step

Example mapping implementation:

def map_espocrm_to_advoware(espo_entity: dict) -> dict:
    """Transform EspoCRM Beteiligter to Advoware format"""
    return {
        'first_name': espo_entity.get('firstName'),
        'last_name': espo_entity.get('lastName'),
        'email': espo_entity.get('email'),
        'phone_number': espo_entity.get('phone'),
        # Add more mappings based on comparison...
    }

Related Files

• services/espocrm.py - EspoCRM API client
• services/advoware.py - Advoware API client
• services/ESPOCRM_SERVICE.md - EspoCRM docs
• config.py - Configuration