motia/bitbylaw/docs/DEVELOPMENT.md

Development Guide

Setup

Prerequisites

• Node.js: 18.x oder höher
• Python: 3.13 oder höher
• Redis: 6.x oder höher
• Git: Für Version Control
• Motia CLI: Wird automatisch via npm installiert

Initial Setup

# 1. Repository navigieren
cd /opt/motia-app/bitbylaw

# 2. Node.js Dependencies installieren
npm install

# 3. Python Virtual Environment erstellen (falls nicht vorhanden)
python3.13 -m venv python_modules

# 4. Python Virtual Environment aktivieren
source python_modules/bin/activate

# 5. Python Dependencies installieren
pip install -r requirements.txt

# 6. Redis starten (falls nicht läuft)
sudo systemctl start redis-server

# 7. Environment Variables konfigurieren (siehe CONFIGURATION.md)
# Erstellen Sie eine .env Datei oder setzen Sie in systemd

# 8. Development Mode starten
npm run dev

Entwicklungsumgebung

Empfohlene IDE: VS Code mit Extensions:

• Python (Microsoft)
• TypeScript (Built-in)
• ESLint
• Prettier

VS Code Settings (.vscode/settings.json):

{
  "python.defaultInterpreterPath": "${workspaceFolder}/python_modules/bin/python",
  "python.linting.enabled": true,
  "python.linting.pylintEnabled": false,
  "python.linting.flake8Enabled": true,
  "editor.formatOnSave": true,
  "files.exclude": {
    "**/__pycache__": true,
    "**/node_modules": true
  }
}

Projektstruktur

bitbylaw/
├── docs/                          # Dokumentation
│   ├── ARCHITECTURE.md           # System-Architektur
│   ├── DEVELOPMENT.md            # Dieser Guide
│   ├── API.md                    # API-Referenz
│   ├── CONFIGURATION.md          # Environment & Config
│   ├── DEPLOYMENT.md             # Deployment-Guide
│   └── TROUBLESHOOTING.md        # Fehlerbehebung
├── steps/                         # Motia Steps (Business Logic)
│   ├── advoware_proxy/           # API Proxy Steps
│   │   ├── README.md             # Modul-Dokumentation
│   │   ├── *.py                  # Step-Implementierungen
│   │   └── *.md                  # Step-Detail-Doku
│   ├── advoware_cal_sync/        # Calendar Sync Steps
│   │   ├── README.md
│   │   ├── *.py
│   │   └── *.md
│   └── vmh/                      # VMH Webhook Steps
│       ├── README.md
│       ├── webhook/              # Webhook Receiver
│       └── *.py
├── services/                      # Shared Services
│   └── advoware.py               # Advoware API Client
├── config.py                      # Configuration Loader
├── package.json                   # Node.js Dependencies
├── requirements.txt               # Python Dependencies
├── tsconfig.json                  # TypeScript Config
├── motia-workbench.json          # Motia Flow Definitions
└── README.md                     # Projekt-Übersicht

Konventionen

Verzeichnisse:

• steps/ - Motia Steps (Handler-Funktionen)
• services/ - Wiederverwendbare Service-Layer
• docs/ - Dokumentation
• python_modules/ - Python Virtual Environment (nicht committen)
• node_modules/ - Node.js Dependencies (nicht committen)

Dateinamen:

• Steps: {module}_{action}_step.py (z.B. calendar_sync_cron_step.py)
• Services: {service_name}.py (z.B. advoware.py)
• Dokumentation: {STEP_NAME}.md oder {TOPIC}.md

Coding Standards

Python

Style Guide: PEP 8 mit folgenden Anpassungen:

• Line length: 120 Zeichen (statt 79)
• String quotes: Single quotes bevorzugt

Linting:

# Flake8 check
flake8 steps/ services/

# Autopep8 formatting
autopep8 --in-place --aggressive --aggressive steps/**/*.py

Type Hints:

from typing import Dict, List, Optional, Any

async def handler(req: Dict[str, Any], context: Any) -> Dict[str, Any]:
    pass

Docstrings:

def function_name(param1: str, param2: int) -> bool:
    """
    Brief description of function.

    Args:
        param1: Description of param1
        param2: Description of param2

    Returns:
        Description of return value

    Raises:
        ValueError: When something goes wrong
    """
    pass

TypeScript/JavaScript

Style Guide: Standard mit Motia-Konventionen

Formatting: Prettier (automatisch via Motia)

Naming Conventions

Variables: snake_case (Python), camelCase (TypeScript) Constants: UPPER_CASE Classes: PascalCase Functions: snake_case (Python), camelCase (TypeScript) Files: snake_case.py, kebab-case.ts

Error Handling

Pattern:

async def handler(req, context):
    try:
        # Main logic
        result = await some_operation()
        return {'status': 200, 'body': {'result': result}}
    
    except SpecificError as e:
        # Handle known errors
        context.logger.error(f"Specific error: {e}")
        return {'status': 400, 'body': {'error': 'Bad request'}}
    
    except Exception as e:
        # Catch-all
        context.logger.error(f"Unexpected error: {e}", exc_info=True)
        return {'status': 500, 'body': {'error': 'Internal error'}}

Logging:

# Use context.logger for Motia Workbench integration
context.logger.debug("Detailed information")
context.logger.info("Normal operation")
context.logger.warning("Warning message")
context.logger.error("Error message", exc_info=True)  # Include stack trace

Motia Step Development

Step Structure

Every Step must have:

1. Config Dictionary: Defines step metadata
2. Handler Function: Implements business logic

Minimal Example:

config = {
    'type': 'api',                    # api|event|cron
    'name': 'My API Step',
    'description': 'Brief description',
    'path': '/api/my-endpoint',       # For API steps
    'method': 'GET',                  # For API steps
    'schedule': '0 2 * * *',          # For cron steps
    'emits': ['topic.name'],          # Events this step emits
    'subscribes': ['other.topic'],    # Events this step subscribes to (event steps)
    'flows': ['my-flow']              # Flow membership
}

async def handler(req, context):
    """Handler function - must be async."""
    # req: Request object (API) or Event data (event step)
    # context: Motia context (logger, emit, etc.)
    
    # Business logic here
    
    # For API steps: return HTTP response
    return {'status': 200, 'body': {'result': 'success'}}
    
    # For event steps: no return value (or None)

Step Types

1. API Steps (type: 'api'):

config = {
    'type': 'api',
    'name': 'My Endpoint',
    'path': '/api/resource',
    'method': 'POST',
    'emits': [],
    'flows': ['main']
}

async def handler(req, context):
    # Access request data
    body = req.get('body')
    query_params = req.get('queryParams')
    headers = req.get('headers')
    
    # Return HTTP response
    return {
        'status': 200,
        'body': {'data': 'response'},
        'headers': {'X-Custom': 'value'}
    }

2. Event Steps (type: 'event'):

config = {
    'type': 'event',
    'name': 'Process Event',
    'subscribes': ['my.topic'],
    'emits': ['other.topic'],
    'flows': ['main']
}

async def handler(event_data, context):
    # Process event
    entity_id = event_data.get('entity_id')
    
    # Emit new event
    await context.emit({
        'topic': 'other.topic',
        'data': {'processed': True}
    })
    
    # No return value needed

3. Cron Steps (type: 'cron'):

config = {
    'type': 'cron',
    'name': 'Daily Job',
    'schedule': '0 2 * * *',  # Cron expression
    'emits': ['job.complete'],
    'flows': ['main']
}

async def handler(req, context):
    # Scheduled logic
    context.logger.info("Cron job triggered")
    
    # Emit event to start pipeline
    await context.emit({
        'topic': 'job.complete',
        'data': {}
    })

Context API

Available Methods:

# Logging
context.logger.debug(msg)
context.logger.info(msg)
context.logger.warning(msg)
context.logger.error(msg, exc_info=True)

# Event Emission
await context.emit({
    'topic': 'my.topic',
    'data': {'key': 'value'}
})

# Flow information
context.flow_id      # Current flow ID
context.step_name    # Current step name

Testing

Unit Tests

Location: Tests neben dem Code (z.B. *_test.py)

Framework: pytest

# test_my_step.py
import pytest
from unittest.mock import AsyncMock, MagicMock
from my_step import handler, config

@pytest.mark.asyncio
async def test_handler_success():
    # Arrange
    req = {'body': {'key': 'value'}}
    context = MagicMock()
    context.logger = MagicMock()
    
    # Act
    result = await handler(req, context)
    
    # Assert
    assert result['status'] == 200
    assert 'result' in result['body']

Run Tests:

pytest steps/

Integration Tests

Manual Testing mit curl:

# API Step testen
curl -X POST "http://localhost:3000/api/my-endpoint" \
  -H "Content-Type: application/json" \
  -d '{"key": "value"}'

# Mit Query Parameters
curl -X GET "http://localhost:3000/advoware/proxy?endpoint=employees"

Motia Workbench: Nutzen Sie die Workbench UI zum Testen und Debugging

Test-Daten

Redis Mock Data:

# Set test token
redis-cli -n 1 SET advoware_access_token "test_token" EX 3600

# Set test lock
redis-cli -n 1 SET "calendar_sync:lock:TEST" "1" EX 300

# Check dedup set
redis-cli -n 1 SMEMBERS "vmh:beteiligte:create_pending"

Debugging

Local Development

Start in Dev Mode:

npm run dev

Enable Debug Logging:

export MOTIA_LOG_LEVEL=debug
npm start

Node.js Inspector:

# Already enabled in systemd (--inspect)
# Connect with Chrome DevTools: chrome://inspect

Motia Workbench

Access: http://localhost:3000/workbench (wenn verfügbar)

Features:

• Live logs
• Flow visualization
• Event traces
• Step execution history

Redis Debugging

# Connect to Redis
redis-cli

# Switch database
SELECT 1

# List all keys
KEYS *

# Get value
GET advoware_access_token

# Check SET members
SMEMBERS vmh:beteiligte:create_pending

# Monitor live commands
MONITOR

---

Utility Scripts

Calendar Sync Utilities

Helper-Scripts für Wartung und Debugging der Calendar-Sync-Funktionalität.

Standort: scripts/calendar_sync/

Verfügbare Scripts:

# Alle Employee-Locks in Redis löschen (bei hängenden Syncs)
python3 scripts/calendar_sync/delete_employee_locks.py

# Alle Google Kalender löschen (außer Primary) - VORSICHT!
python3 scripts/calendar_sync/delete_all_calendars.py

Use Cases:

• Lock Cleanup: Wenn ein Sync-Prozess abgestürzt ist und Locks nicht aufgeräumt wurden
• Calendar Reset: Bei fehlerhafter Synchronisation oder Tests
• Debugging: Untersuchung von Sync-Problemen

Dokumentation: scripts/calendar_sync/README.md

⚠️ Wichtig:

• Immer Motia Service stoppen vor Cleanup: sudo systemctl stop motia
• Nach Cleanup Service neu starten: sudo systemctl start motia
• delete_all_calendars.py löscht unwiderruflich alle Kalender!

---

Common Issues

1. Import Errors:

# Ensure PYTHONPATH is set
export PYTHONPATH=/opt/motia-app/bitbylaw
source python_modules/bin/activate

2. Redis Connection Errors:

# Check Redis is running
sudo systemctl status redis-server

# Test connection
redis-cli ping

3. Token Errors:

# Clear cached token
redis-cli -n 1 DEL advoware_access_token advoware_token_timestamp

Git Workflow

Branch Strategy

• main - Production code
• develop - Integration branch
• feature/* - Feature branches
• fix/* - Bugfix branches

Commit Messages

Format: <type>: <subject>

Types:

• feat: New feature
• fix: Bug fix
• docs: Documentation only
• refactor: Code refactoring
• test: Adding tests
• chore: Maintenance tasks

Examples:

feat: add calendar sync retry logic
fix: prevent duplicate webhook processing
docs: update API documentation
refactor: extract common validation logic

Pull Request Process

1. Create feature branch from develop
2. Implement changes
3. Write/update tests
4. Update documentation
5. Create PR with description
6. Code review
7. Merge to develop
8. Deploy to staging
9. Merge to main (production)

Performance Optimization

Profiling

Python Memory Profiling:

# Install memory_profiler
pip install memory_profiler

# Profile a function
python -m memory_profiler steps/my_step.py

Node.js Profiling:

# Already enabled with --inspect flag
# Use Chrome DevTools Performance tab

Best Practices

Async/Await:

# Good: Concurrent requests
results = await asyncio.gather(
    fetch_data_1(),
    fetch_data_2()
)

# Bad: Sequential (slow)
result1 = await fetch_data_1()
result2 = await fetch_data_2()

Redis Pipelining:

# Good: Batch operations
pipe = redis.pipeline()
pipe.get('key1')
pipe.get('key2')
results = pipe.execute()

# Bad: Multiple round-trips
val1 = redis.get('key1')
val2 = redis.get('key2')

Avoid N+1 Queries:

# Good: Batch fetch
employee_ids = [1, 2, 3]
employees = await advoware.api_call(
    '/employees',
    params={'ids': ','.join(map(str, employee_ids))}
)

# Bad: Loop with API calls
employees = []
for emp_id in employee_ids:
    emp = await advoware.api_call(f'/employees/{emp_id}')
    employees.append(emp)

Code Review Checklist

• Code follows style guide
• Type hints present (Python)
• Error handling implemented
• Logging added at key points
• Tests written/updated
• Documentation updated
• No secrets in code
• Performance considered
• Redis keys documented
• Events documented

Deployment

See DEPLOYMENT.md for detailed deployment instructions.

Quick Deploy to Production:

# 1. Pull latest code
git pull origin main

# 2. Install dependencies
npm install
pip install -r requirements.txt

# 3. Restart service
sudo systemctl restart motia.service

# 4. Check status
sudo systemctl status motia.service

# 5. Monitor logs
sudo journalctl -u motia.service -f

Resources

Documentation

• Motia Framework
• Advoware API (internal)
• Google Calendar API

Tools

• Redis Commander (if installed)
• Motia Workbench

Team Contacts

• Architecture Questions: [Lead Developer]
• Deployment Issues: [DevOps Team]
• API Access: [Integration Team]