Building a Durable Polar AccessLink Import Pipeline

The initial Polar AccessLink integration worked in the narrowest sense: it downloaded data and wrote database records. However, it did not yet behave like a production-grade synchronization system. A partial network timeout left the UI hanging, retries risked duplicate session writes, and webhook notifications required idempotent handling under concurrent background execution.

The Architectural Problem

The original implementation treated one long HTTP request as the entire ingestion job. That tightly coupled four distinct responsibilities:

1. Requesting transaction data via OAuth 2.0 headers,
2. Downloading multi-megabyte JSON payload archives,
3. Parsing sessions, sport mappings, and GPS route points,
4. Updating cached aggregate readiness metrics in the database.

When any single network call timed out, the client received a 504 error. The server had no durable checkpoint, and the frontend could not communicate whether the data was partially imported or completely missing.

A synchronization endpoint should decouple job initialization from execution. HTTP requests should never be responsible for long-running I/O operations.

The Idempotent Pipeline & Job State Machine

The current implementation uses PostgreSQL persistence through Drizzle and idempotency keys derived from Polar session hashes:

type ImportJob = {
  id: string;
  externalTransactionId: string;
  status: 'queued' | 'running' | 'complete' | 'failed';
  processedSessions: number;
  totalSessions: number;
  idempotencyKey: string;
  retryCount: number;
  error?: string;
};

Key Technical Decisions & Trade-Offs

• Idempotent Ingestion Keys: Each workout session is identified by sha256(polar_session_id + start_time). Re-running an import job or receiving duplicate webhooks cleanly upserts without corrupting historical training totals.
• Durable Job State: Import jobs use persisted state transitions (queued -> running -> complete) so the sync lifecycle remains auditable across retries.
• Retry Strategy with Exponential Backoff: Network failures against Polar’s rate-limited endpoints trigger retries with jitter (up to 3 attempts) before moving the job state to failed and alerting system logs.

Operational Trade-Offs & Production Signals

By enforcing an explicit status boundary, the API returns an HTTP 202 Accepted immediately, allowing the client to poll a lightweight status endpoint (/api/sync/status/:jobId).

Production Telemetry Signals Monitored:

• Time spent downloading payload versus parsing JSON records,
• Idempotent skip rate (identifying duplicate webhooks),
• Unmapped sport types falling back to generic telemetry categories,
• Cache invalidation latency for downstream readiness dashboards.