Skip to content

AST 1205 adp to costpoint flow

Create ADP Employee to Costpoint Flow

Implement the main Prefect flow that orchestrates employee data sync from ADP to Costpoint using the connectors and mapping configuration.

Key details:

  • Create in workflows/adp_integration.py
  • Follow ETL pattern: load credentials → extract → transform → load
  • Load ADP and Costpoint credentials from AWS Secrets Manager (see TBD-09)
  • Pass credentials to connector instances
  • Use @task(retries=3) decorator for automatic retry logic
  • Use .map() for parallel employee processing
  • Use unmapped() for shared parameters in map operations
  • Handle partial failures: return error dict with {"error": str(e), "failed_at": "stage"}
  • Track success/failure counts for reporting
  • Create Prefect artifacts for observability
  • Accept parameters: customer_id, workflow_name, source_config, destination_config, mapping_block_name
  • See workflows/examples/pokemon_demo.py for reference pattern

Open questions:

  • Should this be incremental sync (changed records) or full sync?