This opportunity offers the chance to work on complex data engineering initiatives focused on Databricks, PySpark, SQL and large-scale transaction and loyalty data pipelines. The role will support production-ready integration, migration stabilisation, orchestration, monitoring and data quality across critical datasets and workflows. This position is well suited to professionals who enjoy solving pipeline reliability challenges, improving reconciliation processes and delivering robust cloud-based data solutions.
Required Experience:
- Experience delivering and maintaining large-scale data pipelines in production environments
- Experience working with transaction, payment, loyalty, EPOS/WRE and identity-matching data
- Experience with historical, full and incremental loads, high-water marks and reconciliation processes
- Experience in production troubleshooting, dependency analysis, pipeline repointing and migration stabilisation
- Experience supporting workflow orchestration, retry and restart patterns, and operational handover into business-as-usual support
- Experience with integration testing, production validation and data completeness investigations
Required Skills:
- Strong SQL, Python, PySpark and Databricks pipeline engineering skills
- Hands-on ETL/ELT development across cloud-based data platforms
- Workflow orchestration and dependency management
- Idempotent processing, failure recovery, error handling and controlled reloads
- Monitoring, logging, alerting, data-quality checks and freshness controls
- Git, pull requests, unit testing, CI/CD and release support
- API, file-based and SFTP integration patterns
- Technical design preparation and structured testing support
Preferred Qualifications:
- Exposure to reporting or Power BI datasets
- Delta Lake or Delta MERGE knowledge
- Databricks Asset Bundles experience
- Azure DevOps experience
- MLOps or model deployment experience
- Knowledge of SFMC or Comarch integration
- Experience with customer decisioning or personalisation platforms
- Experience with AWS-to-Databricks migration
- Exposure to Informatica, SSIS or legacy ETL environments
- Multi-market SCI knowledge
About Airswift:
Airswift is an international workforce solutions provider within the energy, process and infrastructure industries. Airswift serves as a strategic partner to clients, offering a turnkey workforce solution to capture and deliver the top talent needed to complete successful projects by aligning with unique project needs. With over 1000 employees and 9,000 contractors operating in over 60 countries, Airswift's geographical reach and pool of talent available is unmatched in the industry.