Company:
Hire Resolve
Industry: Consulting
Deadline: Not specified
Job Type: Full Time
Experience: 5 years
Location: Western Cape
Province: Cape Town
Field: Data, Business Analysis and AIÂ , ICT / Computer
Job Description
- A digital intelligence firm that specialises in the development, management, and utilisation of big data technology, is seeking a Senior Data Engineer to step into the hands-on leadership role of Head of Data Platform & AI. You will take full ownership of their data infrastructure, elevating the architecture behind their proprietary Data Management Platform (DMP).
Responsibilities:
- Own and manage our entire GCP-based Data Platform, including core services like BigQuery, Cloud Functions, Composer, Dataflow, and Pub/Sub.
- Architect, build, and manage robust ETL/ELT pipelines to ingest structured and unstructured data from dozens of internal and external sources.
- Optimize the cost, speed, and performance of data storage, pipelines, and compute resources.
- Implement and enforce best practices across data validation, deduplication, enrichment, security, and governance.
- Create and enforce technical documentation, processes, and Standard Operating Procedures (SOPs) across the data team.
- Lead root-cause analysis of pipeline failures and ensure high data accuracy and uptime, including being available for after-hours escalations during mission-critical incidents.
- Mentor and coach data engineers and analysts; oversee technical delivery, quality, and team growth.
- Collaborate with AI, analytics, marketing, and operations teams to design high-impact data products that directly drive decision-making and revenue.
- Build internal tools and dashboards for reporting and marketing analytics teams.
Minimum Requirements:
- 5+ years of experience as a Data Engineer or in a Data Platform Leadership role.
- Advanced proficiency in Google Cloud Platform (GCP), specifically: BigQuery, Dataflow, Pub/Sub, Cloud Storage, and Composer.
- Strong coding skills in Python (with experience in SQLAlchemy or similar ORM), SQL, and demonstrable expertise in either Java or PHP.
- Extensive experience with data modelling, schema design, and performance tuning of large datasets.
- Proven ability to manage complex data workflows, including edge-case debugging and strict data security protocols.
- Comfortable with cloud cost optimisation and designing highly scalable, production-ready architecture.