About the role
Your Impact
As a Data Engineer, you will design, develop, deploy, and maintain data architecture which employs various methods to transform raw data into processed data. You will own the DataOps infrastructure, ensuring the performance, reliability, and scalability of data ingestion and processing pipelines to support evolving business and operational demands.
To succeed in this data engineering position, you should have strong problem-solving skills and the ability to combine data from different sources. Data engineer skills also include familiarity with several programming languages.
What you’ll do
Orchestrate the ingestion, processing, and storage of raw data within structured and unstructured data repositories.
Design, Develop, Deploy and Support data infrastructure, pipelines and architecture.
Implement reliable, scalable, and tested solutions to automate data ingestion.
Development of systems to manage batch processing and real-time streaming of data.
Evaluate business needs and objectives.
Support implementation of data governance requirements.
Facilitate pipelines, which prepare data for prescriptive and predictive modelling.
Working with domain teams to scale the processing of data.
Identify opportunities for data acquisition.
Combine raw information from different sources.
Manage and maintain automated tools for data quality and reliability.
Explore ways to enhance data quality and reliability.
Collaborate with data scientists, IT, and architects on several projects.
What you’ll bring
Successful Candidates will have previous experience as a data or software engineer in a similar role. Requirements include;
Technical expertise in designing, building, and maintaining data pipelines, data warehouses, and leveraging data services.
Proficient in DataOps methodologies and tools, including experience with continuous integration/continuous delivery (CI/CD) pipelines, containerisation, and workflow orchestration.
Familiar with ETL/ELT frameworks, and experienced with Big Data Processing Tools (e.g. Spark, Airflow, Hive, etc.)
Knowledge of programming languages (e.g. Java, Python, SQL)
Hands-on experience with SQL/NoSQL database design.
De…
