Profectus have partnered with an inter-disciplinary research organisation who are striving to solve complex societal problems. They are looking to hire a Data Engineer to help them deliver on a key goal of helping them build data pipeline frameworks, design, and implement data models, and help Data Scientists achieve key insights for data-driven decision making for their clients in the public sector. The suitable candidate will also be instrumental in helping them build their AWS (SaaS) products.

Requirements;

• Industry experience as a Data Engineer or related specialty (e.g., Software Engineer, Business Intelligence Engineer, Data Scientist) with a track record of manipulating, processing, and extracting value from large datasets
• Experience building and operating highly available, distributed systems of data extraction, data ingestion, and processing of large datasets
• Experience building data products and integrating and managing datasets from multiple external sources
• Design, develop and maintain scalable-automated data pipeline frameworks on AWS, and supporting our AWS product delivery services to clients
• Researching and implementing relevant tools and technologies in the data engineering space for our AWS products
• Architect and design reliable, scalable data infrastructure for our AWS Products
• Ability to work with different types of storage systems (Relational database, NoSQL)
• Experience leading large-scale data warehousing and analytics projects, including using AWS technologies – RedShift, S3, EC2
• Design, optimise and implement data models and data schema for different data sources
• Engineering data flows/pipelines to enable scaling and repeatable use of solutions
• Design and develop data ingestion methods for external data sources (different data vendors) into our AWS products, and create useful ETL pipelines
• Design, develop, implement, and maintain necessary custom ETL/ELT pipelines to make sure required processes perform optimally
• Possess skillsets in core SQL Competencies, such as Stored Procedures, Batch Jobs and can implement highly performant SQL code for our AWS products
• Building, designing, refactoring, and optimising AWS data lakes and AWS data warehouses from a variety of data sources
• Consistently liaise and work together with Data Scientists, Data Engineers, Data Analysts, DevOps engineers, Cloud Solutions Architects, and stakeholders such as product owners and business analysts to gather and analyse data requirements

Basic Qualifications required for this role:

• Bachelor's degree in Computer Science, Engineering, Statistics, Mathematics, or related field
• 2-3 years’ experience in Data Engineering
• Extensive experiences in AWS services like RDS, Lambda, Glue, Neptune, Athena, DMS, Redshift, EC2 and machine learning tools
• Good working knowledge of any of the following: Python, Java, SQL
• Experience with version control tools (e.g., Git)
• Background in supporting data scientists and software development team, to conduct large scale data analyses / modelling to support business decision making
• Knowledge of software engineering best practices across the development lifecycle, including agile methodologies, coding standards, code reviews, source management, build processes, testing, and operations
• An ability and interest in working in a fast-paced environment
• Strong verbal/written communication and data presentation skills, including an ability to effectively communicate with both business and technical teams

Highly desirable:

• Experience working with AWS cloud and relevant microservice environments
• Good understanding of software development lifecycle
• Experiences working in Product based developments
• Experiences working with either a MapReduce or MPP system
• Good understanding of data modelling and data engineering tools such
  as Kafka, Spark, and Hadoop
• Experiences with Graph databases

*Please note: This role requires enhanced police and security vetting, which requires a minimum of 3 years UK residence, a criminal records check, and a right to work in the UK check.  Please ensure you meet the residence requirement prior to applying.  Any candidate who is invited for interview must complete a residence and criminal records declaration and produce evidence of right to work in the UK.