Our award-winning Fintech client are looking for a Data Engineer to build complex data pipelines in order to compute a variety of advanced analytics and metrics across our universe of exchange order book data.
You will utilise AWS Cloud Technology and Python toolsets to develop these pipelines for new and existing data content at scale and work with the Product team in a truly agile customer centric environment.
This is an amazing opportunity to join a cutting edge Fintech company with the potential to grow your technical, finance, and cloud and data engineering skills.
Responsibilities
- Owning the full scope of assigned development projects, including liaising with the business to gather requirements, designing the solution with the team, and building and delivering completed functionality to production.
- Design & implementation of the task scheduling (Airflow), distributed computation system (Python, Celery, EMR) and data lakes / data warehouses (S3, PostgreSQL).
- Helping to drive improvements in all areas, including the analytics framework, data tooling, deployment, data quality, testing and CI.
- Understanding the data, how it reaches the business, how we process it and deliver it to customers.
- Some second-line support of the daily parsing processes and the data, shared amongst the team.
Essential
- Industry experience of production-level big data batch pipelines on AWS.
- Industry experience of developing on a Linux platform
- Experience of industry-standard development methodologies such as source code control, unit testing and continuous integration
- Ability to self-organise and deliver a project over a period of up to a few months
- Strong problem solving skills
- Strong communication skills
- Computer science or other STEM degree
What makes this job special:
- The team is a mixture of data engineering and data science individuals. All your work will have a direct impact on the business, as we work closely with our product team.
- You will be part of a team that works on a petabyte scale data lake and an award-winning data science environment to analyse the data.
- You will come in at a time where you will help us shape technology in the firm.
Benefits:
- Competitive salary
- 25 days holiday plus bank holidays
- Share Options after completion of probationary period
- Pension Scheme
- Combination of remote and London based office working
- A Wellbeing Physical Activity budget
- Continuous learning through training and challenging projects
- Collaborative culture
- Weekly team lunches
- Free fruit, snacks and drinks provided throughout the day (when office based)
- Fridge fully stocked with beer and wine (when office based)
- Regular Team Socials
- Cycle to Work Scheme