Data Engineer

Project Overview

The project is an ambitious initiative aimed at overhauling Client’s legacy data systems with a modern, AWS-based Snowflake platform. This transformation is critical for integrating sales data across the UK and North America, improving data reliability, and supporting dynamic business decisions. The project faces challenges due to the diversity and customization of legacy systems, making data consolidation and analysis complex yet vital for success.

Role Summary:

As a Data Engineer on the project, you will play a pivotal role in mapping, analyzing, and ensuring the quality and consistency of data migrating from various legacy systems into the new Snowflake platform. This position requires a keen analytical mind, proficiency in SQL and Python, and an in-depth understanding of data warehousing principles.

We are seeking an individual with strong analytical skills, proficiency in data manipulation and data mapping, and the ability to translate complex data into actionable insights. The role involves collaborating with stakeholders, documenting the business rules and data logic, and communicate findings effectively. Some level of data modelling is beneficial.

Key Responsibilities:

• Conduct comprehensive data analysis to support the mapping of data from 2 old platforms to the new Snowflake-based common data model for North America.
• Work closely with the Data Architect to define and validate data mapping rules, ensuring seamless integration of diverse data sets.
• Develop and execute SQL queries for data extraction, transformation, and loading (ETL) processes, ensuring data integrity and alignment with project objectives.
• Utilize Python for data preprocessing, cleansing, and automation tasks to enhance data quality and efficiency.
• Participate in the testing and validation of data pipelines, employing rigorous analytical methods to identify discrepancies and ensure data accuracy.
• Collaborate with the US-based data engineering team to understand the existing data bricks platform and contribute to the migration strategy to Snowflake.
• Support the continuous improvement of data models and ETL processes, incorporating feedback from stakeholders to meet evolving business needs
• Analyses ticketing data, investigation of complex and or convoluted data issues.
• Documentation of business rules and data logic.
• Presentation and playback of data and issues to data team.
• Prepare and present findings and recommendations to both technical and non-technical audiences.
• Own the data testing and reconciliation efforts.
• Conduct data quality assessments, identify gaps, and implement strategies to enhance data integrity.
• Perform exploratory data analysis (EDA) to understand data trends, identify key insights, and recommend actionable strategies.
• Collaborate with product teams, and stakeholders to gather requirements and translate business needs into technical solutions.
• Monitoring and maintaining data pipelines, ensuring data quality and integrity.

Essential Skills and Qualifications:

• Experience with SQL and familiarity with data warehousing solutions like Snowflake.
• Familiarity with AWS services and DBT.
• Proficient in Python, with a strong preference for candidates with experience in Python.
• Solid understanding of data warehousing principles, ETL processes, and data modeling.
• Experience with data quality assurance and validation techniques.
• Strong analytical skills, with the ability to derive insights from complex datasets and translate these into actionable business strategies.
• Creative problem-solving abilities and a proactive approach to identifying and addressing challenges.
• Excellent communication skills, capable of articulating complex concepts to diverse audiences, including revenue managers and senior stakeholders.
• Ability to work collaboratively in a fast-paced, team-oriented environment, as well as independently with minimal supervision.

Desirable Skills:

• Experience with data bricks or similar platforms.
• Knowledge of data visualization tools and techniques to effectively communicate insights.
• Understanding the entertainment or ticketing industry is a plus.

Personal Attributes:

• A self-starter with a strong desire to take on challenges and achieve ambitious goals.
• Analytical and curious mindset with a strong desire to solve complex data problems.
• Highly adaptable, with the ability to thrive in an evolving project landscape and rapidly changing priorities.
• A team player who values collaboration shares knowledge freely, and supports colleagues in achieving collective objectives.
• Strong ethical standards and integrity, with a commitment to producing high-quality and transparent work.