Lead Data Engineer - AWS
Hybrid
London, United Kingdom
£ 80 / hour
Freelance
03-03-2025
Job Specifications
***This role falls outside IR35***
James Adams is currently looking for an experienced Lead Data Engineer. This is a 3 month initial contract working hybrid remotely in London.
About the Role:
We are seeking a highly skilled Senior Data Engineer to join our team.
This role is ideal for a data engineering expert with a deep understanding of Snowflake,
AWS data tooling, data warehousing, and best practices in cloud-based data platforms. You
will be instrumental in building and optimizing our data pipelines, ensuring scalability,
efficiency, and reliability.
Key Responsibilities:
Design, develop, and maintain robust data ingestion pipelines using AWS services (S3, Lambda, ECR, ECS, DynamoDB).
Develop and optimize data transformation pipelines using a combination of SQL and Python.
Work with Snowflake as a data platform, ensuring efficient and scalable structures and patterns.
Implement best practices for data engineering, including performance tuning, security, and data governance.
Collaborate with data analysts, scientists, and other stakeholders to deliver high-quality data solutions.
Monitor, troubleshoot, and optimize data pipelines for performance and reliability.
Ensure data quality, consistency, and integrity across multiple data sources.
Required Skills & Experience:
Proven experience as a Data Engineer with a strong background in data warehousing concepts.
Expert-level proficiency in SQL and Python, with a focus on simplicity and maintainability.
Hands-on experience with AWS services, specifically S3, Lambda, ECR, ECS, and DynamoDB.
Deep understanding of data modeling, including dimensional modeling and best practices for cloud-based data platforms.
Strong problem-solving skills with the ability to optimize queries and improve data performance.
Experience in handling large-scale datasets and implementing scalable data solutions.
Nice to Have:
Experience in document processing.
Experience with utilising AI in the data engineering space.
Experience with Terraform for infrastructure as code.
Knowledge of CI/CD practices for automating deployment processes.
Familiarity with Data Vault modeling for data warehousing.
Experience in building ETL/ELT pipelines using modern data stack tools.
If of interest, please send your CV now to apply!
About the Company
Hello, we are James Adams. Your technology, change, and digital recruitment partner covering the UK & Europe. We are passionate about people and put our customer's experience at the heart of everything we do. We recognise that each individual we work with is different. Which is why we look to provide a bespoke service tailored to your needs. We are experts in recruiting for technology, change and digital talent across the UK & Europe. We cover the following specialisms on a permanent or interim/contract basis. - Cloud ... Know more
Related Jobs
- Company Name
- TechShack
- Job Title
- Data Engineer
- Job Description
- Data Engineer - 12 Month Contract – Fully Remote - Outside IR35 £400-500pd I am currently working with a leading consultancy who are looking for a data engineer for a real time streaming project. TechStack: Data Engineering Data Pipeline Kafka Flink AWS
- Company Name
- Shareforce Ltd
- Job Title
- Data Architect - Outside IR35
- Job Description
- An opportunity for a Microsoft Data Architect to support a rapidly expanding and innovative Microsoft Partner, working at the forefront of the new Microsoft Data and Artificial Intelligence technologies. The role requires you to work closely with the customers stakeholders and AI strategists to architect and lead the delivery of large intelligent data applications and solutions built on Azure, Fabric and Databricks technologies. Responsibilities will include: Leading the design, development and delivery of enterprise data solutions on Microsoft Azure based architectures. Coordinating and delivering discovery sessions at CxO level, building good working relationships to develop a data strategy. Defining the overall technical and data architecture for solutions from a pragmatic standpoint. Leading data governance initiatives and establish data management frameworks. Playing a key role in client project delivery to help see client engagements through to a commercially successful conclusion. Championing knowledge sharing with Engineering team, helping develop standard architecture patterns and being hands on as needed. Required skills & experiences: Business facing skills and a proven, consultative approach including experience of presenting at CxO level. Background experience leading the technical engineering and delivery of client solutions using the Microsoft Azure / Databricks Lakehouse platform. Strong understanding of Data modelling, SQL and NoSQL databases, Microsoft Fabric and Databricks to define best practices and standards. Demonstrable experience of designing Modern Data Platform architectures preferably using the Cloud Adoption Framework (CAF). Architect end-to-end data solutions incorporating batch and real-time processing Appreciation for Data governance principles including Microsoft Purview, Azure infrastructure and networking, Azure DevOps, Machine Learning & AI. Designs with cloud costs and performance in mind with understanding of concepts such as FinOps Knowledge of modern AI services available within the Microsoft and Databricks ecosystems to architect AI frameworks and platforms Knowledge of how to leverage AI to increase developer productivity and quality Microsoft or equivalent certifications are a nice to have: AZ-104 Azure Administrator Associate DP-203 Azure Data Engineering AZ-305 Azure Solutions Architect Databricks – Data Engineer Associate Databricks – Data Engineer Professional Databricks – Champion Additional Information: Day rate: £600-650 IR35: Outside Duration: 6 months (Initial sign up) Location: Remote with willingness to be onsite with clients as needed (1 a week max) Applications are only invited from candidates with the right to reside and work within the UK
- Company Name
- TieTalent
- Job Title
- Azure DevOps Engineer
- Job Description
- About Azure DevOps Engineer Azure DevOps Engineer - Azure, Azure AD, Terraform, SCCM, Intune 6 month contract position Up to £550 p/d Umbrella rate London, Hybrid Join a growing business to manage Microsoft Azure and Intune/SCCM environments, ensuring high availability, automation, and performance. This role involves Infrastructure as Code (IaC) implementation, pipeline management, and continuous optimisation. The client is London based but may require some travel overseas. Key Responsibilities Design & manage Azure DevOps pipelines and automate deployment processes. Maintain and optimise Azure infrastructure, ensuring security best practices. Manage Intune & SCCM for endpoint management. Implement IaC using Terraform and monitor systems via NetData. Troubleshoot DNS, Web Servers (Azure WebApp, Apache TomCat, NGINX, IIS). Develop automation scripts (PowerShell, Perl, SQL) and collaborate across teams.Technical Skills Required: Azure, Azure AD, Terraform, Citrix XenApp, SCCM, Intune Scripting: PowerShell, Perl, SQL Web Servers: Azure WebApp, NGINX, IIS, Apache TomCat Monitoring: NetData Certifications (Desirable): ITIL v4, CompTIA A+, or equivalent experience This contract role offers the chance to work independently and in teams, solving complex technical challenges in an enterprise cloud environment Nice-to-have skills Azure Terraform InTune Powershell Perl SQL NGINX IIS London, England Work experience DevOps IT Consultant Solutions Architect Languages English
- Company Name
- TieTalent
- Job Title
- SAP Test Manager
- Job Description
- About Role: SAP Test Manager Location: Blackburn, BB1 2FA – Office Based Contract: Full-Time / Permanent Salary: £48,000 - £54,995 (Dependant on Experience) Company: EG Group About The Role EG Group are excited to announce a new vacancy in our SAP Centre or Excellence team! EG Group has embarked on digital transformation and technical upgrade journey for the SAP systems in our application landscape. To support this challenging journey, we’re looking for an experienced Application (SAP) Test manager to join our multi-disciplinary Applications (SAP) Center of Excellence (CoE) Team. In this role you will lead the testing efforts for the SAP projects as well as support the business with the Business Process Change Management (BPCM). The ideal candidate has experience with Test automation, preferably using SAP Cloud ALM If this role matches your experience, we would love to hear from you! What You’ll Be Doing Test Management: Develop and execute comprehensive test strategies for SAP implementations, upgrades and changes in line with business requirements and EG project standards Test Planning & Execution: Oversee the planning, execution, and reporting of the full testing lifecycle including functional, integration (SIT) and user acceptance testing (UAT). Test Team Leadership: Lead and mentor a team of testers, ensuring alignment with project timelines, resources, and quality standards. Assign tasks and track progress to ensure timely delivery. Ensure the relevant project management tools are kept up to date Defect Management: Manage and track defects, working with the functional and technical teams to ensure timely resolution. Prioritize defects based on business impact and criticality. Risk Management: Identify, assess, and mitigate risks related to SAP testing and business process changes. Proactively address any issues or delays that may impact project timelines. Business Process Change Management (BPCM): Collaborate with key business and IT stake holders to ensure smooth adoption of new processes and systems, ensuring that testing activities support the broader change initiatives and relevant documentation is updated, created Continuous improvement: Evaluate new technologies and methodologies to drive innovation, optimization and efficiency within the organization. Training & Documentation: Conduct end-user training sessions and develop comprehensive documentation Collaboration: Work closely with Application (SAP) Project manager, cross-functional teams, Business users, Process Owners, Solution Architects and other Functional Consultants to ensure clear communication and alignment Compliance and Governance: Ensure that all SAP testing activities are fully aligned and in compliance with IT governance policies, data security standards, and regulatory requirements. Ensure alignment with Internal and External IT Control frameworks and future application roadmap This list is not exhaustive and may be added to or amended from time to time. What We’re Looking For Bachelor’s degree in Finance, Accounting, IT or related field Essential: Project Management certification (Prince2 or equivalent) Essential: SAP ECC and S4 Hana Experience Desirable: Certifications in SAP are highly desirable Minimum 6 years of experience in full cycle SAP Testing, with at least 3 years in a Test manager role Proven experience in SAP Testing processes including functional integration, system and UAT testing within SAP environments Experience with Test Automation Hands-on experience in aligning testing strategies with business process changes, ensuring smooth transitions and mitigating risks. Demonstrate in-depth knowledge of SAP project lifecycles and SAP implementation Methodologies Knowledge of Data Migration strategies Excellent communication, presentation and interpersonal skills, capable of working collaboratively with multiple teams across disciplinary boundaries. Why Join EG Group Performance Based Bonus Scheme Flexible working hours (8am – 10am start, 8-hour working day) Access To Apprenticeships And Accredited Qualifications Career development and progression opportunities within a global organization. ASDA Discount Card – 10% off all ASDA stores Free Secure Car Parking Waterside Café - freshly prepared meals at affordable prices Dress Down Fridays Prayer and Ablution Facilities Work Anniversary Rewards Free Eye Test Who are EG Group? EG Group is a leading global convenience retailer, operating a wide range of brands across multiple sectors including fuel, foodservice, and grocery retail. With a presence in up to 9 countries and a commitment to innovation and customer service, EG Group continues to expand its portfolio and reach. Our company is focused on delivering value to its customers, partners, and stakeholders through efficient operations and strategic growth. Please note - the successful applicant will be subject to a DBS check which will be funded by EG Group. INDHO Nice-to-have skills Test Automation Blackburn, Scotland Work experience QA Test Automation Languages English