Lead DevOps / Data Engineer - AWS/Python - Manchester
Hybrid
Manchester, United Kingdom
£ 80,000 / year
Full Time
07-03-2025
Job Specifications
We are looking for Senior Data / DevOps Engineer to join our team and play a crucial role in Data Engineer Team, ensuring the main function is developing, maintaining and improving the end-to-end data pipeline.
You will be working in the Data Engineering team whose main function is developing, maintaining and improving the end-to-end data pipeline that includes real-time data processing; extract, transform, load jobs; artificial intelligence; and data analytics on a complex and large dataset.
Your role will primarily be to perform DevOps, backend and cloud development on the data infrastructure to develop innovative solutions to effectively scale and maintain the data platform. You will be working on complex data problems in a challenging and fun environment, using some of the latest Big Data open-source technologies like Apache Spark, as well as Amazon Web Service technologies including Elastic MapReduce, Athena and Lambda to develop scalable data solutions.
Key Responsibilities:
Adhering to Company Policies and Procedures with respect to Security, Quality and Health & Safety.
Writing application code and tests that conform to standards.
Developing infrastructure automation and scheduling scripts for reliable data processing.
Continually evaluating and contribute towards using cutting-edge tools and technologies to improve the design, architecture and performance of the data platform.
Supporting the production systems running the deployed data software.
Regularly reviewing colleagues’ work and providing helpful feedback.
Working with stakeholders to fully understand requirements.
Be the subject matter expert for the data platform and supporting processes and be able to present to others to knowledge share.
Here’s what we’re looking for:
The ability to problem solve.
Knowledge of AWS or equivalent cloud technologies.
Knowledge of Serverless technologies, frameworks and best practices.
Experience using AWS CloudFormation or Terraform for infrastructure automation.
Knowledge of Scala or OO language such as Java or C#.
SQL or Python development experience.
High-quality coding and testing practices.
Willingness to learn new technologies and methodologies.
Knowledge of agile software development practices including continuous integration, automated testing and working with software engineering requirements and specifications.
Good interpersonal skills, positive attitude, willing to help other members of the team.
Experience debugging and dealing with failures on business-critical systems.
Preferred:
Exposure to Apache Spark, Apache Trino, or another big data processing system.
Knowledge of streaming data principles and best practices.
Understanding of database technologies and standards.
Experience working on large and complex datasets.
Exposure to Data Engineering practices used in Machine Learning training and inference.
Experience using Git, Jenkins and other CI/CD tools.
Mayflower is acting as an Employment Agency in relation to this vacancy
About the Company
MAYFLOWER RECRUITMENT LIMITED is a staffing and recruiting company based out of 89-90 Basingstoke Rd, Reading, BERKSHIRE, United Kingdom. Know more
Related Jobs
- Company Name
- Mondrian Alpha
- Job Title
- Senior DevOps Engineer - Leading Fund - Newcastle
- Job Description
- A highly prestigious, leading European Hedge Fund is seeking an elite senior DevOps engineer to join their team based in the heart of London. This role will give this individual the responsibility of managing and maintaining the continuous integration infrastructure. The successful individual should have prior experience in a buy-side firm and have prior experience as a developer in a language such as C#, Python or Java. My client has committed to paying a strong increase on any base salary and is known for paying market-leading bonuses. This fund also provides only the best benefits, from in-house Michelin star quality food (free for all staff) to the best private healthcare for you and your family. Ideal candidate should have: Prior history as developer or the ability to code Infrastructure As Code – Ansible, Terraform or equivalent Containerisation using Docker, and orchestration via Kubernetes or Docker Swarm Monitoring with ELK, Prometheus, Grafana or equivalent CI/CD with TeamCity Microsoft SQL Server Windows and Linux administration experience Apply now by following the link below or send your resume directly to harvey.gilbert@mondrian-alpha.com
Newcastle Upon Tyne, United Kingdom
On site
Full Time
07-03-2025
- Company Name
- ZILO™
- Job Title
- Cloud Platform Engineer - AI
- Job Description
- Overview: Step forward into the future of technology with ZILO™. We're here to redefine what's possible in technology. While we're trusted by the global Transfer Agency sector, our technology is truly flexible and designed to transform any business at scale. We've created a unified platform that adapts to diverse needs, offering the scalability and reliability legacy systems simply can't match. At ZILO™, our DNA is built on Character, Creativity, and Craftsmanship. We face every challenge with integrity, explore new ideas with a curious mind, and set a high standard in every detail. We are a team of dedicated professionals where everyone, regardless of their role, drives our progress and creates real impact. If you're ready to shape the future, let's talk. About the role: ZILO, a leading technology company in the financial services industry, is seeking a talented Cloud Platform Engineer to join our innovative team. As a Cloud Platform Engineer, you will play a vital role in transforming global transfer agency by leveraging cutting-edge technology solutions. This is an exciting opportunity for individuals who are passionate about creating sustainable value and driving innovation. At ZILO, we are committed to replacing outdated technology and systems that have been in place for over 30 years. Through our design-driven approach and focus on people, we aim to slash costs, reduce risk, and enhance user experience. As a Cloud Platform Engineer, you will contribute to this digital transformation journey by partnering with our customers to modernise and expand their product and service propositions using our single global solution. Joining our highly experienced team of technology, operations, digital, and product experts, you will collaborate to unify the full breadth of global transfer agency into a streamlined platform. With a mission to be the market-leading solution in global transfer agency, ZILO offers a dynamic and innovative work environment where you can make a significant impact. Requirements Bachelor's degree or equivalent in Computer Science, Engineering, or a related field Proven experience as a Cloud Platform Engineer or in a similar role Strong knowledge of AWS cloud Strong knowledge of other cloud platforms such as Microsoft Azure and Google Cloud is a plus Proficiency in scripting languages like Python or Bash Solid understanding of infrastructure-as-code tools like Terraform Experience with containerisation technologies like Docker and Kubernetes Experience with CI/CD tools and processes Excellent problem-solving and communication skills Ability to collaborate effectively in a cross-functional team environment Strong attention to detail and commitment to delivering high-quality results Excellent communication and interpersonal skills Knowledge of industry-standard security frameworks, such as ISO 27001 and NIST Certified Information Systems Security Professional (CISSP) certification is a plus Certified AWS AI Practitioner certification Experience in incident response, risk management and data privacy regulations Strong analytical capability and excellent verbal and written communication skills Experience with cloud AI platforms like AWS Sagemaker Responsibilities: Designing, developing, and maintaining cloud-based solutions Collaborating with cross-functional teams to identify business requirements and translate them into scalable cloud solutions Implementing and managing infrastructure using Amazon Web Services (AWS) or other cloud platforms Ensuring the security, scalability, and reliability of the cloud platform infrastructure Automating deployment, monitoring, and management processes using infrastructure-as-code tools Investigating and resolving technical issues related to the cloud platform infrastructure Staying up-to-date with the latest cloud technologies and industry best practices Benefits Enhanced leave - 38 days inclusive of 8 UK Public Holidays Private Health Care including family cover Life Assurance - 5x salary Flexible working-work from home and/or in our London Office Employee Assistance Program Company Pension (Salary Sacrifice options available) Access to training and development Buy and Sell holiday scheme The opportunity for "work from anywhere/global mobility"
London, United Kingdom
Hybrid
Full Time
07-03-2025
- Company Name
- HM Land Registry
- Job Title
- IT Apprentice - Level 3 - 2 x Apprentice Software Developer & 1 x Apprentice DevOps Engineer
- Job Description
- Plymouth, South West England, PL6 5WS Job Summary It is an exciting time for HM Land Registry (HMLR) as we continue on a major transformation programme. HMLR's ambition is to become the world’s leading land registry for speed, simplicity and an open approach to data. This is your chance to secure a Software Developer or a DevOps Engineer apprenticeship unlike any other. You’ll get all the benefits of an apprenticeship - paid work experience, valuable skills and a recognised qualification. But that’s not all, you will also have the chance to help shape our future. Job Description Through your apprenticeship you will undertake a Level 3 Digital qualification. Upon successful completion of the Level 3 apprenticeship programme and satisfactory performance, conduct and attendance, you will continue on to Level 4 in your particular field and get exposure to traditional methodologies as well as newer agile and digital ones, whilst being supported by your manager and a buddy. On successful completion of your level 3 / 4 apprenticeship, and subject to satisfactory performance and attendance, you will be assessed for a permanent role. We have 2 x Apprentice Software Developer roles and 1 x Apprentice DevOps Engineer role - expected to start in September 2025 As a Apprentice Software Developer, you will contribute to the implementation of software solutions through the production of high-quality code. You will be involved in key stages of the development lifecycle throughout the build and deployment of the software through to providing support during the life of the service. You will bring coding expertise to the team, working alongside others to interpret design and architecture information and to produce quality code that is efficient and maintainable to resolve the problem. The Apprentice DevOps Engineer is part of the IT Operations Practice which sits within Technology & Transformation directorate whose responsibility is for the performance and delivery of all technical infrastructure which, underpins and supports the organisation. It consists of technical teams which provide support, monitor and operate technical solutions. Through your apprenticeship you will undertake a Level 3 Digital qualification. Upon successful completion of the Level 3 apprenticeship programme and satisfactory performance, conduct and attendance, you will continue on to Level 4 in your particular field and get exposure to traditional methodologies as well as newer agile and digital ones, whilst being supported by your manager and a buddy. On successful completion of your level 3 / 4 apprenticeship, and subject to satisfactory performance and attendance, you will be assessed for a permanent role. We have 2 x Apprentice Software Developer roles and 1 x Apprentice DevOps Engineer role - expected to start in September 2025 As a Apprentice Software Developer, you will contribute to the implementation of software solutions through the production of high-quality code. You will be involved in key stages of the development lifecycle throughout the build and deployment of the software through to providing support during the life of the service. You will bring coding expertise to the team, working alongside others to interpret design and architecture information and to produce quality code that is efficient and maintainable to resolve the problem. The Apprentice DevOps Engineer is part of the IT Operations Practice which sits within Technology & Transformation directorate whose responsibility is for the performance and delivery of all technical infrastructure which, underpins and supports the organisation. It consists of technical teams which provide support, monitor and operate technical solutions. Person specification To be successful in this apprenticeship, you’ll need to have at least 5 GCSE's at grade C or grade 4 or above including Maths, English and ICT or a STEM subject. You will understand the commitment needed to complete an apprenticeship and have a motivation for learning. You will be a self-starter able to develop yourself and keep abreast in an everchanging IT environment. You will be a competent user of computer software packages including knowledge of MS Office and email. Through your ability to think ahead, you will be able to anticipate problems and have excellent problem-solving skills. You can work as part of a team to achieve a team goal and can organise and prioritise your own work and tasks. You have effective and easy to understand communication skills (written and verbal) in a variety of situations. Please see the attached candidate pack for further information. Qualifications 5 GCSE's at grade C or grade 4 or above including Maths, English and ICT or STEM subject. Behaviours We'll assess you against these behaviours during the selection process: Developing Self and Others Working Together Alongside your salary of £25,792, HM Land Registry contributes £7,471 towards you being a member of the Civil Service Defined Benefit Pension scheme. Find out what benefits a Civil Service Pension provides. #WeAreHMLR At HM Land Registry our vision is a world leading property market as part of a thriving economy and a sustainable future. We are unique in terms of the economic and social impacts of our work, our ability to make a positive difference to the lives of our customers, our sense of mission, and our great people. They are the foundation of all we have achieved and all we aspire to in the future. We want our people to feel proud to work for HM Land Registry and able to fulfil their full potential. We have a strong and positive culture, a commitment to inclusivity, an emphasis on continuous learning and development, and flexible ways of working. We offer competitive pay and annual leave, attractive pension options and a wide range of other benefits. We have integrity – we value honesty, trust and doing the right thing in the right way. We drive innovation – we are forward-thinking, embrace change and are continually improving our processes. We are professional – we value and grow our knowledge and professional expertise. We give assurance – we guarantee our services and provide confidence to the property market. You can find more information on our rewards package on our website. Selection process details This vacancy is using Success Profiles (opens in a new window), and will assess your Behaviours, Strengths, Ability and Experience. To apply you will need to complete the short application form and take the online Civil Service Verbal Test and Numerical Test. Ideally you should take the tests as soon as possible to allow for the next stage of the application process. Please complete the online application form to include your qualifications and experience and complete the additional questions. What attracted you to working for HM Land Registry? The personal statement section (in no more than 1000 words) should demonstrate evidence of the below criteria: ...
Plymouth, United Kingdom
On site
Full Time
07-03-2025
- Company Name
- CGI
- Job Title
- DevOps Engineer (DV Security Clearance)
- Job Description
- Position Description The Space, Defence and Intelligence business unit in CGI is a true IT Systems Integrator. We work, build, and operate bespoke, technically complex, mission-critical systems which help our clients keep us all safe and secure. We bring innovation to our clients using proven and emerging technologies, agile delivery processes and our deep expertise across the breadth of space, defence, intelligence, aerospace and maritime, all underpinned by our end-to-end cyber capability. We work collaboratively with global technology companies, cutting edge SMEs and academia to deliver the optimal solution for each client. CGI was recognised in the Sunday Times Best Places to Work List 2023 and has been named one of the ‘World’s Best Employers’ by Forbes magazine. We offer a competitive salary, excellent pension, private healthcare, plus a share scheme (3.5% + 3.5% matching) which makes you a member not just an employee. We are committed to inclusivity, building a genuinely diverse community of tech talent and inspiring everyone to pursue careers in our sector, including our Armed Forces, and are proud to hold a Gold Award in recognition of our support of the Armed Forces Corporate Covenant. Join us and you’ll be part of an open, friendly community of experts. We’ll train and support you in taking your career wherever you want it to go. As a DevOps Engineer within the Space, Defence and Intelligence business unit, you will apply your technical expertise to the design, implementation, testing and ongoing support of CGI's tool chain. With strong skills in Terraform and Azure, you will be responsible for the automation of the software development lifecycle. Your future duties and responsibilities Utilise a diverse set of tools such as Azure and Terraform and Jira to automate the software development lifecycle. Participate in Kanban and/or Scrum meetings to plan, prioritise and estimate work. Work with developers and testers to troubleshoot specific issues, including recreating tests, validation configuration, etc. Install, configure and manage both cloud-based and on-premise applications. Required Qualifications To Be Successful In This Role Azure Terraform Continuous Integration Continuous Deployment Docker Linux Amazon Web Services Cloud Desired Competencies: Kubernetes Agile Ansible Security Clearance: Must be a sole UK national – and either hold, or willing to gain higher level clearance. Any individual without security clearance is welcome to apply and will always be considered for this or wider opportunities in CGI, where appropriate. Working pattern: due to the secure nature of the work we are operating with a hybrid working policy however due to the secure nature of some projects with this area we would expect our members to be in their local when required. Some travel to other CGI or client sites might also be required (all travel away from your base office is expensed) Together, as owners, let’s turn meaningful insights into action. Life at CGI is rooted in ownership, teamwork, respect and belonging. Here, you’ll reach your full potential because… You are invited to be an owner from day 1 as we work together to bring our Dream to life. That’s why we call ourselves CGI Partners rather than employees. We benefit from our collective success and actively shape our company’s strategy and direction. Your work creates value. You’ll develop innovative solutions and build relationships with teammates and clients while accessing global capabilities to scale your ideas, embrace new opportunities, and benefit from expansive industry and technology expertise. You’ll shape your career by joining a company built to grow and last. You’ll be supported by leaders who care about your health and well-being and provide you with opportunities to deepen your skills and broaden your horizons. Come join our team—one of the largest IT and business consulting services firms in the world.
Manchester, United Kingdom
On site
Full Time
06-03-2025