Senior Data Engineer
On site
London, United Kingdom
Full Time
17-03-2025
Job Specifications
Full-time
Contract Type: Permanent
Location: Remote - UK
Division: Technology & Product
Salary Currency: GBP Sterling
Department: Data & Analytics
Company Description
We are looking to hire a Senior Data Engineer who will play a key role in supporting the existing HiveAlpha article MI product, and delivering Trenflow (a green field product that will ultimately replace HiveAlpha and support further key use cases).
Key responsibilities will include creating new data ingestion processes, monitoring existing processes and building out the ingestion frameworks to exceptional quality standards.
HiveAlpha and Trenflow are built using AWS services including OpenSearch, DynamoDB, Lambdas, Kinesis, ECS, AppSync.
The Senior Data Engineer reports to the HiveAlpha / Trenflow Tech Lead, manages ZERO people and works closely with a Tech Lead, a Product Manager, a Business Analyst, and Frontend Engineers.
Role Location
The role is fully remote, and so we are therefore open to candidates across the UK.
So, who are we?
We are Reach Plc, you might not have heard of us but we have probably met without you even realising it. We are home to the UK and Ireland’s most iconic digital platforms, magazines and newspapers. From the Mirror, Daily Express and OK!, to our regional titles such as the Liverpool Echo, BirminghamLive and the Manchester Evening News (plus lots more), our brands and the stories we cover are as varied as our people. We help all kinds of people share experiences through our stories because we believe this is what makes us human.
Oh, and we should probably also mention that with a monthly readership of 47 million people, Reach is the largest commercial publisher in the UK. At Reach, not only will you feel better understood, you will also feel you understand other people better too.
Job Description
In this role, you will develop and manage data ingestion processes using Python and NodeJS, requiring you to adapt to various programming environments and technologies.
Your key responsibilities include sourcing data from systems like APIs, S3 buckets, Databases, and Streams, ensuring seamless integration into the data infrastructure.
You will utilise Infrastructure-as-Code (IAC) tools for AWS provisioning, which is essential for efficient cloud resource management and automation to enhance scalability and reliability.
Monitoring the data platform is crucial for you to maintain system integrity, with a focus on implementing observability mechanisms for quick issue resolution.
Collaborating with business analysts, product managers, and stakeholders will ensure that you achieve accurate data ingestion aligned with organisational goals.
Documenting processes and design decisions is vital for maintaining clarity and consistency, aiding in future developments.
Upholding Reach's diversity and inclusion policy is paramount in your role, ensuring all content reflects non-discrimination values and promotes an inclusive work environment.
Qualifications
Ideal candidate will have the following skills;
Advanced knowledge of Python
Strong knowledge of AWS core and data services, particularly OpenSearch, S3, Glue, Lambda, ECS, and DynamoDB
Excellent knowledge of SQL, including DML and DDL
Excellent understanding of unit testing for data processes
Passionate about data and its potential opportunities
Articulate communicator who enjoys fostering collaboration within an organisation and between teams
Able to work independently as well as part of a team
Experience with end-to-end solutions, including web applications, APIs, backend, and frontend technologies, is highly desirable
Additional Information
We are offering a competitive salary and benefits including;
Help saving for your retirement - join our pension scheme and we’ll match contributions up to 6%
25 days’ holiday per year when you join. After three, five and ten years’ service with Reach, you’ll accrue one additional day of annual leave. We also have a Holiday Purchase Scheme
Giving back day - an additional day’s leave each year to support a cause close to your heart
Enhanced family leave - including maternity, paternity and adoption - and you’ll be eligible after just six months’ employment
Private Healthcare Cash Plan - free health cash plan so you can claim back cash for a range of medical expenses
As one of the UK’s top 50 Inclusive Employers everyone’s invited to join our events, programmes and networks to celebrate diversity and build an environment where all our people can thrive
Wellbeing support - including a 24/7 employee assistance programme (EAP) for you and your nearest and dearest
Awesome career opportunities - you’ll have support from your line manager and a range of learning & development programmes
We are also a member of the 30% Club, a global campaign aimed at increasing gender diversity at board and senior management levels
We reserve the right to close this vacancy early if we receive sufficient applications for the role. Therefore, if you are interested please apply early to avoid disappointment.
Reach plc believes diversity brings benefits for our customers, our business and our people. This is why we are committed to being an inclusive employer and encourage applications from all suitable applicants irrespective of background, circumstances, age, disability, gender identity, ethnicity, religion or belief and sexual orientation.
I'm interested
I'm interested
Privacy Policy
About the Company
Hull Live is the leading news, information and sport platform serving Hull and East Yorkshire and one of the largest regional news sites in the country. With over 2.8 million monthly visitors to hulldailymail.co.uk and over 60,000 people reading our daily print title the Hull Daily Mail each week, we have the largest audience for businesses to market their local product or service. Over the past few years media and consumer habits have evolved and the search and purchase process has become more complex. At Hull Live and the ... Know more
Related Jobs
- Company Name
- Instructure
- Job Title
- Senior AI Platform Engineer (remote)
- Job Description
- At Instructure, we empower teachers and learners to grow and succeed by creating intuitive products that simplify learning, facilitate meaningful relationships, and inspire innovation. We are looking for AI Platform Engineers to build and maintain core LLM-powered services and APIs to power the next generation of online learning tools. On this team, you’ll architect and implement services and infrastructure that will allow our feature teams to rapidly develop and deploy LLM-powered features that improve the teaching and learning experience of tens of millions of people around the world while maintaining our high standards for performance, security, and reliability. What You'll Do Design and implement APIs and domain services that abstract the complexity of using an LLM and enable rapid feature development from our product teams Build scalable AWS infrastructure for LLM interactions, including prompt management, context handling, auditing, and response processing Create developer tooling and documentation that makes integration with your services intuitive and easy Establish best practices for AI and LLM use at Instructure, considering things like auditability, cost, and performance Collaborate with feature teams to understand their needs and solve them with platform capabilities Design, develop, and maintain scalable and performant APIs Write clean, maintainable, and well-documented code Participate in code reviews and provide constructive feedback to other developers What You'll Need Prior software engineering experience with API design and distributed system development Strong experience building backend services and REST or GraphQL APIs Knowledge of API design best practices around security, performance monitoring, rate limiting, and more Understanding of LLM integration and usage patterns Strong collaboration and written and verbal communication skills; Past experience with cloud infrastructure, preferably on AWS Experience with Node.js or a desire to learn Experience with TypeScript or a desire to learn Background in developer tooling and API documentation is preferred Why Join Us At Instructure, you’ll lay the foundation for the use of AI in education and make a difference in the lives of millions of learners. We invest in our employees’ learning with thoughtful mentorship, hack weeks, internal conferences, and a culture where innovation and ownership thrive. We’ve always believed in hiring the most awesome people and treating them right. We know that the more diverse we are, the more diverse our ideas will be and when we openly welcome those ideas, our environment is better and our business is stronger. All Instructure employees are required to successfully pass a background check upon being hired.
- Company Name
- Amici Procurement Solutions
- Job Title
- Senior Software Engineer - Data
- Job Description
- GLASGOW (HYBRID, 1 DAY PER WEEK IN OFFICE) Eden Scott is recruiting a Senior Data Software Engineer for Amici to develop a cutting-edge platform. With significant growth and an ambitious technology roadmap, Amici seeks an engineer skilled in Java, Python, and data to shape the future of the MyAmici platform. Why Join Us? You'll work in an agile, collaborative environment, leveraging modern technology stacks to build and optimize a powerful data platform and search engine. With an opportunity to explore vector search, machine learning, and large-scale data processing using Apache Lucene, Solr, or Elasticsearch. The position is hybrid, with one day per week in the Glasgow office. What You’ll Do: Design, build, and optimize a high-performance data platform and search solution. Develop robust search capabilities using Apache Lucene, Solr, or Elasticsearch. Engineer scalable data pipelines in Java or Python. Write high-quality, test-driven code using Agile methodologies. Collaborate with Business Analysts, Data Engineers, and UI Developers. Work across the full stack, from React/TypeScript front-end to Java-based search services. Leverage cloud technologies like Azure Data Factory, Batch Services, Azure SQL. Contribute to DevOps practices, code reviews, and system optimizations. What We’re Looking For: Strong experience in Java development and exposure to Python. Experience with large-scale data processing and search technologies. Expertise in Apache Lucene, Solr, Elasticsearch, or willingness to learn. Hands-on experience with SQL and NoSQL databases. Exposure to modern JavaScript frameworks like ReactJS or VueJS. Experience in Agile environments with modern DevOps and CI/CD practices. A degree in Computer Science/Software Engineering or equivalent experience. Familiarity with writing automated tests and maintaining high code quality. About Amici: Founded in 2005, Amici provides a cloud-based purchasing and inventory management platform for biotech and life sciences organizations. The MyAmici platform supports scientists in their research by handling supply chain and procurement needs. The Innovation Team ensures MyAmici remains at the forefront of technology. What’s In It for You? Work in an intrapreneurial and innovative environment. A flexible, hybrid working approach. A company culture valuing growth, collaboration, and continuous improvement. A fantastic suite of benefits. Join us to be part of a high-impact team transforming the biotech industry. Interested? Let’s talk! Contact our recruitment partners at Eden Scott for an informal discussion: amici@edenscott.com
- Company Name
- Sainsbury's
- Job Title
- Data Engineer - SQL, Python and AWS/Azure
- Job Description
- We’d all like amazing work to do, and real work-life balance. That’s waiting for you at Sainsbury’s. Think about the scale it takes for us to feed the nation. The level of data, transactions and variety it involves. Then you’ll realise that ours is a modern software engineering environment because it has to be. We’ve made serious investment into a Tech Academy and into setting standards and principles. We iterate, learn, experiment and push ways of working such as Agile, Scrum and XP. So you can look forward to awesome opportunities in everything from AI to reusable tech. You’ll be part of one of our Engineering squads working on Ad-tech, interpreting and following architectural and engineering principles, operating frameworks, and new and improved technology and solutions. With your technical craft, curiosity and experimentation, you’ll use judgement to apply specific techniques to deliver focused outcomes that support our customers. We use a variety of tech here and you need to be willing to pick up new tools and techniques. We’d also like you to be yourself - analytical, curious and unafraid to challenge the status quo. You’ll constantly strive to improve the way things are done in your team, contributing as a progressive member of the engineering community. We are looking for an experienced Data Engineer confident in investigating, ingesting and exploiting large datasets with core skills in Python, Pyspark, EMR, AWS, SQL, CI/CD. At Sainsburys Tech, within the Media Agency, we’re unlocking petabytes of untapped potential. We have thousands of shops and millions of customers creating billions of transactions per year. We’re on a mission to use this data better than anyone else. What to expect as a Data Engineer: Work with various data squads at Sainsbury’s to ensure the right data is available for teams to build fantastic products. The aim of our products is to increase efficiency, speed of data ingestion into our reporting platform, data quality and ensure data security and compliance We are product focused and and are looking for engineers interested in data engineering and building brilliant data products from start to finish Writing code, we love writing beautiful code Big on automation including deployments and testing (CICD) Building out Monitoring and Alerting tools for live applications Working as a team during collaborative design hubs Sometimes going down a rabbit hole while exploring new data Jumping on a huddle with the team to discuss a tricky problem Pair programming to diagnose an elusive bug Jumping on a Pull Request to provide a constructive brain dump Building relationships with your colleagues – mentoring them, and teaching things you’ve learned Partnering with Data Modellers, Analytics Engineers, Data Scientists, Architects and Product Managers to deliver world-class data products We’d Love To Hear From You If: You’re interested in the world of Data, and have previous experience as a Data Engineer, Backend Software Engineer or similar domain You have experience building scalable, robust, and well-tested Software products with good coding practices, You’re curious and ask questions about technologies and use initiative to figure things out You like having ownership of your work and seeing it through You’ve got a point of view but are prepared to reconsider your opinion. Experience with these beneficial: Python AWS / Azure / GCP SQL Snowflake Terraform We are committed to being a truly inclusive retailer, so you’ll be welcomed whoever you are and wherever you work. Around here, there’s always the chance to try something new - whether that’s as part of an evolving team or somewhere else across the business - and we take development seriously and promise to support you. We also recognise and celebrate colleagues when they go the extra mile and, where possible, offer flexible working. When you join our team, we’ll also offer you an amazing range of benefits. Here are some of them: Starting off with colleague discount, you'll be able to get 10% off at Sainsbury's, Argos, TU and Habitat after 4 weeks. This increases to 15% off at Sainsbury’s every Friday and Saturday and 15% off at Argos every pay day. We've also got you covered for your future with our pensions scheme and life cover. You'll also be able to share in our success as you may be eligible for a performance-related bonus of up to 10% of salary, depending on how we perform. Your wellbeing is important to us too. You'll receive an annual holiday allowance, and you can buy additional holiday. We also offer other benefits that will help your money go further such as season ticket loans, cycle to work scheme, health cash plans, pay advance (where you can access some of your pay before pay day) as well access to a great range of discounts from hundreds of other retailers. And if you ever need it there is also an employee assistance programme. Moments that matter are as important to us as they are to you which is why we give up to 26 weeks’ pay for maternity or adoption leave and up to 4 weeks’ pay for paternity leave. Please see www.sainsburys.jobs for a range of our benefits (note, length of service and eligibility criteria may apply).
- Company Name
- Microsoft
- Job Title
- Member of Technical Staff, AI Pretraining
- Job Description
- Help deliver one of the best foundational models in the world at Microsoft AI. At Microsoft AI, we are on a mission to train the world’s most capable AI frontier models, pushing the boundaries of scale, performance and product deployment. The Pre-Training team at Microsoft AI tackles some of the most challenging problems in deep learning at scale. As a team, we will deliver one of the best foundation models in the world, forming the foundation of many initiatives across Microsoft AI. About We are looking for outstanding individuals excited about contributing to the next generation of systems that will transform the field. We are looking for candidates who: Have proven expertise in areas of interest, evidenced by an exceptional publication track record and significant technical leadership in high-impact projects Exhibit strong analytical skills, attention to detail, and a commitment to data-driven decision-making Have experience and/or in-depth understandings about large-scale distributed systems Demonstrate an ability to work collaboratively in a fast-paced, innovative environment Responsibilities Develop algorithms, model architectures, data mixtures, and scaling laws for large-scale training using a rigorous data-driven approach grounded in meticulous ablations Drive algorithmic implementations, conduct experiments, and oversee flagship training runs on our in-house large-scale distributed stack Collaborate closely with teams on infrastructure, data, post-training, and multimodality Embody our culture and values. Required/Minimum Qualifications Bachelor's Degree in Computer Science, or related technical discipline AND technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python Proven expertise in the area of pretraining Additional Or Preferred Qualifications Demonstrated experience in large-scale AI. Passionate about conversational AI and its deployment. Demonstrated written and verbal communication skills with the ability to work closely with cross-functional teams, including product managers, designers, and other engineers. Passion for learning new technologies and staying up to date with industry trends, best practices, and emerging technologies in AI. Proven ability to collaborate and contribute to a positive, inclusive work environment, fostering knowledge sharing and growth within the team.