Data Engineer- Energy Sector
Hybrid
London, United Kingdom
£ 65 / year
Full Time
13-03-2025
Job Specifications
Job Title: Data Engineer (Data Flows)
Location: London- 2days/week
Duration: Permanent
Energy Sector Client Experience Required
Job Description
We are seeking a Data Engineer with a strong emphasis on energy industry expertise to join our dynamic team. This role is ideal for candidates who possess deep knowledge of energy sector operations, data flows, and business processes, while being adept at applying this industry insight to support data-driven decision-making. The successful candidate will bridge the gap between technical data management and energy sector strategy, enabling the organization to leverage data as a critical asset.
As a Data Engineer , you’ll be part of an Agile and skilled team of other engineers and analysts working on delivering best in class data solutions for our customers. Working with powerful data processing technologies, you will combine the power of your industry knowledge and industry data flows with the full capabilities of AWS, creating the ultimate data services platform.
Responsibilities
• Work as part of a cross-functional team to design, develop and deploy complex data solutions.
• Provide expert technical advice and recommend solutions to the development resources in the planning, strategizing, and the execution of high profile and complex data processing initiatives.
• Consistently develop with performance in mind to optimize end user experience.
• Work with architects and senior team members to identify new tools and technologies where applicable to expand customer offerings.
• Research and take advantage of new technology to improve and expand solutions.
• Participate in peer code reviews, troubleshoot and correct software defects.
• Work with the internal testing team to ensure appropriate testing is built into the development and ongoing delivery.
• Develop efficient queries to retrieve appropriate datasets from relational databases
• Design, implement and maintain performance of reliable data pipelines and integrations that feed various applications, using both structured and unstructured data
• Create, implement, and maintain data transformation processes
Skills & Knowledge you will need:
• Proven experience in the energy sector with a focus on data-driven projects
• Strong understanding of energy industry concepts, terminology, and key performance indicators.
• Energy Industry Knowledge, specifically around D-Flows their purpose and content.
• Familiarity with data analysis tools and visualization platforms (e.g., Power BI, Tableau) is a plus, but core technical programming skills are secondary to industry knowledge.
• SQL coding skills.
• Ability to translate complex energy data sets into actionable insights for non-technical stakeholders.
• Excellent communication and collaboration skills, with the ability to work effectively across multidisciplinary teams.
• Experience with cloud computing such as Azure or AWS
Nice to have:
• Experience in data modelling.
• Experience of working with large datasets.
• Experience with cloud computing such as Azure or AWS
• SDLC or Product Development experience
• Pentaho Experience
• Knowledge of BI tools such as Tableau
• Python in Data Engineering Experience
• Vertica Experience
• Experience with developing REST APIs and microservices
• Experienced in relational database design
Priyanka Sharma
Senior Delivery Consultant
Office: 02033759240
Email: psharma@vallumassociates.com
About the Company
Vallum Associates offer best in class talent acquisition on a contingency, retained, or project basis. Through our dedicated sector consultants, our specialised brands have the knowledge and connections to provide tailored hiring and project services across industries : * Banking & Financial * Energy, Utilities & Commodities * Engineering & Renewable * Insurance Services Our specialised industry and sector specific consultants are able to offer a personalised experience to fit your needs. Our unique associate consultativ... Know more
Related Jobs
- Company Name
- Workplace Futures Group
- Job Title
- Data Analyst - Graduate Scheme
- Job Description
- We are… At Workplace Futures Group, we provide tenants and office occupiers with a complete package for their relocation or refurbishment needs. Recognised globally for our work, we have received industry accolades in some of the most prestigious UK and international awards. We are a business built on integrity and mutual respect. Hierarchies don’t define us; instead, we empower individuals to deliver their best work. In an industry that often prioritises profit above all else, our values stand apart—happy clients and pride in our craft. Many of our exceptional colleagues have been with us for over a decade, a testament to our culture and commitment to career growth. Our staff turnover is among the lowest in the sector. If you believe your talent is underutilised where you are, and you thrive in an environment that values skill, creativity, and collaboration, we invite you to join us. Your focus will be… We are looking for a highly motivated Graduate Data Analyst to join our team and gain hands-on experience in data analytics, reporting, and dashboard creation across multiple departments. Create and refine Power BI reports and dashboards for various teams. Perform maintenance tasks and administrative duties related to data analytics. Support the IT team in improving reporting processes. Work on dataflows, ensuring efficiency and optimisation. Gain exposure to coding and data processing techniques. Understand and apply testing methodologies in data projects. Collaborate with different departments to refine and enhance data insights. Work with Microsoft Excel, Power Query, and Python for data manipulation and automation. You will bring to the table… Degree in Computer Science, IT Engineering, Data Science, or a related field. Proficiency in Python or SQL, Power Query, and Excel. Microsoft PL-300 certification Strong motivation to learn and develop within a data-driven environment. Internship experience or university projects related to data analytics. Nice to have: Familiarity with HubSpot CRM Understanding of Azure Cloud We will bring to the table… Industry-leading salary Company pension scheme 26 days annual leave + bank holidays Death in Service benefit Season Ticket Loan Cycle to Work Scheme & Electric Car Scheme Regular company social events throughout the year Reward & Recognition scheme Opportunities for CPD and career progression In-house personal development & training Employee mental health & wellbeing support ….and many other company benefits & perks! If you’re passionate about data, eager to learn, and ready to kickstart your career, we’d love to hear from you! Apply Now!
- Company Name
- Focus 5 Recruitment
- Job Title
- Data Engineer
- Job Description
- Data Engineer Warrington - Hybrid £50,000 - £60,000 - Depending on experience Focus 5 Recruitment are working with an exciting software business to help recruit a Data Engineer. The company have just been awarded 2 large contracts with international Mobile Network Operators. Appointed to help them source a Data Engineer, we’re looking for an experienced Data Engineer to design and optimize our client’s data pipelines and storage solutions. This is an amazing opportunity to work with a growing and ambitious software business who have contracts with some of the world’s leading mobile network companies. They are looking for candidates who can come in at a key point in their growth and develop their career as they grow. Key responsibilities for the Data Engineer – Design and build high-performance, low-latency data pipelines capable of processing large volumes of data at high speed. Develop and enhance real-time and batch data processing architectures. Manage both structured and unstructured data, ensuring efficient ingestion, transformation, and storage. Deploy scalable data storage solutions across bare metal and cloud platforms (AWS). Optimize databases, data lakes, and messaging systems for maximum throughput and minimal latency. Collaborate with DevOps and software engineering teams to maintain seamless data integration and flow. Implement monitoring, logging, and alerting systems to track data pipeline performance and integrity. Uphold data security and compliance across all environments. Data Engineer experience we’re looking for - Demonstrated expertise in designing and deploying data architectures for high-velocity, high-throughput systems. Strong proficiency in real-time data streaming technologies such as Kafka, Pulsar, and RabbitMQ. Extensive experience with high-performance databases, including PostgreSQL, ClickHouse, Cassandra, and Redis. In-depth knowledge of ETL/ELT pipelines, data transformation, and storage optimization. Skilled in working with big data frameworks like Spark, Flink, and Druid. Hands-on experience with both bare metal and AWS environments. Strong programming skills in Python, Java, and other relevant languages. Proficiency in containerization technologies (Docker, Kubernetes) and infrastructure as code. Solid understanding of data security, encryption, and compliance best practices. Preferred Qualifications - Experience working with telecom or financial systems. Background in government or defence-sector projects. This is an exclusive role with a key client. For immediate consideration and full details, please submit an application ASAP.
- Company Name
- Kraken
- Job Title
- Data Engineer
- Job Description
- Help us use technology to make a big green dent in the universe! Kraken powers some of the most innovative global developments in energy. We’re a technology company focused on creating a smart, sustainable energy system. From optimising renewable generation, creating a more intelligent grid and enabling utilities to provide excellent customer experiences, our operating system for energy is transforming the industry around the world in a way that benefits everyone. It’s a really exciting time in energy. Help us make a real impact on shaping a better, more sustainable future. Kraken Customer What we do: build the most AI-driven, innovative, forward-thinking platform for energy management. From optimising resources to delivering cost-effective, exceptional customer experiences through advanced Customer Information Systems (CIS), billing, meter data management, CRM, and AI-driven communications, Kraken is powering the next wave of innovation in the energy industry. Why we do it: future energy will not look like energy as we know it today. We need to not just think about our future, but build for it. Now. At Kraken we've developed a data platform that is used by Octopus Energy and our other clients retail energy businesses around the world. The platform empowers users with self-service data analytics and automates our data processing workflows, from simple ETL jobs to ML training and prediction. The data platform team works across the whole customer domain on anything from natural language understanding of our customer communications to processing billions of smart meter readings every day to support clients in creating customised and market leading smart energy tariffs. As the volume, scope and geographical range of our data offerings rapidly expand, we're looking for an experienced data engineer to join the team to help us build and maintain our platform, pipelines and data sources. We're passionate about building great technology to change the way customers use, and think about, energy for the good of the planet. This is a fantastic opportunity to work with us on data problems that genuinely move us closer to Net Zero, and support the energy transition. What You'll Do Build new data sources and data pipelines that deliver key data and insights to the business. Build data pipelines that power and enable the inner workings of the Kraken platform, from billing and settlement to customer facing apps. Build and maintain testing and documentation frameworks for our data sources. Work with the business to scope and deliver new data engineering projects and requirements. Maintain and build on our existing data infrastructure and tools. Support the internationalisation of our data infrastructure as we continue to grow globally. What You'll Have Python, SQL, Spark Experience in assuring data quality Experience deploying data services in a cloud environment (ideally AWS) Experience with APIs designed for high volume traffic serving data to customers or business critical applications. Experience with relational databases (Postgres/Aurora) The projects will be varied and we're looking for someone who can work autonomously and proactively to scope problems and solve and deliver pragmatic solutions. If this sounds like you then we'd love to hear from you. Are you ready for a career with us? We want to ensure you have all the tools and environment you need to unleash your potential. Need any specific accommodations? Whether you require specific accommodations or have a unique preference, let us know, and we'll do what we can to customise your interview process for comfort and maximum magic! Studies have shown that some groups of people, like women, are less likely to apply to a role unless they meet 100% of the job requirements. Whoever you are, if you like one of our jobs, we encourage you to apply as you might just be the candidate we hire. Across Octopus, we're looking for genuinely decent people who are honest and empathetic. Our people are our strongest asset and the unique skills and perspectives people bring to the team are the driving force of our success. As an equal opportunity employer, we do not discriminate on the basis of any protected attribute. Our commitment is to provide equal opportunities, an inclusive work environment, and fairness for everyone.
- Company Name
- Barclay Simpson
- Job Title
- R&D Software and Data Engineer
- Job Description
- PLEASE NOTE - SPONSORSHIP CANNOT BE PROVIDED FOR THIS POSITION R&D Software and Data Engineer Role Overview We are seeking an experienced Senior R&D Software and Data Engineer to play a key role in developing and maintaining data-driven systems. This role involves designing and optimising data pipelines, warehouses, and lakes to ensure data accuracy, accessibility, and security. The successful candidate will contribute to the evolution of digital and analytical capabilities, particularly within the field of cyber fraud prevention and financial crime mitigation. Key Responsibilities Develop and maintain robust data architectures and pipelines to enable seamless data transfer and processing, ensuring durability, consistency, and completeness. Design and implement data warehouses and data lakes that efficiently manage high-volume and high-velocity data while adhering to stringent security standards. Build and optimise data processing and analytical algorithms suitable for handling complex and large-scale datasets. Collaborate with data scientists to develop and deploy machine learning models. Provide guidance on risk mitigation and contribute to governance policies to enhance data security and compliance. Manage risk effectively and implement stronger controls across data engineering projects. Work closely with stakeholders across different areas, ensuring alignment with broader business strategies and objectives. Conduct in-depth analysis of diverse data sources, both internal and external, to develop creative and effective solutions. Communicate complex technical concepts to a wide range of audiences, including senior stakeholders. Influence decision-makers and collaborate cross-functionally to drive impactful data solutions. Key Requirements Experience within financial services, particularly in cyber fraud, financial crime, or security-related roles. Strong understanding of industry fraud and security signals, including digital identity, device authentication, voice recognition, biometrics, and behavioural profiling. Knowledge of cyber fraud attack vectors, such as device spoofing, location manipulation, identity fraud, account takeover, and fraudulent documentation. Proficiency in software development and data analytics tools, such as: Programming languages: Python, PHP, JavaScript, Java Databases: Postgres, MS SQL, Oracle, MySQL, Teradata Data analysis tools: SAS PROC SQL, Hue Database Assistant, Hadoop Familiarity with enterprise security frameworks, such as NIST Cybersecurity Framework and cyber-attack methodologies. Experience with data visualisation and MI tools, including Tableau and Power BI. Strong technical background in computer science, computing, and networks. Desirable Skills Advanced knowledge of cyber fraud tactics and countermeasures within financial services. Experience with tools such as Quantexa, ElasticSearch, SAS and related analytics platforms. Strategic thinking and business acumen, with the ability to drive digital transformation initiatives. Strong understanding of risk and control frameworks within data engineering. Location The successful candidate can work from either Knutsford or Northampton on a hybrid basis.