(SC cleared) Senior Data Engineer - outside IR35
Hybrid
County Of Herefordshire, United Kingdom
Freelance
20-02-2025
Job Specifications
On-site: near Worcester/Great Malvern/Gloucester
Daily rate: up to £700 outside IR35
Duration: 6months with possible extension
Clearance: Must be SC cleared; DV preferred
We are seeking a seasoned Senior Data Engineer (Infrastructure) to join our team. This role is essential for designing, building, and maintaining sophisticated data infrastructure systems that operate across both on-premises and Azure cloud environments. The position involves deploying and managing scalable data operations that support advanced analytics and data-driven decision-making, crucial for our organisational growth and innovation.
Requirements
Develop and Manage Data Pipelines: You will design, construct, and maintain efficient and reliable data pipelines using Python, capable of supporting both streaming and batch data processing across structured, semi-structured, and unstructured data in on-premises and Azure environments.
Hybrid Cloud and Data Storage Solutions: Implement and manage data storage solutions leveraging both on-premises infrastructure and Azure, ensuring seamless data integration and accessibility across platforms.
Containerisation and Orchestration: Utilise Docker for containerisation and Kubernetes for orchestration, ensuring scalable and efficient deployment of applications across both cloud-based and on-premises environments.
Workflow Automation: Employ tools such as Apache NiFi and Apache Airflow to automate data flows and manage complex workflows within hybrid environments.
Event Streaming Experience: Utilise event-driven technologies such as Kafka, Apache NiFi, and Apache Flink to handle real-time data streams effectively.
Security and Compliance: Manage security setups and access controls, incorporating tools like Keycloak to protect data integrity and comply with legal standards across all data platforms.
Data Search and Analytics: Oversee and enhance Elasticsearch setups for robust data searching and analytics capabilities in mixed infrastructure settings.
Database Management: Administer and optimise PostgreSQL databases, ensuring high performance and availability across diverse deployment scenarios.
Essential Skills and Experience
Strong Python Skills: Expertise in Python for scripting and automating data processes across varied environments.
Experience with ETL/ELT: Demonstrable experience in developing and optimising ETL or ELT workflows, particularly in hybrid (on-premises and Azure) environments.
Expertise in Hybrid Cloud Data Architecture: Profound knowledge of integrating on-premises infrastructure with Azure cloud services.
Containerisation and Orchestration Expertise: Solid experience with Docker and Kubernetes in managing applications across both on-premises and cloud platforms.
Proficiency in Workflow Automation Tools: Practical experience with Apache NiFi and Apache Airflow in hybrid data environments.
Experience in Event Streaming: Proven ability in managing and deploying event streaming platforms like Kafka, Apache NiFi, and Apache Flink.
Data Security Knowledge: Experience with implementing security practices and tools, including Keycloak, across multiple platforms.
Search and Database Management Skills: Strong background in managing Elasticsearch and PostgreSQL in environments that span on-premises and cloud infrastructures.
Your Impact
In this role, you will empower business leaders to make informed decisions by delivering timely, accurate, and actionable data insights from a robust, hybrid infrastructure. Your expertise will drive the seamless integration of on-premises and cloud-based data solutions, enhancing both the flexibility and scalability of our data operations. You will champion the adoption of modern data architectures and tooling, and play a pivotal role in cultivating a data-driven culture within the organisation, mentoring team members, and advancing our engineering practices.
Desirable Skills and Experience
Certifications in Azure and Other Relevant Technologies: Certifications in cloud and on-premises technologies are highly beneficial and will strengthen your application.
Experience in Data Engineering: A minimum of 5 years of experience in data engineering, with significant exposure to managing infrastructure in both on-premises and cloud settings.
This role will require you to have or be willing to go through Security Clearance. As part of the onboarding process candidates will be asked to complete a Baseline Personnel Security Standard; details of the evidence required to apply may be found on the government website Gov.UK. If you are unable to meet this and any associated criteria, then your employment may be delayed, or rejected . Details of this will be discussed with you at interview.
About the Company
We're Methods Analytics. We use data to help public and private sector clients solve complex problems and do good things. We combine passionate people with sector-specific expertise. Our approach is collaborative, creative and human. Our outputs are transparent, robust and transformative. From problem identification, data management and data science, to visualisation, interpretation and the delivery of actionable intelligence – we work across the entire data lifecycle. Know more
Related Jobs
- Company Name
- Lorien
- Job Title
- Oracle ERP Data Migration Consultant
- Job Description
- Oracle ERP Data Migration Consultant Reading, Havant or Thatcham - Hybrid working 3 Month Contract £550 - £600/day inside IR35 Lorien's Utilities client are recruiting for an Oracle ERP Data Migration Consultant to assist with the migration of data, into a new Oracle EBS R12 Operating Unit from an existing one, and the retirement of same data in the source Operating Unit. Required to be working in a team that is being established to support a newly starting project, to define, build and deliver Data Migration developments using an existing DM Framework, and then execute and reconcile the migrations through repeated testing iterations, fine tuning and enhancing the migration programs and processes each time. Key Responsibilities Work with business stakeholders and functional teams to understand the project aims and corresponding Data Migration requirements Use the existing PLSQL framework to build efficient and compliant ETLs for a set of data entities Understand the need for and build effective validation, error handling and reconciliation capabilities. Perform data migration trials to test the written code against the selected datasets, resolve errors and create enhancements where necessary Document data migration and data retirement processes and procedures The successful candidate will have the following skills/knowledge: Demonstrable, recent experience in data migration projects as an Oracle EBS Hybrid consultant The project data migration entities will cover all EBS open transactions and master data but you will be proficient in P2P while also being able to render assistance in other areas, which could be Projects, OTC and Inventory, as well as some Master customer and supplier data. Strong understanding of data migration best practices and methodologies Strong understanding of Oracle EBS data lifecycles so as to understand a data sets' state and what it needs to be completed to its conclusion (retirement). Strong analytical and problem-solving skills Excellent communication and analytical skills Ability to work independently and as part of a team Should this position be of interest please submit your CV and I will be in contact as required. Carbon60, Lorien & SRG - The Impellam Group STEM Portfolio are acting as an Employment Business in relation to this vacancy.
- Company Name
- The Citation Group
- Job Title
- BI & Reporting Analyst
- Job Description
- Job Title: BI & Reporting Analyst Reporting To: Head of BI & Reporting Location: Wilmslow/Remote – Hybrid Hours of Work: 09:00 – 17:30 or in line with business needs – 3 Month fixed term contract. Salary: Dependent on Experience The Role As we move into a period of high demand for the data team we require additional support within the BI & Reporting team to deliver key reports and data requests within this period. The successful person will work closely with the business to deliver key information efficiently and accurately. Responsibilities Creating easy to digest business reporting which help to demonstrate why things are happening as well as what is happening. Automating and improving existing manual reporting. Digging into the data in creative ways, using your abilities to drive actionable insight. Collaborating with key stakeholders to understand requirements and design the most effective solutions. Supporting the team by providing additional requirements for the data warehouse. Combining multiple datasets from different sources to ensure that the business has the full picture on client activity. The Person Experience working in an analytical role in a commercial environment. Strong interpersonal skills and the ability to work independently and take initiative. Highly numerate and analytical. Meticulous attention to detail. Great communication skills enabling you to influence and gather information from a range of stakeholders. You’ll love problem solving and being proactive. Eager to learn, as no two days are ever the same. Experience of using Tableau and SQL (or similar) is essential Desirable skills: Ideally has experience with the tech stack (Tableau, Databricks, Snowflake ). Degree or Masters in a numerate subject such as statistics, mathematics, economics, engineering or similar. Experience with Jira/Confluence. Experience with dbt. About Us We are Citation. We are far from your average service provider. Our colleagues bring their brilliant selves to work every day and we create an environment where they can shine. We are a nice bunch. We don’t do office politics or “that’s not my job”. We listen, support and take ownership. We have been proudly delivering valuable HR and Health and Safety services to SME’s across the UK for over 20 years. Passionate about service, we’re on a mission to revolutionise our colleague’s and client’s experience by employing brilliant people who are experts at what they do and smile whilst they are doing it. Working for Citation you will have access to 25 days holiday, plus your birthday off work, gym membership discount, healthcare, childcare vouchers, the opportunity to purchase extra leave, pension contributions and more. It’s a great place to work because of the people we employ. Fun and professional, we want likeminded individuals who love to love their job (no ‘mood hoovers’ here thanks!) and want the Company to succeed. So, if our culture sounds like a good fit for you and you want to be part of our success story, then send us your details.
- Company Name
- Hamilton Barnes
- Job Title
- Data Scientist - 6-Month Contract (Inside IR35) - Wokingham
- Job Description
- Data Scientist - 6-Month Contract (Inside IR35) - Wokingham An exciting opportunity has arisen for a Data Scientist to join a market-leading utilities company on an initial 6-month contract, with the potential for extension. This role offers a competitive daily rate and falls inside IR35. The successful candidate will play a key role in leveraging data to drive insights and business decisions. The position requires 2-3 days per week onsite in Wokingham, with the remainder worked remotely. Key Responsibilities: Analyse and pre-process large-scale time series datasets, identifying relevant variables and addressing data cleansing requirements. Develop and implement advanced statistical models and machine learning algorithms to conduct predictive analysis on time series data. Build and validate forecasting models to predict future trends, patterns, and anomalies, supporting business decision-making. Collaborate with cross-functional teams to understand business challenges and translate them into actionable data science projects. Communicate findings and insights effectively to stakeholders, presenting complex concepts in a clear and concise manner. What you will ideally bring: Proven experience in time series analysis and forecasting, preferably in a commercial or industrial setting. Strong proficiency in Python, including libraries such as Pandas, NumPy, and scikit-learn for data manipulation and modelling. Experience with machine learning algorithms and their application to real-world data problems. Strong understanding of data science and large language model (LLM) use cases, as well as data engineering principles. Experience working with Graph Databases (Graph DB) and Vector Databases (Vector DB). Proficiency in data visualisation techniques to effectively communicate insights from time series data. Strong problem-solving skills and the ability to work with complex and unstructured data. Experience working with cloud platforms and distributed computing frameworks, such as AWS, Azure, or Google Cloud. (Preferred) Familiarity with big data technologies, such as Apache Spark or Hadoop.(Preferred) Contract Details: Duration: 6 Months Location: Wokingham 2-3 Days Per Week/Remote Daily Rate: Up to £500 Per Day (Inside IR35) Data Scientist - 6-Month Contract (Inside IR35) - Wokingham
- Company Name
- Bangura Solutions
- Job Title
- Data Solutions Architect - Data Mesh, Azure
- Job Description
- Data Solutions Architect - Data Mesh, Azure Proven Data Solution Architect with strong recent Data Mesh project experience required to work on a complex large scale transformation program with a financial services environment. The role will involve creating a technical delivery plan for Data Architecture activities and building the solution which will include end to end documentation, functional and non-functional data requirements. You will support on implementation/migration activities for the data solution create. The ideal candidate will have worked within Azure environment and will have recently work another large scale Data Mesh programme as a Senior Data Solutions Architect. Minorities, women, LGBTQ+ candidates, and individuals with disabilities are encouraged to apply. Interviews will take place next week, so please apply immediately to be considered for this exciting career opportunity or call Bangura Solutions to discuss this role further.