Data engineer
On site
Croydon, United Kingdom
£ 500 / day
Freelance
10-12-2024
Job Specifications
- Sector: Public sector/Government
- Clearance required: SC clearance
- Contract length: 6 months
- Rate: 400-500/d
- IR35 status: Outside
- Onsite requirement: 2.d a week
- Role title: Data engineer
Job Description:
Build streaming data ingestion pipelines using AWS MSK and batch data ingestion pipelines using APIs
Prepare data for prescriptive & predictive modelling.
Experience working with various data formats eg, parquet, avro, json
Experience in building ETL pipelines using Lambda, Step Functions and Glue
Experience working with various AWS database services (eg, RDS, DynamoDB and Redshift)
Technical expertise with various data modelling techniques
Hands-on experience with Python and SQL
Kafka Streaming data experience to build the new data pipelines.
Supporting Skills
5 years' experience in an iterative and Agile development environment
Appreciation of Software Development Environments & Languages
Appreciation of Business Analysis Activities
Appreciation of other Scripting language, eg JavaScript, Lua, Perl, PHP, Unix
About the Company
Sanderson Government & Defence was established over 20 years ago as a bespoke solution to a unique talent problem: sourcing and placing high-calibre professional talent to work within the UK government and defence sector. MISSION We provide talent solutions for the UK government and their critical supply chains, including: Ministry of Defence and areas of National Security; Central Government and Arm’s Length Bodies; National Policing and Justice; and the broader Public Sector. EXPERTISE & SKILL Central to our success has b... Know more
Related Jobs
- Company Name
- Amass Technology
- Job Title
- Data Engineer - HealthTech/Healthcare
- Job Description
- Job Title: Data Engineer with FHIR and Healthcare Experience Contract Type: Outside IR35 Duration: 6 to 12 months (minimum, with potential extension) Location: Remote/Hybrid (to be discussed) About the Role We are looking for a talented and experienced Data Engineer with a strong background in the healthcare or healthtech sectors to join our client on a contract basis. This role involves working with FHIR (Fast Healthcare Interoperability Resources) standards to deliver innovative data solutions that support transformative healthcare projects. This is an excellent opportunity for a contractor seeking flexibility and a role outside IR35, with the chance to make a meaningful impact on cutting-edge healthcare technology initiatives. Key Responsibilities Design, develop, and maintain data pipelines that are scalable, reliable, and efficient, tailored for healthcare applications. Work extensively with FHIR standards, enabling the integration and transformation of healthcare data for interoperability. Collaborate with multidisciplinary teams, including data scientists, developers, and healthcare professionals, to deliver project requirements. Optimise and manage data storage solutions, ensuring integrity and security in line with healthcare regulations. Diagnose and resolve data engineering challenges, ensuring timely delivery of high-quality solutions. Essential Skills & Experience Proven experience as a Data Engineer within the healthcare or healthtech sector. Strong knowledge and practical experience with FHIR standards and frameworks. Expertise in ETL processes, data modelling, and data integration techniques. Proficiency in programming languages such as Python, Scala, or Java. Experience with cloud platforms (eg, AWS, Azure, or GCP) and modern data architectures. Solid understanding of healthcare data privacy regulations, including GDPR and equivalent standards. Desirable Skills Familiarity with healthcare-specific datasets, such as EHR/EMR systems. Experience working with APIs for healthcare data exchange. Exposure to machine learning and analytics in healthcare settings. If interested, please apply or reach out to me
- Company Name
- Henderson Scott
- Job Title
- BI Developer
- Job Description
- BI Developer - Power BI We require an experienced BI Developer to work for our market leading, family run client on a 3 month contract. They are moving from Crystal Reports to Power BI. Responsibilities include but are not limited to - Interpret, maintain and simplify business reporting requirements Implement BI reports and create technical documentation in conjunction with maintaining and simplifying business reporting. Implement and maintain BI tools for integration with business data Excellent Power BI knowledge with Crystal reports being Experience in database operations and optimisation for SQL Server 2016 General support with the wider business This role requires 3-4 days per week on client site in South Oxfordshire
- Company Name
- Freshtechit
- Job Title
- Data Engineer
- Job Description
- Data Engineer- 6 month contract ( remote working) A leading travel company are seeking a Data Engineer to join on a 6 month rolling contract. Data Pipeline Management: Oversee and maintain end-to-end data pipelines, ensuring data integrity and availability for internal and external stakeholders. Reporting & Insights: Develop and manage dashboards, visualizations, and business reporting using tools like Power BI, Tableau, or similar. Conduct ad-hoc data analysis using SQL and other tools. Stakeholder Engagement: Collaborate with internal teams to gather requirements, scope projects, and deliver actionable insights. Self-Service Enablement: Train staff on using self-service reporting tools and support their data needs. Trend Analysis: Identify key trends in data and provide recommendations to drive business value. Issue Resolution: Troubleshoot data pipeline and reporting issues promptly and effectively. As a Data Engineer you will have the following experience: Data visualization and reporting tools (e.g., Power BI, Tableau). Data pipeline development and ETL tools (e.g., DBT, Apache Airflow). Advanced SQL scripting. We are seeking someone with a strong understanding of data warehousing, relational databases, data cleansing, and dimensional modeling. Please note: The contract is to start in January, we are unable to consider long notice periods. Please contact hello@freshtechit.com for more information
- Company Name
- eTeam
- Job Title
- Data Engineer
- Job Description
- Job Title: SC cleared Data Engineer (Denodo ) Clearance required: SC clearance is mandatory Duration: 6 months Location: Telford, UK Contract Inside IR35 Experience in migrating data between Denodo versions (Denodo 7 to Denodo 8). Expertise in Denodo VQL and strong understanding of Logical Data Warehousing (LDW). Problem solving and leadership skills (a support team of engineers will be provided to support the VQL engineer provided who the engineer is expected to lead and support) Ability to refactor and deploy views in Denodo 8 based on existing Denodo 7 configurations. Understanding of HMRC estate and supporting applications such as Git, Jenkins etc. Understanding of Cache and data migrations of cache.