Python developer/ bigdata engineer
Hybrid
London, United Kingdom
Freelance
14-03-2025
Job Specifications
Job Title: ETL Developer (Python / Big Data Engineer)
Location: Hybrid (2-3 days in customer office)
Mode of Work: Hybrid work environment, with 2-3 days onsite at the customer office.
Type: Contract (Initial duration 6 months)
Job Description:
We are looking for an experienced ETL Developer with a strong background in Python development and Big Data technologies to join our team. As an ETL Developer, you will be responsible for the design, development, and implementation of data processing pipelines using Python, Spark, and other related technologies to handle large-scale data efficiently. You will also be involved in ensuring the integration of data into cloud environments such as Azure, alongside basic DevOps tasks and RDBMS fundamentals.
Responsibilities:
Develop and maintain ETL pipelines using Python for data extraction, transformation, and loading.
Utilize Apache Spark for big data processing to handle large datasets and optimize performance.
Work with cloud technologies, particularly Azure, to deploy and integrate data solutions.
Implement key Python concepts and leverage libraries/packages like Pandas, NumPy, and others for data manipulation.
Perform data integration tasks involving various data sources and structures.
Collaborate with cross-functional teams to design and implement robust, scalable data solutions.
Apply basic DevOps practices to manage and automate workflows within the ETL process.
Ensure best practices in database management and integration with RDBMS systems.
Participate in troubleshooting, optimization, and performance tuning of data processing systems.
Required Skills and Experience:
Proficient in Python with hands-on experience in key libraries (Pandas, NumPy, etc.) and a deep understanding of Python programming concepts.
Solid experience in Big Data Processing using Apache Spark for large-scale data handling.
Basic DevOps knowledge and familiarity with CI/CD pipelines for automating workflows.
Understanding of Azure Fundamentals and cloud data solutions.
Strong understanding of RDBMS database fundamentals (SQL, relational data modelling, etc.).
Previous experience in ETL development and data integration.
Senior/Lead level experience with hands-on development in relevant technologies.
Excellent problem-solving skills and ability to optimize data workflows.
Additional Desirable Skills:
Familiarity with cloud-based data storage and processing technologies in Azure.
Experience working in Agile or other collaborative development environments.
#ETL #contract #work #London #developer #hybrid
About the Company
Ntrinsic Consulting is a specialist talent business servicing the technology and business support sector across the UK and the Benelux Region. Placing the client at the epicentre of our strategy, we develop tailored solutions catered to unique challenges and realities of the business. We are dedicated to building long-lasting partnerships through bespoke, tailored services that allow us to showcase an understanding of our client’s business and the impact new hires can have on their projects. Ntrinsic Consulting is now part... Know more
Related Jobs
- Company Name
- Square One Resources
- Job Title
- Remedy Developer (ARS v8)
- Job Description
- Job Title: Remedy Developer (ARS v8) - Hybrid/UK Location: Ideally Reading office once or twice a week - happy to consider other UK locations Salary/Rate: Up to £589 a day Inside IR35 Start Date: March / April Job Type: 16 Week Contract (with scope to extend) Company Introduction We are looking for a Remedy Developer to join our client in the Telecommunication industry. **Must have Remedy ARS V8 experience for this role** You will contribute to the Design, Development, Enhancement & Support of hybrid (bespoke & vendor) Service Management IT solutions and provides subject matter expertise to enable others in the team to meet delivery needs & challenges. You will ensure the Service Management technical environment(s), processes, tools, capability and capacity are fit for purpose and shape/influence the operating model to improve the delivery service provided (costs, timescales & quality). The successful candidate for this role must be able to travel to their closest office once or twice a week, as well as the Reading office occasionally for meetings. Knowledge & Experience: Excellent verbal & written experience of the English language. Extensive recent knowledge and practical experience of one or more of the following software products:- BMC Remedy ARS (on premise) BMC Atrium Integrator Awareness & understanding of multiple software development/integration technologies (see Technical Skills & Abilities). Proven use of different Development/Delivery methodologies (e.g. Waterfall, Agile…). Ability to effectively communicate with all levels to ensure mutual and shared understanding. Builds and maintains strong internal and external relationships through mutual understanding. Excellent trouble-shooting and problem solving skills Experience of analysing, impact assessing, planning and resolving complex technology requirements, problems and issues. Technical Skills & Abilities: Customisation, Development and Integration skills, that can be evidenced/showcased, in BMC Remedy (ARS V8 - as "greenfield" bespoke dev). Ability to integrate Remedy and 3rd party tools (on-prem). Experience on rest API, Atrium Integrator configuration and Remedy workflow development. Experience of BMC CMDB and the ability to troubleshoot issues related to data normalization, reconciliation Excellent SQL skills across industry RDBMS technologies (Oracle, MS SQLServer..) Awareness/Experience across industry RDBMS development languages (stored procedures/scripting) Proficient Windows/Unix skills (OS cmd line/Shell Scripting/Services). Recent experience in source code/version control management & delivery collaborative tools (e.g. Jira, Subversion, Microsoft Team Foundation Server…). Understands, explains and presents complex technical ideas or issues to both technical and non-technical audiences at all levels in a persuasive and convincing manner. Authentic, transparent and considerate. Works hard to create a Winning Team. Displays Energy and passion to achieve and exceed stretching objectives often delivering within tight timescales Job Responsibilities/Objectives Participates & Supports in impact assessment and analysis of hybrid Service Management platform solutions through direct fault fix or subject matter expertise to other support partners. Participates and Supports in the Quality Assurance of solution delivery (Unit, System, Integration, User Acceptance, Operational Readiness and Post Implementation test cycles). Data Configuration, Customisation and Development of bespoke Remedy ARS platform and Atrium CMDB & Integrator components. Support the production of delivery Impact Assessments, for software development, against business requirements, to allow timescales, costs and complexity to be socialized and agreed before any delivery engagement. Production of Detailed/Functional Design of hybrid Service Management platform solutions, based on business requirements and in line with architecture steer and security & compliance obligations. Peer review of code, configuration and documentation, ensuring it meets industry and organisational standards. Maintains and improves relationships with key customers, peers in other functions/operating countries and key vendor partners. Ensures that system development is implemented in a secure, compliant (i.e. CAS-T, IL3 and PSN obligations), coherent and consistent manner and that planned systems developments are compatible with the organisation's existing systems, infrastructure and strategic plans. If you are interested in this opportunity, please apply now with your updated CV in Microsoft Word/PDF format. Disclaimer Notwithstanding any guidelines given to level of experience sought, we will consider candidates from outside this range if they can demonstrate the necessary competencies. Square One is acting as both an employment agency and an employment business, and is an equal opportunities recruitment business. Square One embraces diversity and will treat everyone equally. Please see our website for our full diversity statement.
- Company Name
- DataAnnotation
- Job Title
- Math Tutor - AI Trainer
- Job Description
- We are looking for a *math tutor* to join our team to train AI models. You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of each model. Maximise your chances of a successful application to this job by ensuring your CV and skills are a good match. In this role you will need to hold an expert level of mathematical reasoning- a completed or in progress Masters/PhD is preferred but not required. Other related fields include, but are not limited to: Applied Math and/or Computer Science. Benefits This is a full-time or part-time REMOTE position You’ll be able to choose which projects you want to work on You can work on your own schedule Projects are paid hourly starting at $40+ USD per hour, with bonuses on high-quality and high-volume work Responsibilities Give AI chatbots diverse and complex mathematics problems and evaluate their outputs Evaluate the quality produced by AI models for correctness and performance Qualifications Fluency in English (native or bilingual level) Detail-oriented Proficient in mathematics, arithmetic, algebra, geometry, calculus, probability, statistics, and inductive/ deductive reasoning A current, in progress, or completed Masters and/or PhD is preferred but not required Note: Payment is made via PayPal. We will never ask for any money from you. PayPal will handle any currency conversions from USD. This job is only available to those in the US, UK, Canada, Australia, or New Zealand. Those located outside of these countries will not see work or assessments available on our site at this time. #INDUKMA Job Types: Full-time, Part-time Pay: From £31.64 per hour Expected hours: 1 – 40 per week Work Location: On the road
- Company Name
- DataAnnotation
- Job Title
- Mathematician - AI Trainer
- Job Description
- We are looking for a math expert to join our team to train AI models. You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of each model. Please ensure you read the below overview and requirements for this employment opportunity completely. In this role you will need to hold an expert level of mathematical reasoning- a completed or in progress Masters/PhD is preferred but not required. Other related fields include, but are not limited to: Applied Math and/or Computer Science. Benefits This is a full-time or part-time REMOTE position You’ll be able to choose which projects you want to work on You can work on your own schedule Projects are paid hourly starting at $40+ USD per hour, with bonuses on high-quality and high-volume work Responsibilities Give AI chatbots diverse and complex mathematics problems and evaluate their outputs Evaluate the quality produced by AI models for correctness and performance Qualifications Fluency in English (native or bilingual level) Detail-oriented Proficient in mathematics, arithmetic, algebra, geometry, calculus, probability, statistics, and inductive/ deductive reasoning A current, in progress, or completed Masters and/or PhD is preferred but not required Note: Payment is made via PayPal. We will never ask for any money from you. PayPal will handle any currency conversions from USD. #INDUKMA Job Types: Full-time, Part-time Pay: From £31.64 per hour Expected hours: 1 – 40 per week Work Location: Remote
- Company Name
- DataAnnotation
- Job Title
- AI Training, Physics
- Job Description
- We are looking for an advanced physicist to join our team to train AI models. You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of each model. In this role you will need to hold an expert understanding of physics- a completed or in progress Masters/PhD is preferred but not required. Other related fields include, but are not limited to: Astrophysics, Biophysics, Electrical Engineering, Nuclear Engineering, Chemical Engineering, Mathematics. Benefits: This is a full-time or part-time REMOTE position You’ll be able to choose which projects you want to work on You can work on your own schedule Projects are paid hourly starting at $40+ USD per hour, with bonuses on high-quality and high-volume work Responsibilities: Give AI chatbots diverse and complex physics problems and evaluate their outputs Evaluate the quality produced by AI models for correctness and performance Qualifications: Fluency in English (native or bilingual level) Detail-oriented Proficient in mathematics, inductive/deductive reasoning, physical/temporal/ spatial reasoning A current, in progress, or completed Masters and/or PhD is preferred but not required Note: Payment is made via PayPal. We will never ask for any money from you. PayPal will handle any currency conversions from USD. This job is only available to those in the US, UK, Canada, Australia, or New Zealand. Those located outside of these countries will not see work or assessments available on our site at this time.