Python developer/ bigdata engineer
Hybrid
London, United Kingdom
Freelance
14-03-2025
Job Specifications
Job Title: ETL Developer (Python / Big Data Engineer)
Location: Hybrid (2-3 days in customer office)
Mode of Work: Hybrid work environment, with 2-3 days onsite at the customer office.
Type: Contract (Initial duration 6 months)
Job Description:
We are looking for an experienced ETL Developer with a strong background in Python development and Big Data technologies to join our team. As an ETL Developer, you will be responsible for the design, development, and implementation of data processing pipelines using Python, Spark, and other related technologies to handle large-scale data efficiently. You will also be involved in ensuring the integration of data into cloud environments such as Azure, alongside basic DevOps tasks and RDBMS fundamentals.
Responsibilities:
Develop and maintain ETL pipelines using Python for data extraction, transformation, and loading.
Utilize Apache Spark for big data processing to handle large datasets and optimize performance.
Work with cloud technologies, particularly Azure, to deploy and integrate data solutions.
Implement key Python concepts and leverage libraries/packages like Pandas, NumPy, and others for data manipulation.
Perform data integration tasks involving various data sources and structures.
Collaborate with cross-functional teams to design and implement robust, scalable data solutions.
Apply basic DevOps practices to manage and automate workflows within the ETL process.
Ensure best practices in database management and integration with RDBMS systems.
Participate in troubleshooting, optimization, and performance tuning of data processing systems.
Required Skills and Experience:
Proficient in Python with hands-on experience in key libraries (Pandas, NumPy, etc.) and a deep understanding of Python programming concepts.
Solid experience in Big Data Processing using Apache Spark for large-scale data handling.
Basic DevOps knowledge and familiarity with CI/CD pipelines for automating workflows.
Understanding of Azure Fundamentals and cloud data solutions.
Strong understanding of RDBMS database fundamentals (SQL, relational data modelling, etc.).
Previous experience in ETL development and data integration.
Senior/Lead level experience with hands-on development in relevant technologies.
Excellent problem-solving skills and ability to optimize data workflows.
Additional Desirable Skills:
Familiarity with cloud-based data storage and processing technologies in Azure.
Experience working in Agile or other collaborative development environments.
#ETL #contract #work #London #developer #hybrid
About the Company
Ntrinsic Consulting is a specialist talent business servicing the technology and business support sector across the UK and the Benelux Region. Placing the client at the epicentre of our strategy, we develop tailored solutions catered to unique challenges and realities of the business. We are dedicated to building long-lasting partnerships through bespoke, tailored services that allow us to showcase an understanding of our client’s business and the impact new hires can have on their projects. Ntrinsic Consulting is now part... Know more
Related Jobs
- Company Name
- DataAnnotation
- Job Title
- Mathematician - AI Trainer
- Job Description
- We are looking for a math expert to join our team to train AI models. You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of each model. Please ensure you read the below overview and requirements for this employment opportunity completely. In this role you will need to hold an expert level of mathematical reasoning- a completed or in progress Masters/PhD is preferred but not required. Other related fields include, but are not limited to: Applied Math and/or Computer Science. Benefits This is a full-time or part-time REMOTE position You’ll be able to choose which projects you want to work on You can work on your own schedule Projects are paid hourly starting at $40+ USD per hour, with bonuses on high-quality and high-volume work Responsibilities Give AI chatbots diverse and complex mathematics problems and evaluate their outputs Evaluate the quality produced by AI models for correctness and performance Qualifications Fluency in English (native or bilingual level) Detail-oriented Proficient in mathematics, arithmetic, algebra, geometry, calculus, probability, statistics, and inductive/ deductive reasoning A current, in progress, or completed Masters and/or PhD is preferred but not required Note: Payment is made via PayPal. We will never ask for any money from you. PayPal will handle any currency conversions from USD. #INDUKMA Job Types: Full-time, Part-time Pay: From £31.64 per hour Expected hours: 1 – 40 per week Work Location: Remote
- Company Name
- DataAnnotation
- Job Title
- AI Training, Physics
- Job Description
- We are looking for an advanced physicist to join our team to train AI models. You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of each model. In this role you will need to hold an expert understanding of physics- a completed or in progress Masters/PhD is preferred but not required. Other related fields include, but are not limited to: Astrophysics, Biophysics, Electrical Engineering, Nuclear Engineering, Chemical Engineering, Mathematics. Benefits: This is a full-time or part-time REMOTE position You’ll be able to choose which projects you want to work on You can work on your own schedule Projects are paid hourly starting at $40+ USD per hour, with bonuses on high-quality and high-volume work Responsibilities: Give AI chatbots diverse and complex physics problems and evaluate their outputs Evaluate the quality produced by AI models for correctness and performance Qualifications: Fluency in English (native or bilingual level) Detail-oriented Proficient in mathematics, inductive/deductive reasoning, physical/temporal/ spatial reasoning A current, in progress, or completed Masters and/or PhD is preferred but not required Note: Payment is made via PayPal. We will never ask for any money from you. PayPal will handle any currency conversions from USD. This job is only available to those in the US, UK, Canada, Australia, or New Zealand. Those located outside of these countries will not see work or assessments available on our site at this time.
- Company Name
- DataAnnotation
- Job Title
- Research and Development Mathematician- AI Trainer
- Job Description
- We are looking for a math expert to join our team to train AI models. You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of each model. Please ensure you read the below overview and requirements for this employment opportunity completely. In this role you will need to hold an expert level of mathematical reasoning- a completed or in progress Masters/PhD is preferred but not required. Other related fields include, but are not limited to: Applied Math and/or Computer Science. Benefits This is a full-time or part-time REMOTE position You’ll be able to choose which projects you want to work on You can work on your own schedule Projects are paid hourly starting at $40+ USD per hour, with bonuses on high-quality and high-volume work Responsibilities Give AI chatbots diverse and complex mathematics problems and evaluate their outputs Evaluate the quality produced by AI models for correctness and performance Qualifications Fluency in English (native or bilingual level) Detail-oriented Proficient in mathematics, arithmetic, algebra, geometry, calculus, probability, statistics, and inductive/ deductive reasoning A current, in progress, or completed Masters and/or PhD is preferred but not required Note: Payment is made via PayPal. We will never ask for any money from you. PayPal will handle any currency conversions from USD. #INDUKMA Job Types: Full-time, Part-time Pay: From £31.64 per hour Expected hours: 1 – 40 per week Work Location: Remote
- Company Name
- Shell
- Job Title
- Software Engineer
- Job Description
- Must Haves: Minimum 3 years of professional software development experience Strong expertise in C++ development, with a deep understanding of object-oriented programming, data structures, and algorithms Solid and practical experience in developing robust components for multi-process event-driven systems Experience in back-end development Experience with trading systems, market data, order routing, and systematic trading Pluses: Knowledge of Python Familiarity with connectivity and exchange systems (desirable but not mandatory) Day to Day: Software Engineering is responsible for the delivery of software, which may include new development, configuration and installation of third-party packages, and software integration efforts. It may include tasks from every part of the software development lifecycle, including software and website design, development, programming, and software deployment (both the installation and decommissioning of software). This role is to work on one of the leading software development projects enhancing trading capabilities. The role will involve taking an active part in the development of multiple components, including order placement, routing and management, connectivity to exchanges, pricing, integration with internal systems and other functions. Although the overall programme is a large team, the members working in this specific part of the programme are part of a self-contained team, so this role is ideal for someone looking to be recognized for their individual contributions. It will involve working closely and collaboratively with a high performing team, including the Trading Desk/Business.