DATA ENGINEERING
FOCUS
Specialist Focus Areas
At Harnham, we specialize in the following key areas within Data Engineering:
-
- Analytics Engineering: Bridging the gap between data engineering and data science to enable sophisticated analytics.
-
- Cloud Engineering: Developing and managing scalable cloud-based data solutions.
-
- Data & DevOps: Integrating data management with DevOps practices to streamline workflows and enhance productivity.
-
- Data Architecture: Designing and implementing the overall data framework and architecture for your organization.
-
- Data Engineering & Big Data: Handling large-scale data processing and building data pipelines.
-
- Data Platform Engineering: Creating and managing data platforms that support analytics and data science operations.
-
- Data Product Management: Overseeing the development and management of data products.
-
- DevOps Engineering: Combining software development and IT operations to improve deployment and efficiency.
-
- Platform Engineering: Building and maintaining the platforms that support data operations.
-
- Software Engineering: Developing software solutions that enhance data engineering capabilities.
JOBS
LATEST Data engineering
OPPORTUNITIES
With over 17 years of experience, Harnham has established itself as the leading global authority in Data and AI Recruitment
Data Engineer
London
£700 - £800
+ Data Engineering
ContractLondon
To Apply for this Job Click Here
Data Engineer – Contract
Duration: 6 months
Location: Central London – 2/3 days per week in the office.
Rate: £750 Inside IR35
I’m partnering with a technology-led organisation to hire a Senior Data Engineer with strong cloud and Python experience and hands-on exposure to PyTorch-based ML workloads. This role sits firmly in data engineering rather than data science or pure ML engineering and will suit someone who has supported ML pipelines and datasets in production cloud environments.
You’ll work on large-scale, cloud-based data systems, managing high volumes of unstructured data and enabling machine learning workflows to run reliably and efficiently. The focus is on robust engineering, not experimentation.
Key Responsibilities
* Design, build, and maintain scalable cloud data pipelines supporting ML workloads
* Manage large volumes of unstructured data using cloud object storage (GCS / S3)
* Support PyTorch-based data loading and dataset management in production environments
* Work closely with ML practitioners to enable training and inference pipelines
* Ensure efficient memory usage and performance when handling large datasets
* Integrate data from SQL-based systems into cloud and ML pipelines
* Apply best practices around reliability, monitoring, and scalability
Required Experience
* Strong commercial experience as a Data Engineer
* Strong Python development skills
* Hands-on experience with cloud object storage (GCS preferred, or AWS S3)
* Practical PyTorch experience (e.g. supporting training pipelines, dataset handling, data loaders)
* Experience working in cloud environments with large-scale file-based data
Desired Experience
* BigQuery (GCP)
* SQL databases (Microsoft SQL Server preferred; PostgreSQL also acceptable)
* Memory management and performance optimisation
* Exposure to ML workflows (without being a dedicated ML Engineer)
Nice to Have
* Broader GCP experience (Cloud Run, Cloud SQL, Cloud Scheduler, etc.)
* Pharma or life sciences domain exposure (or strong interest in the space)
* TensorFlow experience (acceptable alternative to PyTorch)
This is an excellent opportunity for a Data Engineer who has worked alongside ML teams and understands how to operationalise PyTorch workloads in the cloud.
Apply below!
To Apply for this Job Click Here
Senior Software Developer
£65000 - £80000
+ Data Engineering
PermanentSurrey
To Apply for this Job Click Here
Senior Software Developer
Location: Surrey (1 Day a Week in Office)
Salary: £80,000
Join a team where your ideas shape real-world solutions across aviation, defence, and the public sector, step into a role that gives you true autonomy and room to lead.
The Opportunity
This role offers the opportunity to lead the delivery of impactful software projects across civil aviation, defence, and public-sector domains. It provides significant autonomy, allowing the successful candidate to shape their own work approach and contribute to development best practices.
The position supports a blend of home and office working, giving flexibility to manage time and location effectively. It involves close collaboration with leadership to help guide processes and ensure high-quality outcomes. Applicants who are organised, adaptable, and experienced in managing complex project demands will thrive in this environment.
Role and Responsibilities
The role involves delivering enterprise-grade software solutions using a full-stack skill set that includes Java, Angular, RESTful services, SQL, and secure coding practices. The successful candidate will design, build, and maintain high-quality applications while applying SOLID principles, test-driven development, and robust security standards.
They will collaborate closely with development leadership to shape best practices and ensure consistent technical excellence. The position also includes managing workload independently, choosing effective working environments, and contributing to project planning, proposals, and timelines.
Successful applicant will need to be eligible for security clearance!
Interview Process – TBC
If this role sounds interesting to you, and you feel that you could make a positive impact, apply now!
To Apply for this Job Click Here
GCP Data Engineer
London
£700 - £750
+ Data Engineering
ContractLondon
To Apply for this Job Click Here
GCP Data Engineer
£700 – £750 per day inside IR35
6-month contract
Hybrid working in London
We’re working with a global healthcare and AI research organisation at the forefront of applying data engineering and machine learning to accelerate scientific discovery. Their work supports large-scale, domain-specific datasets that power research into life-changing treatments.
They’re now looking for a GCP Data Engineer to join a multidisciplinary team responsible for building and operating robust, cloud-native data infrastructure that supports ML workloads, particularly PyTorch-based pipelines.
The Role
You’ll focus on designing, building, and maintaining scalable data pipelines and storage systems in Google Cloud, supporting ML teams by enabling efficient data loading, dataset management, and cloud-based training workflows.
You’ll work closely with ML engineers and researchers, ensuring that large volumes of unstructured and structured data can be reliably accessed, processed, and consumed by PyTorch-based systems.
Key Responsibilities
-
Design and build cloud-native data pipelines using Python on GCP
-
Manage large-scale object storage for unstructured data (Google Cloud Storage preferred)
-
Support PyTorch-based workflows, particularly around data loading and dataset management in the cloud
-
Build and optimise data integrations with BigQuery and SQL databases
-
Ensure efficient memory usage and performance when handling large datasets
-
Collaborate with ML engineers to support training and experimentation pipelines (without owning model development)
-
Implement monitoring, testing, and documentation to ensure production-grade reliability
-
Participate in agile ceremonies, code reviews, and technical design discussions
Tech Stack & Experience
Must Have
-
Strong Python development experience
-
Hands-on experience with cloud object storage for unstructured data
(Google Cloud Storage preferred; AWS S3 also acceptable) -
PyTorch experience, particularly:
-
Dataset management
-
Data loading pipelines
-
Running PyTorch workloads in cloud environments
We are not looking for years of PyTorch experience – one or two substantial 6-12 month projects is ideal
-
-
5+ years cloud experience, ideally working with large numbers of files in cloud buckets
Nice to Have
-
Experience with additional GCP services, such as:
-
Cloud Run
-
Cloud SQL
-
Cloud Scheduler
-
-
Exposure to machine learning workflows (not ML engineering)
-
Some pharma or life sciences experience, or a genuine interest in working with domain-specific scientific data
Please send your CV
To Apply for this Job Click Here
Data Engineer
London
£55000 - £70000
+ Data Engineering
PermanentLondon
To Apply for this Job Click Here
Data Engineer
Location: London (Hybrid)
Salary: £70,000
Shape the future of audio analytics at a global media giant by building scalable data platforms, standardised infrastructure and cutting-edge frameworks that power smarter audience insight, stronger engagement, and high-impact digital experiences across multiple international markets.
The Opportunity
This opportunity offers the chance to join a global media organisation as it modernises and elevates its audio data capabilities. You’ll play a key role in building unified data platforms and standardised infrastructure that will be used across multiple international markets. Central to the role is creating robust pipelines and analytics layers that enable deeper audience insights and more impactful digital engagement.
Working within a small, forward-thinking engineering team, you’ll act as part of an internal consultancy driving best practice across the business. It’s a chance to influence architecture, shape frameworks from the ground up, and directly enhance everything from competitions to large-scale campaign performance.
Role and Responsibilities
In this role, you’ll build and maintain robust data pipelines using Python and SQL to ensure clean, reliable, and well-structured data. You’ll design and optimise workflows using tools such as Airflow and dbt to support scalable, high-quality analytics layers. You’ll help architect and evolve cloud-based data platforms across technologies like BigQuery, Redshift, and Terraform.
You’ll contribute to creating a standardised, self-serve data infrastructure that can be adopted across multiple international teams. You’ll also work closely with engineers and stakeholders to validate data quality, improve processes, and drive best-practice engineering across Kubernetes-based environments.
Interview Process
- Introductory Conversation – Discussing Experience and CV
- Technical Test
- Culture Fit Conversation
If you’re excited by the chance to help build standardised data platforms, enhance audience analytics, and influence architecture within a global media group’s growing audio data team, apply now.
To Apply for this Job Click Here
Principal Software Engineer – Data Platform
San Francisco
$230000 - $280000
+ Data Engineering
PermanentSan Francisco, California
To Apply for this Job Click Here
Principal Software Engineer – Data Platform
San Francisco, CA (Remote Eligible – US Only)
$230K-$280K base + equity
Are you ready to lead the design and build of a world-class data platform from the ground up?
A high-growth, product-led tech company is looking for a Principal Software Engineer to lead the build-out of a modern, scalable data platform from the ground up. If you’re excited about solving massive data challenges and shaping the infrastructure that supports real-time analytics and AI/ML at scale, this role is for you.
As the most senior technical IC on the Data Platform team, you’ll partner with leadership across Engineering, Product, Security, and Compliance to design a low-latency, cloud-native data architecture using best-in-class tools like Kafka, Iceberg, ClickHouse, Tinybird, and Snowflake.
Why this role?
- Join a growing Data Platform team
- Own the technical vision and architecture end-to-end
- Tackle petabyte-scale data challenges and real-time streaming use cases
- Work with a cutting-edge tech stack (Kafka, Iceberg, ClickHouse, Snowflake, etc.)
- Drive platform evolution to support AI/ML and company-wide analytics
What You’ll Do
- Design and build a next-gen data platform to power product and internal use cases
- Architect for batch + streaming, scalability, fault tolerance, and low latency
- Lead data modeling, warehousing, governance, and ingestion best practices
- Champion engineering excellence and mentor senior engineers
- Align technical decisions with long-term business strategy
- Optimize for performance, cost, and resiliency in a modern cloud environment
What We’re Looking For
- 8+ years of experience in data engineering or platform architecture
- 5+ years at the Principal Engineer level or equivalent
- Deep expertise with Kafka (and its ecosystem), streaming data systems
- Strong experience with ClickHouse, Iceberg, Snowflake, or similar tools
- Cloud-native architecture expertise (AWS, GCP, or Azure)
- Track record of designing data systems for compliance and governance
- Experience scaling systems to support petabyte-level workloads
- Excellent communication, collaboration, and leadership skills
- Bonus: Experience enabling AI/ML platforms or preparing for IPO-scale growth
To Apply for this Job Click Here
Senior Software Engineer – Lua
San Francisco
$200000 - $280000
+ Data Engineering
PermanentSan Francisco, California
To Apply for this Job Click Here
Senior Software Engineer – Lua
Remote (US) | $200,000 – $280,000 base + equity + benefits
A market leader in cloud infrastructure and developer tooling is seeking a Senior Software Engineer to help design, scale, and optimize a next-generation global CDN. This team builds the backbone of high-performance web delivery-handling billions of requests per day and powering some of the fastest, most reliable online experiences in the world.
The Role
As part of the CDN Infrastructure team, you’ll build and enhance distributed systems that make the internet faster and more resilient. You’ll design large-scale systems, write production-grade code, and work closely with product and infrastructure engineers to deliver high-impact improvements in routing, caching, and network performance.
What You’ll Do
- Design, implement, and optimize new CDN features and edge performance improvements
- Diagnose and reduce latency across a global network serving petabytes of data
- Improve reliability and scalability for billions of daily requests
- Build internal tools and workflows for infrastructure developers (primarily in Lua)
- Collaborate across teams to deliver next-generation web infrastructure
What We’re Looking For
- 5+ years of professional experience with Lua
- Deep understanding of HTTP, web servers, and CDN architecture
- Proven experience with distributed systems and large-scale performance optimization
- Strong problem-solving skills and a passion for experimentation
- Bonus: experience with Golang or other backend languages
Why Join
- Work on systems that serve petabytes of data and billions of requests daily
- Competitive compensation including equity and full health coverage
- Flexible, remote-friendly work environment
- Opportunity to shape the next generation of internet-scale infrastructure
To Apply for this Job Click Here
Senior Software Engineer – Go
San Francisco
$190000 - $240000
+ Data Engineering
PermanentSan Francisco, California
To Apply for this Job Click Here
Senior Software Engineer – Go (CDN / Distributed Systems)
Remote – US or Canada
$190-240k base + RSUs
A market leader in cloud infrastructure and developer tooling is seeking a Senior Software Engineer (Go) to help design, scale, and optimize a next-generation global CDN.
This team builds the backbone of high-performance web delivery-handling billions of requests per day and powering some of the fastest, most reliable online experiences in the world.
If you’re passionate about distributed systems, high-performance networking, and building software that powers millions of developers, this is an opportunity to have a direct impact on how the web runs at scale.
The Role
As part of the CDN Infrastructure group, you’ll build and enhance distributed systems that make the internet faster and more resilient. You’ll design large-scale systems, write production-grade code, and collaborate closely with product and infrastructure teams to deliver high-impact improvements in routing, caching, and network performance.
You’ll work across key initiatives that include:
- Evolving an internal CDN framework into a fully production-grade product that customers can use directly.
- Designing systems that enable incremental customer migration onto the platform.
- Scaling redirect handling from thousands to billions of requests per site.
- Adding HTTP/3 support and exploring post-quantum cryptography for next-generation security.
- Driving performance optimization across the edge network and core infrastructure.
This is a role for engineers who love building systems that operate at global scale, thrive on technical depth, and want to shape the future of web infrastructure.
What We’re Looking For
We’re looking for backend engineers with strong distributed systems experience and the ability to design, implement, and scale complex infrastructure.
You’ll bring:
- Proven experience with distributed systems, CDN, caching, or ingress infrastructure.
- Go experience in production (C/C++ backgrounds highly transferable).
- Deep understanding of the HTTP layer and modern cloud networking.
- Experience running systems at scale in a cloud product or infrastructure environment.
- Bonus: experience in Linux kernel tuning, performance optimization, or relevant open-source contributions.
This role is ideal for engineers who are hands-on, performance-oriented, and eager to work on problems of scale.
Why Join
You’ll join a team that values technical excellence, autonomy, and impact-where your work directly shapes how developers build and deploy at the edge.
We operate globally, handle traffic at massive scale, and give engineers the ownership to build systems that redefine what’s possible in web performance and reliability.
We’re a remote-first organization that empowers engineers to make meaningful architectural decisions, ship quickly, and see their impact on real-world products at scale.
To Apply for this Job Click Here
AI Software Engineer
Remote
$120000 - $160000
+ Data Engineering
PermanentOhio
To Apply for this Job Click Here
AI Software Engineer
Remote (US – ET/CT Time Zone)
$120K-$160K + Benefits
I’m partnering with a global leader in digital learning technology that’s transforming how millions of students and educators experience personalized, AI-powered learning. Following major investment in generative AI, the company is scaling its AI Platform Engineering team to build the next generation of adaptive learning tools.
This is a hands-on engineering role focused on building production-grade generative AI applications that power multiple products across the business.
The Role:
You’ll design and build scalable, full-stack systems that integrate large language models and AI capabilities into digital learning platforms. Working closely with data scientists, product managers, and engineers, you’ll help take GenAI prototypes from experimentation to production-driving forward real-world AI impact.
Key Responsibilities
- Develop and maintain AI-powered applications and platform services that are reliable, scalable, and secure
- Build and optimize LLM-based and RAG-powered solutions using frameworks such as LangChain or LangGraph
- Collaborate with Data Science teams to productionize models and streamline deployment pipelines
- Lead and contribute to technical design, architecture, and CI/CD improvements
- Ensure applications meet accessibility (WCAG 2.2 AA), performance, and security standards
- Stay current on the latest advancements in GenAI and advocate for best practices across the team
About You
- 5+ years of professional software engineering experience
- Strong full-stack skills across:
- Backend: Python, Node.js, or Go
- Frontend: Angular or React
- Databases: MySQL/PostgreSQL + NoSQL (e.g., DynamoDB)
- Cloud/Infra: AWS or Azure, Terraform, CI/CD tools
- Hands-on experience building AI/GenAI-enabled applications (Azure OpenAI, Amazon Bedrock, etc.)
- Practical understanding of RAG techniques, vector databases, and AI orchestration tools
- Strong system design, performance optimization, and collaboration skills
- Passionate about building impactful technology and staying ahead in the AI space
To Apply for this Job Click Here
Senior Software Engineer – CDN & Go
San Francisco
$190000 - $240000
+ Data Engineering
PermanentSan Francisco, California
To Apply for this Job Click Here
Senior Software Engineer – Go (CDN and Distributed Systems)
Remote, US or Canada
$190-204k base + RSUs
A leading cloud infrastructure company is hiring a Senior Software Engineer with strong Go and distributed systems experience to help build and scale its next generation global CDN. This team operates at massive scale, handling billions of requests per day and powering some of the fastest, most reliable experiences on the web.
The Role
You will design and build high performance distributed systems that make the internet faster and more resilient. As part of the CDN Infrastructure group, you will work on projects such as:
- Evolving an internal CDN into a production ready customer platform
- Designing systems that support safe, incremental customer migration
- Scaling redirect handling from thousands to billions of requests
- Implementing HTTP/3 and exploring post quantum cryptography
- Driving performance improvements across the edge network
This role is ideal for engineers who enjoy deep technical challenges and want to influence how modern web infrastructure operates at global scale.
What We Are Looking For
- Strong experience with distributed systems, CDN, caching, or networking infrastructure
- Production Go experience. C and C plus plus backgrounds transfer well
- Solid understanding of the HTTP layer and cloud networking
- Experience operating systems at scale in a cloud or infra environment
- Bonus: Linux tuning, performance optimization, or open source work
To Apply for this Job Click Here
CAN’T FIND THE RIGHT OPPORTUNITY?
GET IN TOUCH
today
If you can’t see what you’re looking for right now, send us your CV anyway – we’re always getting fresh new roles through the door.
Industry Hub
HARNHAM
News & Blog
With over 10 years experience working solely in the Data & AI sector our consultants are able to offer detailed insights into the industry.
Visit our Blogs & News portal or check out our recent posts below.
Testimonials
Client and
candidate
testimonials
A trusted partner of professionals
across the globe.
We understand the challenges our
customers face and offer the
recruitment solutions needed to drive
business success through Data &
AI.
Candidate – Data Analyst
I’ve had a great experience with Harnham. The recruiter provided great advice and ensured that I understood everything, and even followed up with me after I started my new role to see how I was settling in. I would recommend Harnham as a great recruitment company for anyone looking for a role in data and analytics.
Candidate /
Data Analyst
Client – Data Direction – Broadcast Media
I have worked with Harnham for over 2 years now as our main Analytics Recruiter partner and I am very pleased with the overall service and the people we have hired via Harnham. Harnham’s team is very good at understanding our recruitment needs and always come back with relevant and well scanned CVs, which saves me precious time.
Client /
Data Direction – Broadcast Media
Candidate – Digital Media and Analytics
I've recently got a short-term contract role through Harnham, and things couldn't have gone better! My recruiter was incredibly helpful throughout the entire process. They also provided incredible admin support so that I could focus on the actual job in hand, rather than having to spend too much time dealing with these matters.
Candidate /
Digital Media and Analytics
Candidate – Chief Data Officer
It was a real treat to work with Harnham. My recruiter consistently went above and beyond to ensure that I navigated through the recruitment process as smooth as possible. I had been in contact with other professional at Harnham in the past and I can attest that the level of professionalism is always top-notch.
Candidate /
Chief Data Officer
Candidate – Senior Data Engineer
I've been a Senior Data Engineer for about 4-5 years and Harnham is one of the best recruitment agencies I worked with. There was a time when my offer fell through and the recruiter I was working with at Harnham jumped in to the rescue and scheduled interviews for me within a couple of days. Really good agency and some great people.
Candidate /
Senior Data Engineer
Client – Director of Customer Data Science – Apparel
The team at Harnham have been outstanding in helping our company build a market-leading data science team. Harnham’s communication is great, and the recruiter’s take a consultative approach that’s really focused on provided value for both their clients and their candidates. They provide strong candidates that help us fill our roles with high achievers who are also a great fit for our culture.
Client /
Director of Customer Data Science – Apparel
Candidate – Market Research Analyst
Thanks to Harnham, I am now in a role that suits me. My recruiter was able to quickly identify the skills I had to offer and match me up with his client. It was a win-win for everyone. I was given great advice throughout the process, and I also appreciated the follow-ups that I received.
Candidate /
Market Research Analyst
Client – Director of People Operations
We’ve been collaborating with Harnham to recruit some key positions for our company, and they have been incredible partners. They team has been great and efficient. They understood our needs perfectly and provided us very talented candidates. We'll keep collaborating with them in the future. Thank you for your excellent job!
