DATA ENGINEERING AND BIG DATA
TALENT SOLUTIONS
Processing Big Data has become crucial to countless businesses. Those who decide to pursue a career within Big Data face complex development challenges, too tough for relational database systems. After all, there is no Data Science without Data Engineers.
Whilst businesses may have vast quantities of data at the ready, it holds no real value unless it can be stored, harnessed and utilised. It’s estimated that most UK companies have at least 100 terabytes of data stored which could be used for various purposes around the business, as well as newly accessible hard-to-process sources such as web data, image data, and social media data.
This is where those who work in Data Engineering become extremely valuable, as they develop Data Platforms that allow this kind of volume to be processed and used by Data Scientists and analysts across the business.
WHY
HARNHAM?
Our esteemed position as a global leader in data recruitment stems from our deep expertise in Big Data and Data Engineering, supported by hundreds of dedicated specialists located across the United States, Europe, and the United Kingdom.
Our extensive network enables us to deliver exceptional recruitment services, expertly matching our clients with top-tier Data Engineering talent.
We acknowledge the distinct challenges and requirements of each organization's data infrastructure.
Therefore, our recruitment strategies are meticulously tailored, emphasizing a thorough understanding of and alignment with your specific business objectives and data technological needs.
OUR
SERVICES
-
Permanent and Contract Recruitment: We offer both permanent and contract recruitment solutions, adapting flexibly to support the dynamic requirements of your Big Data and Data Engineering projects.
Â
-
Executive Search: Our executive search service focuses on identifying and securing top-tier leaders in the Data Engineering domain who can advance your strategic goals and technological advancements.
Â
-
Industry-Specific Expertise: Our operations span multiple industries, providing specialized recruitment solutions tailored to the distinctive challenges and opportunities of your sector.
Â
Contact us today to discover how our custom talent solutions can bolster your organization's Data Engineering prowess.
JOBS
LATEST
OPPORTUNITIES
Harnham are a specialist Data & AI recruitment business with teams that only focus on niche areas.
Senior Software Developer
£65000 - £80000
+ Data Engineering
PermanentSurrey
To Apply for this Job Click Here
Senior Software Developer
Location: Surrey (1 Day a Week in Office)
Salary: £80,000
Join a team where your ideas shape real-world solutions across aviation, defence, and the public sector, step into a role that gives you true autonomy and room to lead.
The Opportunity
This role offers the opportunity to lead the delivery of impactful software projects across civil aviation, defence, and public-sector domains. It provides significant autonomy, allowing the successful candidate to shape their own work approach and contribute to development best practices.
The position supports a blend of home and office working, giving flexibility to manage time and location effectively. It involves close collaboration with leadership to help guide processes and ensure high-quality outcomes. Applicants who are organised, adaptable, and experienced in managing complex project demands will thrive in this environment.
Role and Responsibilities
The role involves delivering enterprise-grade software solutions using a full-stack skill set that includes Java, Angular, RESTful services, SQL, and secure coding practices. The successful candidate will design, build, and maintain high-quality applications while applying SOLID principles, test-driven development, and robust security standards.
They will collaborate closely with development leadership to shape best practices and ensure consistent technical excellence. The position also includes managing workload independently, choosing effective working environments, and contributing to project planning, proposals, and timelines.
Successful applicant will need to be eligible for security clearance!
Interview Process – TBC
If this role sounds interesting to you, and you feel that you could make a positive impact, apply now!
To Apply for this Job Click Here
GCP Data Engineer
London
£700 - £750
+ Data Engineering
ContractLondon
To Apply for this Job Click Here
GCP Data Engineer
£700 – £750 per day inside IR35
6-month contract
Hybrid working in London
We’re working with a global healthcare and AI research organisation at the forefront of applying data engineering and machine learning to accelerate scientific discovery. Their work supports large-scale, domain-specific datasets that power research into life-changing treatments.
They’re now looking for a GCP Data Engineer to join a multidisciplinary team responsible for building and operating robust, cloud-native data infrastructure that supports ML workloads, particularly PyTorch-based pipelines.
The Role
You’ll focus on designing, building, and maintaining scalable data pipelines and storage systems in Google Cloud, supporting ML teams by enabling efficient data loading, dataset management, and cloud-based training workflows.
You’ll work closely with ML engineers and researchers, ensuring that large volumes of unstructured and structured data can be reliably accessed, processed, and consumed by PyTorch-based systems.
Key Responsibilities
-
Design and build cloud-native data pipelines using Python on GCP
-
Manage large-scale object storage for unstructured data (Google Cloud Storage preferred)
-
Support PyTorch-based workflows, particularly around data loading and dataset management in the cloud
-
Build and optimise data integrations with BigQuery and SQL databases
-
Ensure efficient memory usage and performance when handling large datasets
-
Collaborate with ML engineers to support training and experimentation pipelines (without owning model development)
-
Implement monitoring, testing, and documentation to ensure production-grade reliability
-
Participate in agile ceremonies, code reviews, and technical design discussions
Tech Stack & Experience
Must Have
-
Strong Python development experience
-
Hands-on experience with cloud object storage for unstructured data
(Google Cloud Storage preferred; AWS S3 also acceptable) -
PyTorch experience, particularly:
-
Dataset management
-
Data loading pipelines
-
Running PyTorch workloads in cloud environments
We are not looking for years of PyTorch experience – one or two substantial 6-12 month projects is ideal
-
-
5+ years cloud experience, ideally working with large numbers of files in cloud buckets
Nice to Have
-
Experience with additional GCP services, such as:
-
Cloud Run
-
Cloud SQL
-
Cloud Scheduler
-
-
Exposure to machine learning workflows (not ML engineering)
-
Some pharma or life sciences experience, or a genuine interest in working with domain-specific scientific data
Please send your CV
To Apply for this Job Click Here
Data Engineer
London
£55000 - £70000
+ Data Engineering
PermanentLondon
To Apply for this Job Click Here
Data Engineer
Location: London (Hybrid)
Salary: £70,000
Shape the future of audio analytics at a global media giant by building scalable data platforms, standardised infrastructure and cutting-edge frameworks that power smarter audience insight, stronger engagement, and high-impact digital experiences across multiple international markets.
The Opportunity
This opportunity offers the chance to join a global media organisation as it modernises and elevates its audio data capabilities. You’ll play a key role in building unified data platforms and standardised infrastructure that will be used across multiple international markets. Central to the role is creating robust pipelines and analytics layers that enable deeper audience insights and more impactful digital engagement.
Working within a small, forward-thinking engineering team, you’ll act as part of an internal consultancy driving best practice across the business. It’s a chance to influence architecture, shape frameworks from the ground up, and directly enhance everything from competitions to large-scale campaign performance.
Role and Responsibilities
In this role, you’ll build and maintain robust data pipelines using Python and SQL to ensure clean, reliable, and well-structured data. You’ll design and optimise workflows using tools such as Airflow and dbt to support scalable, high-quality analytics layers. You’ll help architect and evolve cloud-based data platforms across technologies like BigQuery, Redshift, and Terraform.
You’ll contribute to creating a standardised, self-serve data infrastructure that can be adopted across multiple international teams. You’ll also work closely with engineers and stakeholders to validate data quality, improve processes, and drive best-practice engineering across Kubernetes-based environments.
Interview Process
- Introductory Conversation – Discussing Experience and CV
- Technical Test
- Culture Fit Conversation
If you’re excited by the chance to help build standardised data platforms, enhance audience analytics, and influence architecture within a global media group’s growing audio data team, apply now.
To Apply for this Job Click Here
Principal Software Engineer – Data Platform
San Francisco
$230000 - $280000
+ Data Engineering
PermanentSan Francisco, California
To Apply for this Job Click Here
Principal Software Engineer – Data Platform
San Francisco, CA (Remote Eligible – US Only)
$230K-$280K base + equity
Are you ready to lead the design and build of a world-class data platform from the ground up?
A high-growth, product-led tech company is looking for a Principal Software Engineer to lead the build-out of a modern, scalable data platform from the ground up. If you’re excited about solving massive data challenges and shaping the infrastructure that supports real-time analytics and AI/ML at scale, this role is for you.
As the most senior technical IC on the Data Platform team, you’ll partner with leadership across Engineering, Product, Security, and Compliance to design a low-latency, cloud-native data architecture using best-in-class tools like Kafka, Iceberg, ClickHouse, Tinybird, and Snowflake.
Why this role?
- Join a growing Data Platform team
- Own the technical vision and architecture end-to-end
- Tackle petabyte-scale data challenges and real-time streaming use cases
- Work with a cutting-edge tech stack (Kafka, Iceberg, ClickHouse, Snowflake, etc.)
- Drive platform evolution to support AI/ML and company-wide analytics
What You’ll Do
- Design and build a next-gen data platform to power product and internal use cases
- Architect for batch + streaming, scalability, fault tolerance, and low latency
- Lead data modeling, warehousing, governance, and ingestion best practices
- Champion engineering excellence and mentor senior engineers
- Align technical decisions with long-term business strategy
- Optimize for performance, cost, and resiliency in a modern cloud environment
What We’re Looking For
- 8+ years of experience in data engineering or platform architecture
- 5+ years at the Principal Engineer level or equivalent
- Deep expertise with Kafka (and its ecosystem), streaming data systems
- Strong experience with ClickHouse, Iceberg, Snowflake, or similar tools
- Cloud-native architecture expertise (AWS, GCP, or Azure)
- Track record of designing data systems for compliance and governance
- Experience scaling systems to support petabyte-level workloads
- Excellent communication, collaboration, and leadership skills
- Bonus: Experience enabling AI/ML platforms or preparing for IPO-scale growth
To Apply for this Job Click Here
Senior Software Engineer – Lua
San Francisco
$200000 - $280000
+ Data Engineering
PermanentSan Francisco, California
To Apply for this Job Click Here
Senior Software Engineer – Lua
Remote (US) | $200,000 – $280,000 base + equity + benefits
A market leader in cloud infrastructure and developer tooling is seeking a Senior Software Engineer to help design, scale, and optimize a next-generation global CDN. This team builds the backbone of high-performance web delivery-handling billions of requests per day and powering some of the fastest, most reliable online experiences in the world.
The Role
As part of the CDN Infrastructure team, you’ll build and enhance distributed systems that make the internet faster and more resilient. You’ll design large-scale systems, write production-grade code, and work closely with product and infrastructure engineers to deliver high-impact improvements in routing, caching, and network performance.
What You’ll Do
- Design, implement, and optimize new CDN features and edge performance improvements
- Diagnose and reduce latency across a global network serving petabytes of data
- Improve reliability and scalability for billions of daily requests
- Build internal tools and workflows for infrastructure developers (primarily in Lua)
- Collaborate across teams to deliver next-generation web infrastructure
What We’re Looking For
- 5+ years of professional experience with Lua
- Deep understanding of HTTP, web servers, and CDN architecture
- Proven experience with distributed systems and large-scale performance optimization
- Strong problem-solving skills and a passion for experimentation
- Bonus: experience with Golang or other backend languages
Why Join
- Work on systems that serve petabytes of data and billions of requests daily
- Competitive compensation including equity and full health coverage
- Flexible, remote-friendly work environment
- Opportunity to shape the next generation of internet-scale infrastructure
To Apply for this Job Click Here
Senior Software Engineer – Go
San Francisco
$190000 - $240000
+ Data Engineering
PermanentSan Francisco, California
To Apply for this Job Click Here
Senior Software Engineer – Go (CDN / Distributed Systems)
Remote – US or Canada
$190-240k base + RSUs
A market leader in cloud infrastructure and developer tooling is seeking a Senior Software Engineer (Go) to help design, scale, and optimize a next-generation global CDN.
This team builds the backbone of high-performance web delivery-handling billions of requests per day and powering some of the fastest, most reliable online experiences in the world.
If you’re passionate about distributed systems, high-performance networking, and building software that powers millions of developers, this is an opportunity to have a direct impact on how the web runs at scale.
The Role
As part of the CDN Infrastructure group, you’ll build and enhance distributed systems that make the internet faster and more resilient. You’ll design large-scale systems, write production-grade code, and collaborate closely with product and infrastructure teams to deliver high-impact improvements in routing, caching, and network performance.
You’ll work across key initiatives that include:
- Evolving an internal CDN framework into a fully production-grade product that customers can use directly.
- Designing systems that enable incremental customer migration onto the platform.
- Scaling redirect handling from thousands to billions of requests per site.
- Adding HTTP/3 support and exploring post-quantum cryptography for next-generation security.
- Driving performance optimization across the edge network and core infrastructure.
This is a role for engineers who love building systems that operate at global scale, thrive on technical depth, and want to shape the future of web infrastructure.
What We’re Looking For
We’re looking for backend engineers with strong distributed systems experience and the ability to design, implement, and scale complex infrastructure.
You’ll bring:
- Proven experience with distributed systems, CDN, caching, or ingress infrastructure.
- Go experience in production (C/C++ backgrounds highly transferable).
- Deep understanding of the HTTP layer and modern cloud networking.
- Experience running systems at scale in a cloud product or infrastructure environment.
- Bonus: experience in Linux kernel tuning, performance optimization, or relevant open-source contributions.
This role is ideal for engineers who are hands-on, performance-oriented, and eager to work on problems of scale.
Why Join
You’ll join a team that values technical excellence, autonomy, and impact-where your work directly shapes how developers build and deploy at the edge.
We operate globally, handle traffic at massive scale, and give engineers the ownership to build systems that redefine what’s possible in web performance and reliability.
We’re a remote-first organization that empowers engineers to make meaningful architectural decisions, ship quickly, and see their impact on real-world products at scale.
To Apply for this Job Click Here
AI Software Engineer
Remote
$120000 - $160000
+ Data Engineering
PermanentOhio
To Apply for this Job Click Here
AI Software Engineer
Remote (US – ET/CT Time Zone)
$120K-$160K + Benefits
I’m partnering with a global leader in digital learning technology that’s transforming how millions of students and educators experience personalized, AI-powered learning. Following major investment in generative AI, the company is scaling its AI Platform Engineering team to build the next generation of adaptive learning tools.
This is a hands-on engineering role focused on building production-grade generative AI applications that power multiple products across the business.
The Role:
You’ll design and build scalable, full-stack systems that integrate large language models and AI capabilities into digital learning platforms. Working closely with data scientists, product managers, and engineers, you’ll help take GenAI prototypes from experimentation to production-driving forward real-world AI impact.
Key Responsibilities
- Develop and maintain AI-powered applications and platform services that are reliable, scalable, and secure
- Build and optimize LLM-based and RAG-powered solutions using frameworks such as LangChain or LangGraph
- Collaborate with Data Science teams to productionize models and streamline deployment pipelines
- Lead and contribute to technical design, architecture, and CI/CD improvements
- Ensure applications meet accessibility (WCAG 2.2 AA), performance, and security standards
- Stay current on the latest advancements in GenAI and advocate for best practices across the team
About You
- 5+ years of professional software engineering experience
- Strong full-stack skills across:
- Backend: Python, Node.js, or Go
- Frontend: Angular or React
- Databases: MySQL/PostgreSQL + NoSQL (e.g., DynamoDB)
- Cloud/Infra: AWS or Azure, Terraform, CI/CD tools
- Hands-on experience building AI/GenAI-enabled applications (Azure OpenAI, Amazon Bedrock, etc.)
- Practical understanding of RAG techniques, vector databases, and AI orchestration tools
- Strong system design, performance optimization, and collaboration skills
- Passionate about building impactful technology and staying ahead in the AI space
To Apply for this Job Click Here
Senior Software Engineer – CDN & Go
San Francisco
$190000 - $240000
+ Data Engineering
PermanentSan Francisco, California
To Apply for this Job Click Here
Senior Software Engineer – Go (CDN and Distributed Systems)
Remote, US or Canada
$190-204k base + RSUs
A leading cloud infrastructure company is hiring a Senior Software Engineer with strong Go and distributed systems experience to help build and scale its next generation global CDN. This team operates at massive scale, handling billions of requests per day and powering some of the fastest, most reliable experiences on the web.
The Role
You will design and build high performance distributed systems that make the internet faster and more resilient. As part of the CDN Infrastructure group, you will work on projects such as:
- Evolving an internal CDN into a production ready customer platform
- Designing systems that support safe, incremental customer migration
- Scaling redirect handling from thousands to billions of requests
- Implementing HTTP/3 and exploring post quantum cryptography
- Driving performance improvements across the edge network
This role is ideal for engineers who enjoy deep technical challenges and want to influence how modern web infrastructure operates at global scale.
What We Are Looking For
- Strong experience with distributed systems, CDN, caching, or networking infrastructure
- Production Go experience. C and C plus plus backgrounds transfer well
- Solid understanding of the HTTP layer and cloud networking
- Experience operating systems at scale in a cloud or infra environment
- Bonus: Linux tuning, performance optimization, or open source work
To Apply for this Job Click Here
Staff Analytics Engineer
$200000 - $250000
+ Data Engineering
PermanentCalifornia
To Apply for this Job Click Here
Staff Analytics Engineer
Location: Remote
Salary: $200-250k
Are you a data expert ready to drive transformative analytics at a cutting-edge cloud platform company? This fast-growing Series C innovator is revolutionizing how teams build, preview, and deploy high-performance digital applications.
About the Role
The Staff Analytics Engineer will lead the design and implementation of robust data models and pipelines, enabling business units to uncover insights and make data-driven decisions. This role is a unique chance to collaborate with engineering, product, and business teams, building scalable analytics systems that power operational efficiency and growth.
Key Responsibilities
- Design and maintain scalable data models and pipelines to serve as a single source of truth across the enterprise.
- Develop tools to enhance data accessibility, auditing, and validation, ensuring consistent, trustworthy datasets.
- Partner with cross-functional teams to uncover actionable insights and support decision-making.
- Build self-service analytics tools and dashboards for key stakeholders, streamlining workflows.
- Collaborate on advancing the company’s data platform using cutting-edge tools like dbt, Snowflake, and Superset.
- Lead initiatives to improve data quality, governance, and discoverability, ensuring alignment with business goals.
About You
- 8+ years of experience in analytics engineering or a similar role in a modern tech company.
- Advanced proficiency in SQL, with experience working on complex datasets.
- Expertise in data platforms like Snowflake or BigQuery and transformation tools like dbt.
- Familiarity with programming languages such as Python, Java, or Go.
- Experience with BI tools like Hashboard or Superset, and a strong grasp of schema design and dimensional modeling.
- Strong communication skills and the ability to collaborate in a fast-paced environment.
Be part of a forward-thinking team where data is at the core of every decision. If you’re passionate about transforming data into meaningful insights, this role offers the chance to work on impactful projects, collaborate with top talent, and make a lasting difference.
To Apply for this Job Click Here
CAN’T FIND THE RIGHT OPPORTUNITY?
GET IN TOUCH
today
If you can’t see what you’re looking for right now, send us your CV anyway – we’re always getting fresh new roles through the door.
Industry Hub
HARNHAM
News & Blog
With over 10 years experience working solely in the Data & AI sector our consultants are able to offer detailed insights into the industry.
Visit our Blogs & News portal or check out our recent posts below.
How a Leading European Media Group Unified Its Data Strategy with Harnham
by Jamie Smith, Senior Manager at Harnham, UK. A major privately owned media organisation in Europe, employing…
From Start-Up to Scale-Up: Hire Exceptional Data Talent in Two Weeks
By Guillian Eller, Manager AI/ML – Harnham You’re growing fast. You needed data talent yesterday. The reality…
Reskilling & Upskilling: What US Data Professionals Are Doing
By Luc Simpson-Kent, Managing Consultant – Harnham US Candidates want growth. Employers want retention. And in today’s…
What’s Driving Northern UK’s Data & AI Hiring in 2025?
by Jamie Smith, Senior Manager at Harnham, UK. At a glance The Northern data and tech market…
The Key to a Strong AI Team: Diversity
by Tom Brammer, Senior Manager – AI and Machine Learning US Team Every company wants to build…
Why US Companies Are Scaling AI with Contractors in 2025
Artificial intelligence has arrived in the workplace and it’s changing how companies are structured, resourced, and scaled. …
How Sacher.AI Strengthened Its GenAI Delivery in Just 12 Days
by Thomas Savidge, Principal Consultant – AI, Machine Learning and Data Science Sacher.AI, a UK-based innovation lab…
From Zero to AI: How One Investment Firm Built a World-Class Team in 18 Days
A global investment advisor managing over $268 billion across real estate, equity, and credit needed to build…
How to Maximise Your Data’s Value: The Use of AI in Advanced Analytics
By Roshni Baillie, Recruitment Consultant – Marketing & Insight, Harnham One thing I’m hearing again and again…
AI Governance Starts with People
By Kiran Ramasamy, Managing Consultant, Harnham When it comes to AI it is no longer a question…
Testimonials
Client and
candidate
testimonials
A trusted partner of professionals
across the globe.
We understand the challenges our
customers face and offer the
recruitment solutions needed to drive
business success through Data &
AI.
Candidate – Data Analyst
I’ve had a great experience with Harnham. The recruiter provided great advice and ensured that I understood everything, and even followed up with me after I started my new role to see how I was settling in. I would recommend Harnham as a great recruitment company for anyone looking for a role in data and analytics.
Candidate /
Data Analyst
Candidate – Digital Media and Analytics
I've recently got a short-term contract role through Harnham, and things couldn't have gone better! My recruiter was incredibly helpful throughout the entire process. They also provided incredible admin support so that I could focus on the actual job in hand, rather than having to spend too much time dealing with these matters.
Candidate /
Digital Media and Analytics
Candidate – Chief Data Officer
It was a real treat to work with Harnham. My recruiter consistently went above and beyond to ensure that I navigated through the recruitment process as smooth as possible. I had been in contact with other professional at Harnham in the past and I can attest that the level of professionalism is always top-notch.
Candidate /
Chief Data Officer
Client – Data Direction – Broadcast Media
I have worked with Harnham for over 2 years now as our main Analytics Recruiter partner and I am very pleased with the overall service and the people we have hired via Harnham. Harnham’s team is very good at understanding our recruitment needs and always come back with relevant and well scanned CVs, which saves me precious time.
Client /
Data Direction – Broadcast Media
Client – Director of People Operations
We’ve been collaborating with Harnham to recruit some key positions for our company, and they have been incredible partners. They team has been great and efficient. They understood our needs perfectly and provided us very talented candidates. We'll keep collaborating with them in the future. Thank you for your excellent job!
Client /
Director of People Operations
Client – Director of Customer Data Science – Apparel
The team at Harnham have been outstanding in helping our company build a market-leading data science team. Harnham’s communication is great, and the recruiter’s take a consultative approach that’s really focused on provided value for both their clients and their candidates. They provide strong candidates that help us fill our roles with high achievers who are also a great fit for our culture.
Client /
Director of Customer Data Science – Apparel
Candidate – Market Research Analyst
Thanks to Harnham, I am now in a role that suits me. My recruiter was able to quickly identify the skills I had to offer and match me up with his client. It was a win-win for everyone. I was given great advice throughout the process, and I also appreciated the follow-ups that I received.
Candidate /
Market Research Analyst
Candidate – Senior Data Engineer
I've been a Senior Data Engineer for about 4-5 years and Harnham is one of the best recruitment agencies I worked with. There was a time when my offer fell through and the recruiter I was working with at Harnham jumped in to the rescue and scheduled interviews for me within a couple of days. Really good agency and some great people.
