Harnham

Data Engineering and Big Data jobs

What We Do

Processing Big Data has become crucial to countless businesses. Those who decide to pursue a career within Big Data face complex development challenges, too tough for relational database systems. After all, there is no Data Science without Data Engineers.

Whilst businesses may have vast quantities of data at the ready, it holds no real value unless it can be stored, harnessed and utilised. It’s estimated that most UK companies have at least 100 terabytes of data stored which could be used for various purposes around the business, as well as newly accessible hard-to-process sources such as web data, image data, and social media data. 

This is where those who work in Data Engineering become extremely valuable, as they develop Data Platforms that allow this kind of volume to be processed and used by Data Scientists and analysts across the business.

Whether you are a company looking to build a data platform for advanced analytics, or you are a candidate who can build systems and applications to process vast, complex data sources, Harnham are here to help. 

Technical skills: AWS, Google Cloud Platform and Azure, Spark, Hadoop, Java, Scala, Python

Meet the team

Latest Jobs

Senior Data Engineer - Python + SQL + AWS
Salary

US$120000 - US$150000 per year + Comprehensive Benefits

Location

New York

Description

Our client is a leading Non-profit Organization that is seeking to massively improve education outcomes through the efforts of political campaigning!

ADD TO SHORTLIST
Director of Data Engineering
Salary

US$170000 - US$200000 per year + Equity + Benefits

Location

San Francisco, California

Description

This is an opportunity to join a Fintech business in SF, who are dramatically changing the credit industry and fast becoming a household name globally.

ADD TO SHORTLIST
Senior Data Engineer
Salary

US$180000 - US$210000 per year + Bonus + Equity

Location

San Francisco, California

Description

One of our pivotal clients in the SF Digital Health space is looking for a talented individual to help solve real-world problems in healthcare!

ADD TO SHORTLIST
Software Engineer - Data
Salary

US$120000 - US$150000 per year + Bonus

Location

San Francisco, California

Description

This is a chance to disrupt the future of driving and to be a part of history in the making, our generation of "race to the moon". Are you a software engineer?

ADD TO SHORTLIST
Senior Data Engineer
Salary

US$150000 - US$170000 per year + Equity + Benefits

Location

San Francisco, California

Description

Pre-IPO client is revolutionizing the fintech industry, creating financial profiles for billions of people worldwide by using their product thats powered by ML

ADD TO SHORTLIST
Data Engineer Manager
Salary

US$150000 - US$180000 per year + Bonus + Equity

Location

San Francisco, California

Description

One of the top pre-IPO CRM software company is looking for an Analytics Manager to join them in San Francisco.

ADD TO SHORTLIST

Harnham blog & news

With over 10 years experience working solely in the Data & Analytics sector our consultants are able to offer detailed insights into the industry.

Visit our Blogs & News portal or check out our recent posts below.

Why it is hard to build a Big Data team

Increasingly, I speak to managers who are adopting big data tools and developing PoCs to prove how they can make use of them. Just last week I spoke to a data architect who mentioned that if he didn’t get exposure to big data tech sooner rather than later, his current RDBMS skills may become redundant within the next few years. While that is likely an exaggeration, it is certainly an interesting point. Companies that would have never previously had the capability to interpret ‘Big Data’ are now exploring a variety of NoSQL platforms. In particular, the massive performance benefits gained from Spark and real-time/streaming tools have opened up a whole new world beyond just MapReduce. I don’t claim to be a data engineer, but as a recruiter for this sector, what I do is spend all day, every day interacting with big data developers, architects and managers (as well as keeping a close eye on the latest Apache incubator projects). Due to this, I have seen some recurring themes that have become trends when companies look to create and build their big data teams that are coming to the fore. Candidate demand The demand for Big Data professionals is very much a present day issue as the data companies have grand plans for is waiting for the right data developer to use the best tech to extract valuable insights from it. The best candidates receive massive interest, often gain multiple offers from a range of companies. Your business is now no longer just competing with large corporations such as Facebook, Twitter or Yahoo. Startups and SMEs are also vying for the best candidates. Candidates are seeing pay rises twice that of the normal rate, as illustrated in our salary guide. Candidate shortage The number of candidates with hands-on, production level Big Data experience is incredibly limited. We go to great lengths to find the candidates who can add real value to companies. The growth and exciting future for the big data industry has led to increased interest in big data jobs, particularly for those from RDBMS or software. engineering backgrounds. This leaves the industry in a difficult predicament: high demand + low supply = massive competition. There are countless examples of companies that have failed to recruit a Big Data team after a year of looking. Competition to get ahead and stand out Planning - Companies need to have a data road map detailing their future plans. Candidates want to clearly know what they are getting into and what to expect from a job. Innovation - Why get stuck on batch processing? The most exciting positions that candidates love are in data innovations teams, playing with real-time/streaming tech and new languages. Personal development, growth and training – with the data science market experiencing similar growth, many big data engineers are looking for a job that not only offers the chance to work with machine learning and similar fields; but training, mentoring towards clear career progression as standard. Speed – the length of the interview process is often seen as a reflection of the amount of red tape developers have to go through to get a job. The longer and more convoluted the process, the more put off some people may be. Complacency – don’t rest on your laurels, it’s unlikely that you’ll get 10s of CVs through when you are looking to fill a data role, so when you find a candidate you like, move swiftly to show your interest to them as quality candidates don’t come around often. By implementing these small but effective improvements to your recruiting process and how you develop data talent will see you create a team that is a success in this ever more digital analytics landscape. Companies who don’t create and nurture strong, dynamic teams will fall by the wayside. It’s Harnham’s job to help you achieve this goal. Get in touch with us to tell you how. T: (020) 8408 6070 E: info@harnham.com

Ross Whatling

06. June 2017

HOW TO MAXIMISE YOUR TRAVEL DATA WITH THE RIGHT ANALYTICS

Summer is here and over half will be planning their holiday via mobile phone. Heathrow Airport, one of the busiest airports in the world, will be filled to the brim with eager travelers. Hotels and cruise ships will be bursting at the seams. But, will the people traveling today, be the same people traveling next summer? How will the travel organisations know?  They’ll need data analytics to help them make sense of all their data with advice and direction on how best to utilise nuggets of actionable insights.   It wasn’t that long ago, at the busiest time of the year, Heathrow, as well as most of the airports around Europe, shut down. Why? Because no had planned for something of that magnitude – the snowstorm of the century some said. Tangible evidence of planning for every scenario, no matter how unlikely.   What Would a Data Professional Do?  Data professionals ask questions. They create scenarios. In the above instance, what might a Data Engineer have asked? In a data-driven world, could the airport shutdowns have been prevented? Would scenarios and algorithms have helped to scale corrections faster?  These are just a few of the questions a data professional might ask. But, in order to provide solutions, first you need a builder – a data engineer. You don’t always have to build from the ground up, though. Sometimes, reconstruction and refurbishment are just what’s needed to bring a project back to its former glory.  Digital Transformation in Travel  As travel companies prepare for the upcoming Digital Travel Summit at the end of this month, they’ve released a report targeting certain areas on which to focus. Not the least of which is the need for stronger data analytics with less than half reporting they have a plan in place but are looking to improve it.  One of the main areas of focus according to the report is lack of programming skills coupled with a hesitant approach by management. The complexity and difficulty of trying to analyse and personalise the sheer amount of consumer data available is also a factor.  And without the right tools, it’s even more difficult. Knowing what data is useful and what is isn’t is a major challenge, as well as the ability to track large amounts of data in real time can be a daunting task. These challenges and more, offer the perfect opportunity for a seasoned Data Engineer to step in, and take on a leadership role.  Take the Next the Step  Are you a Data Engineer ready to take your career to the next level? Are you a wizard at big data technologies with a leadership bent? Then we may have a role for you. A leading e-commerce company is looking to transform their data infrastructure (Python, AWS, Airflow) by hiring a senior data engineer.  Check out our other current vacancies or contact Joshua Carter, Recruitment Consultant with a Data Engineering focus at +44 20 8408 6070 or email joshuacarter@harnham.com to learn more.

Joshua Carter

07. June 2018

Recently Viewed jobs