Harnham
SEARCH
Shortlist

Big Data Jobs

Get the most out of your data

Big Data has become a crucial component to countless businesses and those who decide to pursue a career within it face challenges including analysis, capture and visualization of data sets too large for traditional data processing applications.

Recruitment is not just about filling vacancies, it’s about creating lasting relationships while helping to advance the industry we work in. At Harnham, we aim to place only the best talent whilst ensuring our clients and candidates have positive experiences.

Whilst businesses may have vast quantities of data at the ready, it holds no real value unless it can be stored, harnessed and utilized. It’s estimated that most US companies have at least 100 terabytes of data stored which could be used for various purposes around the business. This is where those who work in Big Data become extremely valuable, as they develop the strategies, techniques and digital analytics architecture to organize the information.

At Harnham, we are constantly trying to assist the development of the data and analytics industry by constantly providing our own analysis. We work with the sector to produce annual salary guides based on our candidates and clients data.

Latest Jobs

Lead Data Engineer - Fortune 500
Salary

US$440 - US$520 per day + Additional Benefits

Location

New York

Description

Hi All, I'm currently recruiting for this position. Please click on the job title below to view the Job Description and apply to it!

ADD TO SHORTLIST
Principal Software Engineer
Salary

US$185000 - US$210000 per annum

Location

San Francisco, California

Description

Work with one of the largest Hadoop clusters in the world as our client looks to migrate PBs of data to empower their client's customers.

ADD TO SHORTLIST
Big Data Engineer
Salary

US$120000 - US$150000 per annum + Unlimited PTO - 401k Match

Location

New York

Description

Big Data Engineer

ADD TO SHORTLIST
LEAD DATA ENGINEER
Salary

US$170000 - US$200000 per year + Bonus

Location

San Francisco, California

Description

The Lead Data Engineer will work at a global leader in their field who due to an increase in business are looking to grow their Data Engineering team.

ADD TO SHORTLIST
Data Engineer
Salary

US$130000 - US$150000 per annum + Benefits

Location

Detroit, Michigan

Description

An IOT technology company are looking for a Senior Data Engineer to join them in the heart of Detroit! Click below to read more!

ADD TO SHORTLIST

Harnham blog & news

With over 10 years experience working solely in the Data & Analytics sector our consultants are able to offer detailed insights into the industry.

Visit our Blogs & News portal or check out our recent posts below.

Privacy Is Big Business For Small And Mid-Size Businesses

Privacy is Big Business for Small and Mid-Size Businesses

If you’re a small to mid-size business and think cyber criminals only go after big business; think again. It’s just as important, if not more important for you to have privacy plans in place. This goes way beyond GDPR and state-to-state rules, this is about how you care for your customers personal information. The return on investment will set the tone for future years of your business. After all, according to a 2018 report by Verizon, 58% of cyber-attacks targeted small business. While it may seem counter-intuitive and larger businesses are bigger fish to go after, they can be difficult to get into. After all, they’ve got the resources to protect their customer’s Data and are hyper aware of what it can be to their business if they don’t. Smaller and mid-size businesses generally don’t have the resources of the larger businesses, and may not focus on cybersecurity like they should which leaves their business wide open for cybercriminals. Chinks in the Armor of Your Data Cybercriminals excel at finding “chinks in the armor” of your Data. They’ll use any advantage to break in from the usual hacking and malware to physical breaches. One improperly secured device can be just the entry they need into your entire system.  What can you do? Be focused in your approach to Data security. Many small businesses tend to put out fires, rather than have a focused strategy. And each approach to tighten security can lead to more opportunity for hacking.Communicate your strategy to every member of your team. Something as small as clicking on the wrong link can lead to a Data breach.Train your staff on measures they can take such as to not click on a link they’re not expecting, to check email addresses and ensure they’re approved or white-listed as okay to access. The more aware your staff are, the better able they’ll be able to help ensure the security of your business’ Data. While staff may be on the front lines, this also requires a commitment from senior executives as well. Understand that just because you’re not dealing in billions of dollars, you may actually be at greater risk. Why? Because unlike the larger companies, your business may not survive the fallout of a cyber-attack. How to Protect Your SMB You can protect your business by creating a Data Security Strategy and consider the following: Encrypt your data;Authenticate your users by either a 2-step verification process or having them enter some kind of code;Authorize access to trusted sources. Encrypting Data helps protect the private and sensitive information and makes it unreadable without the correct key. To ensure only those who are trusted sources have access is through authentication.  Authentication can include username/password, code, tokens, phone number, and image association such as click only the boxes with pictures of street lights or stop signs. This helps your business control who has access and gives you tighter rein over who sees sensitive information and what they can do with it.  By defining the rules and regulations of access to information, training your employees to be aware and what to do to ensure security, you can strike a balance of increased security and transparency to your customers. In other words, the efforts you go through to protect their Data will put you ahead of the competition as you make inroads toward a Data privacy strategy while others take action as things happen. One Final Thought Ensuring your business’ Data is protected and detecting times when it may have been breached is increasingly important to help minimize damage. One issue SMBs face is that it may take longer to detect if there isn’t a Data security plan in place. The more quickly you can detect an issue, the more quickly you can reduce its impact and the more quickly and effectively you can respond, the better.  Interestingly, smaller businesses tend to have a better overall picture of their assets than larger businesses. This can be a boon when you communicate your new cybersecurity strategy to your teams and offers a significant return on investment of your resources. If you’re interested in Big Data and Analytics, we may have a role for you. Take a look at our current vacancies or contact one of our recruitment consultants to learn more.  For our West Coast Team, call (415) 614 - 4999 or send an email to sanfraninfo@harnham.com.  For our Mid-West and East Coast Teams, call (212) 796 - 6070 or send an email to newyorkinfo@harnham.com.

Sandra Namatovu

Sandra Namatovu

14. November 2019

Data Engineer Or Software Engineer: What Does Your Business Need?

We are in a time in which what we do with Data matters. Over the last few years, we have seen a rapid rise in the number of Data Scientists and Machine Learning Engineers as businesses look to find deeper insights and improve their strategies. But, without proper access to the right Data that has been processed and massaged, Data Scientists and Machine Learning Engineers would be unable to do their job properly.   So who are the people who work in the background and are responsible to make sure all of this works? The quick answer is Data Engineers!... or is it? In reality, there are two similar, yet different profiles who can help help a company achieve their Data-driven goals.  Data Engineers  When people think of Data Engineers, they think of people who make Data more accessible to others within an organization. Their responsibility is to make sure the end user of the Data, whether it be an Analyst, Data Scientist, or an executive, can get accurate Data from which the business can make insightful decisions. They are experts when it comes to data modeling, often working with SQL.  Frequently, “modern” Data Engineers work with a number of tools including Spark, Kafka, and AWS (or any cloud provider), whilst some newer Databases/Data Warehouses include Mongo DB and Snowflake. Companies are choosing to leverage these technologies and update their stack because it allows Data teams to move at a much faster pace and be able to deliver results to their stakeholders.   An enterprise looking for a Data Engineer will need someone to focus more on their Data Warehouse and utilize their strong knowledge of querying information, whilst constantly working to ingest/process Data. Data Engineers also focus more on Data Flow and knowing how each Data sets works in collaboration with one another.    Software Engineers - Data Similar to a Data Engineers, Software Engineers - Data ( who I will refer to as Software Data Engineers in this article) also build out Data Pipelines. These individuals might go by different names like Platform or Infrastructure Engineer. They have to be good with SQL and Data Modeling, working with similar technologies such as Spark, AWS, and Hadoop. What separates Software Data Engineers from Data Engineers is the necessity to look at things from a macro-level. They are responsible for building out the cluster manager and scheduler, the distributed cluster system, and implementing code to make things function faster and more efficiently.  Software Data Engineers are also better programers. Frequently, they will work in Python, Java, Scala, and more recently, Golang. They also work with DevOps tools such as Docker, Kubernetes, or some sort of CI/CD tool like Jenkins. These skills are critical as Software Data Engineers are constantly testing and deploying new services to make systems more efficient.   This is important to understand, especially when incorporating Data Science and Machine Learning teams. If Data Scientists or Machine Learning Engineers do not have a strong Software Engineers in place to build their platforms, the models they build won’t be fully maximized. They also have to be able to scale out systems as their platform grows in order to handle more Data, while finding ways to make improvements. Software Data Engineers will also be looking to work with Data Scientists and Machine Learning Engineers in order to understand the prerequisites of what is needed to support a Machine Learning model.   Which is right for your business?  If you are looking for someone who can focus extensively on pulling Data from a Data source or API, before transforming or “massaging” the Data, and then moving it elsewhere, then you are looking for a Data Engineer. Quality Data Engineers will be really good at querying Data and Data Modeling and will also be good at working with Data Warehouses and using visualization tools like Tableau or Looker.   If you need someone who can wear multiple hats and build highly scalable and distributed systems, you are looking for a Software Data Engineer. It's more common to see this role in smaller companies and teams, since Hiring Managers often need someone who can do multiple tasks due to budget constraints and the need for a leaner team. They will also be better coders and have some experience working with DevOps tools. Although they might be able to do more than a Data Engineer, Software Data Engineers may not be as strong when it comes to the nitty gritty parts of Data Engineering, in particular querying Data and working within a Data Warehouse.  It is always a challenge knowing which type of job to recruit for. It is not uncommon to see job posts where companies advertise that they are looking for a Data Engineer, but in reality are looking for a Software Data Engineer or Machine Learning Platform Engineer. In order to bring the right candidates to your door, it is crucial to have an understanding of what responsibilities you are looking to be fulfilled. That's not to say a Data Engineer can't work with Docker or Kubernetes. Engineers are working in a time where they need to become proficient with multiple tools and be constantly honing their skills to keep up with the competition. However, it is this demand to keep up with the latest tech trends and choices that makes finding the right candidate difficult. Hiring Managers need to identify which skills are essential for the role from the start, and which can be easily picked up on the job. Hiring teams should focus on an individual's past experience and the projects they have worked on, rather than looking at their previous job titles.  If you're looking to hire a Data Engineer or a Software Data Engineer, or to find a new role in this area, we may be able to help.  Take a look at our latest opportunities or get in touch if you have any questions. 

Bryan Minami

Bryan Minami

26. September 2019

Recently Viewed jobs