Hadoop Jobs in New York

Search for our latest Hadoop Jobs based in New York, apply online today here.

Latest Jobs

Salary

US$180000 - US$220000 per annum + Additional Benefits

Location

New Jersey

Description

Hi All, I'm currently recruiting for this position. Please click on the job title below to view the Job Description and apply to it!

Salary

US$120000 - US$150000 per annum + Unlimited PTO - 401k Match

Location

New York

Description

Big Data Engineer

Salary

US$134117 - US$146310 per annum + Benefits

Location

New York

Description

An AI technology company are looking for a Data Engineer to join them in the heart of NYC! Click below to apply!

Salary

US$180000 - US$190000 per annum

Location

New York

Description

A hip start up company in the real estate industry is looking to for a talented Lead Data Engineer to manage the company's data platform!

Salary

US$110000 - US$130000 per annum + Additional Benefits

Location

Englewood Cliffs, New Jersey

Description

Hi All, I'm currently recruiting for this position. Please click on the job title below to view the Job Description and apply to it!

Harnham blog & news

With over 10 years experience working solely in the Data & Analytics sector our consultants are able to offer detailed insights into the industry.

Visit our Blogs & News portal or check out our recent posts below.

Why You Should Always Be Learning In Data Science: Tips From Kevin Tran

Last month we sat down with Kevin Tran, a Senior Data Scientist at Stanford University, to chat about Data Science trends, improvements in the industry, and his top tips for success in the market.  As one of LinkedIn’s Top Voices of 2019 within Data & Analytics. his thoughts on the industry regularly garner hundreds of responses, with debates and discussions bubbling up in the comments from colleagues eager to offer their input.  This online reputation has allowed him to make a name for himself, building out his own little corner of the internet with his expertise. But for Tran, it’s never been about popularity. “It’s not about the numbers,” he says without hesitation. “I don’t care about posting things just to see the number of likes go up.” His goal is always connection, to speak with others and learn from them while teaching from his own background. He’s got plenty of stories from his own experiences. For him, sharing is a powerful way to lead others down a path he himself is still discovering.  When asked about the most important lesson he’s learned in the industry, he says it all boils down to staying open to new ideas.  “You have to continue to learn, and you have to learn how to learn. If you stop learning, you’ll become obsolete pretty soon, particularly in Data Science. These technologies are evolving every day. Syntax changes, model frameworks change, and you have to constantly keep yourself updated.”  He believes that one of the best ways to do that is through open discussion. His process is to share in order to help others. When he has a realisation, he wants to set it in front of others to pass along what he’s learned; he wants to see how others react to the same problem, if they agree or see a different angle. It’s vital to consider what you needed to know at that stage. Additionally, this exchange of ideas allows Tran to learn from how others tackle the same problems, as well as get a glimpse into other challenges he may have not yet encountered.  “When I mentor people, I’m still learning, myself,” Tran confesses. “There’s so much out there to learn, you can’t know it all. Data Science is so broad." At the end of the day, it all comes down to helping each other and bringing humanity back to the forefront. In fact, this was his biggest advice for both how to improve the industry and how to succeed in it. It’s a point he comes back to with some regularity in his writing. “It doesn’t matter how smart you are, stay humble and respect everyone,” one post reads. “Everyone can teach you something you don’t know.” Treating people well, understanding their needs, and consciously working to see them as people instead of numbers or titles—this, Tran argues, is how you succeed in the business. To learn and grow, you must work with people, especially people with different skills and mindsets. Navigating your career is not all technical, even in the world of Data. “The thing that cannot be automated is having a heart,” he tells me sagely. Beyond this, Tran stresses the need for a solid foundation. The one thing you can’t afford to do is take shortcuts. You have to learn the practicalities and how to apply them, but to be strong in theory as well.  Understanding what is happening underneath the code will keep you moving forward. He compares knowing the tools to learning math with a calculator. “If you take the calculator away, you still need to be able to do the work. You need the underlying skills too, so that when you’re in a situation without the calculator, you can still provide solutions.” By constantly striving to collaborate and improve, Tran believes the Data industry has the best chance of innovating successfully.  If you’re looking for a new challenge in an innovative and collaborative environment, we may have a role for you. Take a look at our latest opportunities or get in touch with one of our expert consultants to find out more. 

Data Engineer Or Software Engineer: What Does Your Business Need?

We are in a time in which what we do with Data matters. Over the last few years, we have seen a rapid rise in the number of Data Scientists and Machine Learning Engineers as businesses look to find deeper insights and improve their strategies. But, without proper access to the right Data that has been processed and massaged, Data Scientists and Machine Learning Engineers would be unable to do their job properly.   So who are the people who work in the background and are responsible to make sure all of this works? The quick answer is Data Engineers!... or is it? In reality, there are two similar, yet different profiles who can help help a company achieve their Data-driven goals.  Data Engineers  When people think of Data Engineers, they think of people who make Data more accessible to others within an organization. Their responsibility is to make sure the end user of the Data, whether it be an Analyst, Data Scientist, or an executive, can get accurate Data from which the business can make insightful decisions. They are experts when it comes to data modeling, often working with SQL.  Frequently, “modern” Data Engineers work with a number of tools including Spark, Kafka, and AWS (or any cloud provider), whilst some newer Databases/Data Warehouses include Mongo DB and Snowflake. Companies are choosing to leverage these technologies and update their stack because it allows Data teams to move at a much faster pace and be able to deliver results to their stakeholders.   An enterprise looking for a Data Engineer will need someone to focus more on their Data Warehouse and utilize their strong knowledge of querying information, whilst constantly working to ingest/process Data. Data Engineers also focus more on Data Flow and knowing how each Data sets works in collaboration with one another.    Software Engineers - Data Similar to a Data Engineers, Software Engineers - Data ( who I will refer to as Software Data Engineers in this article) also build out Data Pipelines. These individuals might go by different names like Platform or Infrastructure Engineer. They have to be good with SQL and Data Modeling, working with similar technologies such as Spark, AWS, and Hadoop. What separates Software Data Engineers from Data Engineers is the necessity to look at things from a macro-level. They are responsible for building out the cluster manager and scheduler, the distributed cluster system, and implementing code to make things function faster and more efficiently.  Software Data Engineers are also better programers. Frequently, they will work in Python, Java, Scala, and more recently, Golang. They also work with DevOps tools such as Docker, Kubernetes, or some sort of CI/CD tool like Jenkins. These skills are critical as Software Data Engineers are constantly testing and deploying new services to make systems more efficient.   This is important to understand, especially when incorporating Data Science and Machine Learning teams. If Data Scientists or Machine Learning Engineers do not have a strong Software Engineers in place to build their platforms, the models they build won’t be fully maximized. They also have to be able to scale out systems as their platform grows in order to handle more Data, while finding ways to make improvements. Software Data Engineers will also be looking to work with Data Scientists and Machine Learning Engineers in order to understand the prerequisites of what is needed to support a Machine Learning model.   Which is right for your business?  If you are looking for someone who can focus extensively on pulling Data from a Data source or API, before transforming or “massaging” the Data, and then moving it elsewhere, then you are looking for a Data Engineer. Quality Data Engineers will be really good at querying Data and Data Modeling and will also be good at working with Data Warehouses and using visualization tools like Tableau or Looker.   If you need someone who can wear multiple hats and build highly scalable and distributed systems, you are looking for a Software Data Engineer. It's more common to see this role in smaller companies and teams, since Hiring Managers often need someone who can do multiple tasks due to budget constraints and the need for a leaner team. They will also be better coders and have some experience working with DevOps tools. Although they might be able to do more than a Data Engineer, Software Data Engineers may not be as strong when it comes to the nitty gritty parts of Data Engineering, in particular querying Data and working within a Data Warehouse.  It is always a challenge knowing which type of job to recruit for. It is not uncommon to see job posts where companies advertise that they are looking for a Data Engineer, but in reality are looking for a Software Data Engineer or Machine Learning Platform Engineer. In order to bring the right candidates to your door, it is crucial to have an understanding of what responsibilities you are looking to be fulfilled. That's not to say a Data Engineer can't work with Docker or Kubernetes. Engineers are working in a time where they need to become proficient with multiple tools and be constantly honing their skills to keep up with the competition. However, it is this demand to keep up with the latest tech trends and choices that makes finding the right candidate difficult. Hiring Managers need to identify which skills are essential for the role from the start, and which can be easily picked up on the job. Hiring teams should focus on an individual's past experience and the projects they have worked on, rather than looking at their previous job titles.  If you're looking to hire a Data Engineer or a Software Data Engineer, or to find a new role in this area, we may be able to help.  Take a look at our latest opportunities or get in touch if you have any questions. 

Recently Viewed jobs