Big Data Jobs in New York

Big Data keeps getting bigger, in fact we only at the start of an industry that will affect every business and person alive.

Get in early and make your career in big data, start your search today for our latest Big Data Jobs based in New York, apply online now.

Latest Jobs

Salary

US$200000 - US$220000 per year

Location

Boston, Massachusetts

Description

A global technology company are looking for a Big Data Engineering Manager to join them in the heart of Boston! Click below to find out more!

Salary

US$140000 - US$170000 per year

Location

Boston, Massachusetts

Description

Senior Data Architect

Salary

US$130000 - US$150000 per year

Location

Boston, Massachusetts

Description

One of the fastest growing car rental services is on the market for a well versed data engineer to immediately contribute the company in the heart of Boston.

Salary

US$150000 - US$170000 per year

Location

Boston, Massachusetts

Description

Senior Data Engineer

Salary

US$160000 - US$180000 per year + 1.5% equity

Location

New York

Description

One of the most exciting start-ups in NewYork are looking for a Lead Software Engineer with Ruby on Rails to join them! Click below to read more!

Harnham blog & news

With over 10 years experience working solely in the Data & Analytics sector our consultants are able to offer detailed insights into the industry.

Visit our Blogs & News portal or check out our recent posts below.

Data Engineer Or Software Engineer: What Does Your Business Need?

We are in a time in which what we do with Data matters. Over the last few years, we have seen a rapid rise in the number of Data Scientists and Machine Learning Engineers as businesses look to find deeper insights and improve their strategies. But, without proper access to the right Data that has been processed and massaged, Data Scientists and Machine Learning Engineers would be unable to do their job properly.   So who are the people who work in the background and are responsible to make sure all of this works? The quick answer is Data Engineers!... or is it? In reality, there are two similar, yet different profiles who can help help a company achieve their Data-driven goals.  Data Engineers  When people think of Data Engineers, they think of people who make Data more accessible to others within an organization. Their responsibility is to make sure the end user of the Data, whether it be an Analyst, Data Scientist, or an executive, can get accurate Data from which the business can make insightful decisions. They are experts when it comes to data modeling, often working with SQL.  Frequently, “modern” Data Engineers work with a number of tools including Spark, Kafka, and AWS (or any cloud provider), whilst some newer Databases/Data Warehouses include Mongo DB and Snowflake. Companies are choosing to leverage these technologies and update their stack because it allows Data teams to move at a much faster pace and be able to deliver results to their stakeholders.   An enterprise looking for a Data Engineer will need someone to focus more on their Data Warehouse and utilize their strong knowledge of querying information, whilst constantly working to ingest/process Data. Data Engineers also focus more on Data Flow and knowing how each Data sets works in collaboration with one another.    Software Engineers - Data Similar to a Data Engineers, Software Engineers - Data ( who I will refer to as Software Data Engineers in this article) also build out Data Pipelines. These individuals might go by different names like Platform or Infrastructure Engineer. They have to be good with SQL and Data Modeling, working with similar technologies such as Spark, AWS, and Hadoop. What separates Software Data Engineers from Data Engineers is the necessity to look at things from a macro-level. They are responsible for building out the cluster manager and scheduler, the distributed cluster system, and implementing code to make things function faster and more efficiently.  Software Data Engineers are also better programers. Frequently, they will work in Python, Java, Scala, and more recently, Golang. They also work with DevOps tools such as Docker, Kubernetes, or some sort of CI/CD tool like Jenkins. These skills are critical as Software Data Engineers are constantly testing and deploying new services to make systems more efficient.   This is important to understand, especially when incorporating Data Science and Machine Learning teams. If Data Scientists or Machine Learning Engineers do not have a strong Software Engineers in place to build their platforms, the models they build won’t be fully maximized. They also have to be able to scale out systems as their platform grows in order to handle more Data, while finding ways to make improvements. Software Data Engineers will also be looking to work with Data Scientists and Machine Learning Engineers in order to understand the prerequisites of what is needed to support a Machine Learning model.   Which is right for your business?  If you are looking for someone who can focus extensively on pulling Data from a Data source or API, before transforming or “massaging” the Data, and then moving it elsewhere, then you are looking for a Data Engineer. Quality Data Engineers will be really good at querying Data and Data Modeling and will also be good at working with Data Warehouses and using visualization tools like Tableau or Looker.   If you need someone who can wear multiple hats and build highly scalable and distributed systems, you are looking for a Software Data Engineer. It's more common to see this role in smaller companies and teams, since Hiring Managers often need someone who can do multiple tasks due to budget constraints and the need for a leaner team. They will also be better coders and have some experience working with DevOps tools. Although they might be able to do more than a Data Engineer, Software Data Engineers may not be as strong when it comes to the nitty gritty parts of Data Engineering, in particular querying Data and working within a Data Warehouse.  It is always a challenge knowing which type of job to recruit for. It is not uncommon to see job posts where companies advertise that they are looking for a Data Engineer, but in reality are looking for a Software Data Engineer or Machine Learning Platform Engineer. In order to bring the right candidates to your door, it is crucial to have an understanding of what responsibilities you are looking to be fulfilled. That's not to say a Data Engineer can't work with Docker or Kubernetes. Engineers are working in a time where they need to become proficient with multiple tools and be constantly honing their skills to keep up with the competition. However, it is this demand to keep up with the latest tech trends and choices that makes finding the right candidate difficult. Hiring Managers need to identify which skills are essential for the role from the start, and which can be easily picked up on the job. Hiring teams should focus on an individual's past experience and the projects they have worked on, rather than looking at their previous job titles.  If you're looking to hire a Data Engineer or a Software Data Engineer, or to find a new role in this area, we may be able to help.  Take a look at our latest opportunities or get in touch if you have any questions. 

AR & VR Breathe Life Back Into Our Lives

AR & VR Breathe Life Back Into Our Lives

Remember the bicycles piled in the yard that showed where your friends were? What about the jingle of coins in your pocket as you headed to the arcade? Stranger Things may have recalled these distant memories, but what if you could still see bicycles piled against your neighbors’ door or listen to the jingle of coins as you played with your friends? This is Augmented Reality and its aim now is to get you off the couch, exercising, playing, and enjoying camaraderie. Ready, Player One? Five Trends to Watch As we navigate the next stage of our virtual and augmented realities, there are a few trends to watch. AR, VR, and AI Will “See” Objects - Augmented Reality (AR), combined with Artificial Intelligence (AI) and Computer Vision, will help computers “see” and label what’s being seen. Machine Learning will ramp up to offer increasingly correct identification of objects whether it’s the dinosaur exhibit at the natural history museum or connects the dots for stargazing in a planetarium.VR Gets More “Real” for the Mainstream – In other words, developers are able to offer more immersive experiences. New developments in hardware technology, such as eyeball tracking and field-of-view help power the idea users can interact and explore less like a video game and more like real-life.AR Can Help Keep Your Eyes on the Road – Remember KIT? The car who talked, could self-diagnose, and navigated with barely a hand on the wheel? It’s not so far-fetched now as it once seemed as vehicle manufacturers increasingly opt for voice assistants and some begin to offer graphics’ overlay of footage around the car. Others go a step farther, projecting data onto the car windshield, assisting with navigation, lane identification, and potential hazards along the way.AR-Based Entertainment Branches Out – No more piled high pizza boxes, sitting in the dark of your parent’s basement. AR today is about getting moving; physical activity and balancing tech life with real life. Toys and Gaming companies are on the bandwagon to get families moving. Location-based gaming has grown by leaps and bounds as has interactive projection-based technology. Some applications can transform your immediate environment into an immersive gaming experience while others can transform playgrounds which, when unlocked by parents, can offer choose-your-own adventure types of projects to complete. Enter whole being of mind, body, spirit. And we’ve barely scratched the surface.Worldbuilding, Personalized Character Building, and Augmented Board Games – Each of these are about bridging the gap between the physical and digital world and is most changed when it comes to board games.  As exciting as these trends are, it’s not all fun and games when it comes to the next advances of AR. This technology is also being used to help discover early onset of Alzheimer’s and most recently has found its way into the operating room. Or at least, it will be soon as companies look to bring the AR and VR technologies into the OR. Training and Teaching with AR AR & VR technologies are being used for teaching and training in just about every industry it seems. From Walmart to the Military, business is seeing the benefits of allowing individuals to practice their roles without the associated real-world risks. But when it comes to the healthcare industry, this is where AR begins to really shine. What if we could predict not only detect early onset dementia, but catch the onset of mental illness as well? One company, recently cleared by the FDA, is working toward just that, with the specific goal of predicting Alzheimer’s early. With an estimated cost of around $290 billion which could rise to over $1 Trillion by 2050, their claims of 94% accuracy in detection 6 to 10 years early is good news for families. Memories are how we share our stories and no family wants to miss theirs. Could technology as disparate as it once made us be bringing us closer together? With these latest, that just may be the case. So, whether you want to let out your inner child, practice a new skill without real-world ramifications, or simply know your memories will be with you for a lifetime, AR offers something for everyone. Every industry. Every business. It has infused our world and is breathing life back into our lives.  Want to be the wizard behind the curtain, the Data Analyst to know what’s next, or the Data Engineer who builds the next great technological advance in the AR spectrum of services and capabilities? We may have a role for you. Check out our current vacancies or contact one of our recruitment consultants to learn more. For our West Coast Team, call (415) 614 - 4999 or send an email to sanfraninfo@harnham.com.  For our Mid-West and East Coast Teams, call (212) 796 - 6070 or send an email to newyorkinfo@harnham.com.

Recently Viewed jobs