Harnham
SEARCH
Shortlist

Lead Data Engineer and Technical Consultant

New York
US$170000 - US$200000 per annum + Additional Benefits
Apply Save job

Lead Data Engineer + Technical Consultant

New York, New York

$170,000-$200,000 base annual salary + additional benefits

THE COMPANY

This data-forward tech startup is looking for an intelligent, motivated, multi-talented professional to join their NYC team as a Lead Data Engineer and Technical Consultant full time!

If you are someone who looks forward to coming into work wearing multiple hats and facing varying day to day schedules working with cutting edge technologies, read below and apply today!

THE ROLE

As the Lead Data Engineer and Technical Consultant, you will be taking ownership of a hybrid data environment to support client data needs by specifically partaking in the following key tasks:

  • Designing and building data pipelines from scratch primarily coding in Python, also some Scala and Java
  • Building and automating data platform infrastructure using Docker, Kubernetes, and Jenkins
  • Demo-ing and explaining back end tech products' value to prospect clients
  • Catering to individual client data engineering needs

YOUR SKILLS AND EXPEIRENCE

In order to be considered for the position of Lead Data Engineer and Technical Consultant, you are someone with the following background and skillset:

  • You have a Bachelor's degree, preferably in Computer Science or a related field
  • You are preferably coming out of/have had experience working in a start up environment where you have had to wear multiple hats
  • You have extensive hands-on commercial experience as a data engineer
  • You have strong technical capabilities writing Python code and using Spark and Airflow
  • You are commercially familiar working with DevOps tools such as Docker, Kubernetes, and Jenkins
  • You are able to translate business needs into technical code and vice versa
  • You have strong stakeholder relations with nontechnical business professionals

THE BENEFITS

  • $170,000-$200,000 base annual salary
  • Annual performance-based bonus and stock options offered varying according to experience level
  • Medical, Dental, Vision Insurance
  • 401K
  • PTO and STO days
  • WFH flexibility, remote options till 2021

****Unfortunately, this client is unable to offer sponsorship/transfer of sponsorship for those who will need it now or in the future.****

HOW TO APPLY

Please register your interest by sending your résumé to Kavya Kannan via the Apply link on this page.

KEYWORDS

Big Data, Data Engineering, Data Pipeline, Data Architecture, Design, Python, SQL, AWS, Real Time, Streaming, Batch Processing, ETL, ELT, Data Formatting, Data Structure, Data Modeling, Data Governance, Data Cleansing, DevOps, CI/CD, GCP, Azure, Kubernetes, Jenkins, Docker, Ansible, Terraform, Presales

Apply Save job
Send similar jobs by email
99129/KK16
New York
US$170000 - US$200000 per annum + Additional Benefits
  1. Permanent
  2. Big Data

Similar Jobs

Data Engineer
Salary

£60000 - £70000 per annum

Location

London

Description

Large Health insurance company looking for talented Data Engineers

ADD TO SHORTLIST
Senior Data Engineer
Salary

US$130000 - US$140000 per annum

Location

Cincinnati, Ohio

Description

Looking for Senior Data Engineers to design/build data pipelines in a multi-cloud environment for data science teams.

ADD TO SHORTLIST
Azure Data Engineer
Salary

£60000 - £70000 per annum

Location

London

Description

Large Health insurance company looking for talented Data Engineers

ADD TO SHORTLIST
Senior Data Engineer - CPG eCommerce
Salary

US$120000 - US$130000 per annum + Additional Benefits

Location

Cincinnati, Ohio

Description

A CPG business in Cincinnati are looking for passionate and energetic seasoned senior data engineers to join their business!

ADD TO SHORTLIST

Harnham blog & news

With over 10 years experience working solely in the Data & Analytics sector our consultants are able to offer detailed insights into the industry.

Visit our Blogs & News portal or check out our recent posts below.

The Search For Toilet Paper: A Q&A With The Data Society

We recently spoke Nisha Iyer, Head of Data Science, and Nupur Neti, a Data Scientist from Data Society.  Founded in 2014, Data Society consult and offer tailored Data Science training for businesses and organisations across the US. With an adaptable back-end model, they create training programs that are not only tailored when it comes to content, but also incorporate a company’s own Data to create real-life situations to work with.  However, recently they’ve been looking into another area: toilet paper.  Following mass, ill-informed, stock-piling as countries began to go into lockdown, toilet paper became one of a number of items that were suddenly unavailable. And, with a global pandemic declared, Data Society were one of a number of Data Science organisations who were looking to help anyway they could.  “When this Pandemic hit, we began thinking how could we help?” says Iyer. “There’s a lot of ways Data Scientists could get involved with this but our first thought was about how people were freaking out about toilet paper. That was the base of how we started, as kind of a joke. But then we realised we already had an app in place that could help.” The app in question began life as a project for the World Central Kitchen (WCK), a non-profit who help support communities after natural disasters occur.  With the need to go out and get nutritionally viable supplies upon arriving at a new location, WCK teams needed to know which local grocery stores had the most stock available.  “We were working with World Central Kitchen as a side project. What we built was an app that supposed to help locate resources during disasters. So we already had the base done.” The app in question allows the user to select their location and the products they are after. It then provides information on where you can get each item, and what their nutritional values are, with the aim of improving turnaround time for volunteers.  One of the original Data Scientists, Nupur Neti, explained how they built the platform: “We used a combination of R and Python to build the back-end processing and R Shiny to build the web application. We also included Google APIs that took your location and could find the closest store to you. Then, once you have the product and the sizes, we had an internal ranking algorithm which could rank the products selected based on optimisation, originally were based on nutritional value.”  The team figured that the same technology could help in the current situation, ranking based on stock levels rather than nutritional value. With an updated app, Iyer notes “People won’t have to go miles and stand in lines where they are not socially distancing. They’ll know to visit a local grocery store that does have what they need in stock, that they’ve probably not even thought of before.” However, creating an updated version presented its own challenges. Whereas the WCK app utilised static Data, this version has to rely on real-time Data. Unfortunately this isn’t as easy to come by, as Iyer knows too well:  “When we were building this for the nutrition app we reached out to groceries stores and got some responses for static Data. Now, we know there is real-time Data on stock levels because they’re scanning products in and out. Where is that inventory though? We don’t know.” After putting an article out asking for help finding live Data, crowdsourcing app OurStreets got in touch. They, like Data Society, were looking to help people find groceries in short supply. But, with a robust front and back-end in place, the app already live, and submissions flying in across the States, they were looking for a Data Science team who could make something of their findings.  “We have the opportunity,” says Iyer “to take the conceptual ideas behind our app and work with OurStreets robust framework to create a tool that could be used nationwide.” Before visiting a store, app users select what they are looking for. This allows them to check off what the store has against their expectations, as well as uploading a picture of what is available. They can also report on whether the store is effectively practising social distancing. Neti explains, that this Data holds lots of possibilities for their Data Science team: “Once we take their Data, our system will clean any submitted text using NLP and utilise image recognition on submitted pictures using Deep Learning. This quality Data, paired with the Social Distancing information, will allow us to gain better insights into how and what people are shopping for. We’ll then be able to look at trends, see what people are shopping for and where. Ultimately, it will also allow us to make recommendations as to where people should then go if they are looking for a product.”  In addition to crowdsourced information, Data Society are still keen to get their hands on any real-time Data that supermarkets have to offer. If you know where they could get their hands on it, you can get in touch with their team.  Outside of their current projects, Iyer remains optimistic for the world when it emerges from the current situation: “Things will return to normal. As dark a time as this is, I think it’s going to exemplify why people need to use Artificial Intelligence and Data Science more. If this type of app is publicised during the Coronavirus, maybe more people will understand the power of what Data and Data Science can do and more companies that are slow adaptors will see this and see how it could be helpful to their industry.”   If you want to make the world a better place using Data, we may have a role for you, including a number of remote opportunities. Or, if you’re looking to expand and build out your team with the best minds in Data, get in touch with one of expert consultants who will be able to advise on the best remote and long-term processes. 

Nick Mandella

Nick Mandella

14. April 2020

News

Why You Should Always Be Learning In Data Science: Tips From Kevin Tran

Last month we sat down with Kevin Tran, a Senior Data Scientist at Stanford University, to chat about Data Science trends, improvements in the industry, and his top tips for success in the market.  As one of LinkedIn’s Top Voices of 2019 within Data & Analytics. his thoughts on the industry regularly garner hundreds of responses, with debates and discussions bubbling up in the comments from colleagues eager to offer their input.  This online reputation has allowed him to make a name for himself, building out his own little corner of the internet with his expertise. But for Tran, it’s never been about popularity. “It’s not about the numbers,” he says without hesitation. “I don’t care about posting things just to see the number of likes go up.” His goal is always connection, to speak with others and learn from them while teaching from his own background. He’s got plenty of stories from his own experiences. For him, sharing is a powerful way to lead others down a path he himself is still discovering.  When asked about the most important lesson he’s learned in the industry, he says it all boils down to staying open to new ideas.  “You have to continue to learn, and you have to learn how to learn. If you stop learning, you’ll become obsolete pretty soon, particularly in Data Science. These technologies are evolving every day. Syntax changes, model frameworks change, and you have to constantly keep yourself updated.”  He believes that one of the best ways to do that is through open discussion. His process is to share in order to help others. When he has a realisation, he wants to set it in front of others to pass along what he’s learned; he wants to see how others react to the same problem, if they agree or see a different angle. It’s vital to consider what you needed to know at that stage. Additionally, this exchange of ideas allows Tran to learn from how others tackle the same problems, as well as get a glimpse into other challenges he may have not yet encountered.  “When I mentor people, I’m still learning, myself,” Tran confesses. “There’s so much out there to learn, you can’t know it all. Data Science is so broad." At the end of the day, it all comes down to helping each other and bringing humanity back to the forefront. In fact, this was his biggest advice for both how to improve the industry and how to succeed in it. It’s a point he comes back to with some regularity in his writing. “It doesn’t matter how smart you are, stay humble and respect everyone,” one post reads. “Everyone can teach you something you don’t know.” Treating people well, understanding their needs, and consciously working to see them as people instead of numbers or titles—this, Tran argues, is how you succeed in the business. To learn and grow, you must work with people, especially people with different skills and mindsets. Navigating your career is not all technical, even in the world of Data. “The thing that cannot be automated is having a heart,” he tells me sagely. Beyond this, Tran stresses the need for a solid foundation. The one thing you can’t afford to do is take shortcuts. You have to learn the practicalities and how to apply them, but to be strong in theory as well.  Understanding what is happening underneath the code will keep you moving forward. He compares knowing the tools to learning math with a calculator. “If you take the calculator away, you still need to be able to do the work. You need the underlying skills too, so that when you’re in a situation without the calculator, you can still provide solutions.” By constantly striving to collaborate and improve, Tran believes the Data industry has the best chance of innovating successfully.  If you’re looking for a new challenge in an innovative and collaborative environment, we may have a role for you. Take a look at our latest opportunities or get in touch with one of our expert consultants to find out more. 

William Reichert

William Reichert

19. March 2020

News

Recently Viewed jobs