Senior Data Engineer |Manhattan, New York

Manhattan, New York
US$140000 - US$170000 per year

THE COMPANY

Harnham is working exclusively with a growing organization who is actively looking to hire a Senior Data Engineer out of their Manhattan office. This companies focus is on bringing the science of forecasting across industries. Their vision is to service clients from offices around the globe, employing only the best and the brightest data engineers, scientists, quants and researchers. The exponentially increasing amounts of data available to corporations and society today is astonishing. Using this companies quantitative and predictive talent will allow them to produce millions of forecasts based on thousands of data sets, resulting in the best possible predictions.

THE ROLE

As the Senior Data Engineer, you will be required to:

  • Play a key role in setting the technical direction as well as establishing the architecture as they build a robust data processing pipeline and the machine learning platform
  • As the lead of the Data Operations team, you will be tasked with architecting the data ingest process for multiple types of data.
  • Implementing secure, reliable and efficient processes
  • Your success will be evaluated in how fast the system will be able to ingest and process new data sources or updates to existing content.
  • Work alongside the CTO and the team of engineers

YOUR QUALIFICATIONS

  • Be able to make technology choices for the business
  • 4 Year College Degree in Computer Science or equivalent degree
  • Experience with Talend and Airflow or Beam and Airflow
  • 5+ years of experience developing large-scale enterprise solutions using agile, CICD and cloud
  • Naturally curious, detail-oriented person who can work in a highly collaborative environment

THE BENEFITS

Full-time, salaried position with a comprehensive benefits package. Compensation for this position is commensurate with experience.

HOW TO APPLY

Please register your interest by sending your resume to Thomas Daughtrey via the Apply link on this page.

Keywords

Data Engineering | CICD | ETL | Python | Agile | Airflow | AWS | Cloud | SQL | Beam | Talend | #ZR

Send similar jobs by email
H032519TD2
Manhattan, New York
US$140000 - US$170000 per year
  1. Permanent
  2. Big Data

Similar Jobs

Salary

Up to US$160000 per year

Location

Hanover, New Jersey

Description

Manager of Big Data | Hanover, NJ

Harnham blog & news

With over 10 years experience working solely in the Data & Analytics sector our consultants are able to offer detailed insights into the industry.

Visit our Blogs & News portal or check out our recent posts below.

The Culture Rush of Working in the Pacific Northwest

When you think of the Pacific Northwest, images of Pike’s Place at Seattle’s Waterfront and Starbucks’ birthplace may come immediately to mind. Or shoot down the I-5 Corridor toward the Silicon Forest, Oregon’s tech hub nod and nickname, a play on Silicon Valley just a little further South down the I-5 to San Francisco. One thing which may stand out immediately on the drive is the ocean vista view. Relax and unwind with a good cup of coffee at a local shop, bask in the mild climate, and curl up with a good book when you get a bit of free time. Here, the mantra is to ‘work hard, play harder’. The Land of Tech Seattle, also known as Emerald City, is one of the largest tech hubs of the eight in the US seeking top tech talent. Available jobs are outpacing the skilled workers to fill them. While they could outsource to other countries, many businesses are choosing to offer remote work options. Commuting Like a Local For most people, commuting involves keeping your car gassed up and maintained whether your office is a few miles down the road or in the next town. But, in the Pacific Northwest, commuting is part of an active lifestyle, while at the same time lowering your carbon footprint. How?  Well, with plenty of walking, hiking, and biking trails as well as a robust public transportation system, it’s a breeze to get to work. And you’ll be helping to support a sustainable environment as well. Biking is the most popular commuting option. If you’re a biker or plan to become one, be sure your outfitted with good reflective gear, a waterproof windbreaker, and of course, a helmet. If you’re going sans car, don’t forget a waterproof back. After all, you’ll need something to carry your stuff in, right?  Work/Life Balance Whether it's because of the temperate climate, the mild winters, or the laid-back atmosphere, the work/life balance in Washington and Oregon offer a peaceful setting to refresh and recharge. Available communities to spend time with like-minded people and the abundance of tech-focused jobs available in the area offer an amazing work/life balance. Spending Time in Nature – plenty of islands, parks, and trails offer a chance to commune with nature whether your commuting to and from work or planning for the weekend. The Arts - Portland and Seattle are both known for their arts and culture scenes including art festivals, indie bands, symphonies, museums, and more besides.Activities – From hiking to surfing to mountain climbing to yoga, there’s an activity for everyone to enjoy, meet people, and make friends.  One Final Takeaway… The quirky culture of the Pacific Northwest is an exercise in the unexpected. Whether listening to local bands, dressing up in costume at random, taking advantage of sunbreaks when the clouds part, or hiking in the rain, the reward is a good meal with good friends. If you’re new to the area, don’t be shy. Invite yourself along. Our brick-and-mortar locations are in San Francisco, New York, Berlin, and London, but we have clients around the world. Whether you’re looking for a permanent or a contract role, we’ve got you covered. Check out our current vacancies for opportunities or get in touch to learn more. For our West Coast Team, call (415) 614 - 4999 or send an email to sanfraninfo@harnham.com.   For our Mid-West and East Coast Teams, call (212) 796 - 6070 or send an email to newyorkinfo@harnham.com. 

Computer Vision: Keeping an Eye on You

Computer Vision: Keeping an Eye on You

5 Animal Farm, Fahrenheit 451, and 1984 all wove tales of Big Brother watching. Remember when only Superman had X-ray vision and leapt tall buildings in a single bound? Well, as big tech grapples with the cause and effect of its place when it comes to facial recognition, security, and object detection, there’s a new superhero in town. Computer Vision.  The Eyes Which Sweep Our World While we still can’t be everywhere or see everything at once, we have eyes everywhere. Despite security cameras, motion detectors, satellites, computers, and smartphones, there are still dangers we may miss. So, by making our machines and computers visually-enabled using Artificial Intelligence, we significantly ramp up what we can see and how fast we can process it. Still growing, Computer Vision remains in an infant stage, given that autonomous cars still can’t differentiate between a rock and paper bag or worse, a person and a static object. Despite this, there are plenty of startups on the scene working toward a solution. As Artificial Intelligence increasingly blends with Biometric technology, it lends itself more easily to image recognition, allowing computers to correctly match fingerprints and facial patterns. But, no longer is it just matching two images. Now, it’s being taught to learn the difference between static images and liveness. This could prove invaluable for: Spotting weaponsSuspicious behaviorsDangerous object detection Safety Begins at Home With products such as security cameras prevailing within the security industry, businesses are hard at work creating and improving their products using the latest technologies.  One such company is working to boost the clarity of their home security cameras. Think grainy gray, blurry images from night vision options or overly bright and distorted in the day. Their camera chip will be HDR and will be able to take much clearer pictures even in low-light. At home, you might find this product in a doorbell camera which could prove quite useful for smart homes which offer the option of allowing service people into your home from a remote app on your phone.  And these cameras aren’t limited to your front door: ATMsIndoor/Outdoor camerasCCTV CamerasNumber Plate Recognition Cameras Though cameras have played a role in all of these areas for some time, the idea now is to keep them from being hacked and causing damage on both a product and a personal level. And like any type of Artificial Intelligence recognition system, these camera applications are created with advanced features to protect against hacking as well super speed processing of whether or not an object is an animal, a shadow, or some kind of inclement weather.  The best part? We’re only a few months away from products which can boost the benefits of many of available security cameras.  The Caveat of Image Recognition Systems More and more people around the world own security cameras, not the least of which is security personnel. With this increased level of ownership, the market is expected to have over $20 billion in revenue in less than five years. With such high demand, it’s no wonder only about five percent of footage ever gets viewed. But what if security professionals could navigate up to 80 video sources on a single server? What if the algorithms, analytics, and video processing worked with cameras of all types including, but not limited to:  Car dashboardsBody vestsDrone mounts With all these cameras surrounding us, the one thing to watch is that it is still humans who input the information the cameras use to process. When it comes to security professionals such as law enforcement, investigation, and government authorities its import to make sure the camera doesn’t discriminate or operate on bias. This an important issue even that even our most advanced intelligence agencies admits the algorithms for their facial recognition software are wrong about 15 percent of the time. However, one organization is working to improve and make these algorithms more ethically developed when it comes to image recognition as well as respecting individual privacy concerns.  As we navigate the growing pains of new technologies, it's important to understand these solutions are meant to ensure customers, the public, and communities can trust the solutions being created. If you’re looking for a new role in Computer Vision, take a look at our latest opportunities or get in touch with one of our expert consultants: For our West Coast Team, call (415) 614 - 4999 or send an email to sanfraninfo@harnham.com.   For our Mid-West and East Coast Teams, call (212) 796 - 6070 or send an email to newyorkinfo@harnham.com. 

Recently Viewed jobs