DATA ENGINEER

Amsterdam, North Holland
€60000 - €80000 per annum

DATA ENGINEER
UP TO €80000
AMSTERDAM
Are you a data engineer looking for a new challenge? If the answer is yes, a growing, international team based in the hub of Amsterdam is looking for a data engineer to join their team, with the challenging task of helping them to best use of the data they receive every day, by modifying and creating their data infrastructure. This is a fast-growing, team-focused business within the ever-changing financial industry.
THE COMPANY
You will be working for a leading, tech-focused, specialist trading company, which has major trading desks around the world and is looking to constantly improve and innovate how it uses data. The company has an inventive, entrepreneurial and team focused working culture, where you can really see the impact of your contributions and continuously improve your technical knowledge, alongside a likeminded team.
THE ROLE
As a data engineer you will be responsible for helping the company to develop and increase its use of data, by improving its data infrastructure in the cloud. You will do this by providing technical expertise, which will include:
* Building, testing and modifying its data applications and data infrastructure by coding in Python
* Working alongside and collaborating with a range of employees from different teams, which include analysts, quantitative traders, and people from the business side of the company
* Assisting the support team to roll out and configure its data infrastructure
* Ensuring that all data collected is efficiently stored and available for use, to maximise its potential
YOUR SKILLS AND EXPERIENCE
To be successful in this role, you will need to
* Have graduated (BSc/MSc) preferably in Computer Science or another degree where programming is taught and used
* Have experience in developing and maintaining data architecture, using Spark, Kafka, Hadoop and Spark and come from a strong engineering background
* Be proficient in advanced programming skills, such as in Python, Java, SQL and Scala to be able to code from scratch
* Know how to use cloud technologies such as AWS, GCP and Azure, using tools including S3 and Redshift and work independently
* Be fluent in English and be a skilled communicator, as you will be working with a range of people in different teams and roles
THE BENEFITS
* You will receive a highly competitive salary of up to €80000, depending on your experience and skills alongside a arrange of great benefits, such as team events
* Working within a diverse and fun team, which has a collaborative focus and a unique culture, where you can really get to know your team
* Getting to use a range of the latest, cutting edge technology and thanks to a great onboarding scheme, being able to be continually develop your technical knowledge
* Working for a company that is looking to scale up and expand its team, allowing you tot grow with the data and analytics team

HOW TO APPLY
Please register your interest by sending you CV to Mathieu Wilken via the Apply link on this page

Send similar jobs by email
0910/MW
Amsterdam, North Holland
€60000 - €80000 per annum
  1. Permanent
  2. Big Data

Similar Jobs

Salary

US$180000 - US$200000 per annum + Additional Benefits

Location

New York

Description

A NYC healthcare leader is looking to bring in a Director of Data Engineering to join their team!

Salary

£500 - £600 per day

Location

London

Description

Data Engineer 6 month contract (Outside IR35) Azure, Python, Spark Remote/London £600 - £700.day

Salary

£500 - £600 per day

Location

London

Description

Data Engineer 6 month contract (Outside IR35) Azure, Python, Spark Remote/London

Salary

€40000 - €50000 per annum + Additional benefits

Location

Amsterdam, North Holland

Description

A start-up is looking data engineer to join their team to transform the experience of online supermarket shopping, who will be building ETL pipelines.

Harnham blog & news

With over 10 years experience working solely in the Data & Analytics sector our consultants are able to offer detailed insights into the industry.

Visit our Blogs & News portal or check out our recent posts below.

The Search For Toilet Paper: A Q&A With The Data Society

We recently spoke Nisha Iyer, Head of Data Science, and Nupur Neti, a Data Scientist from Data Society.  Founded in 2014, Data Society consult and offer tailored Data Science training for businesses and organisations across the US. With an adaptable back-end model, they create training programs that are not only tailored when it comes to content, but also incorporate a company’s own Data to create real-life situations to work with.  However, recently they’ve been looking into another area: toilet paper.  Following mass, ill-informed, stock-piling as countries began to go into lockdown, toilet paper became one of a number of items that were suddenly unavailable. And, with a global pandemic declared, Data Society were one of a number of Data Science organisations who were looking to help anyway they could.  “When this Pandemic hit, we began thinking how could we help?” says Iyer. “There’s a lot of ways Data Scientists could get involved with this but our first thought was about how people were freaking out about toilet paper. That was the base of how we started, as kind of a joke. But then we realised we already had an app in place that could help.” The app in question began life as a project for the World Central Kitchen (WCK), a non-profit who help support communities after natural disasters occur.  With the need to go out and get nutritionally viable supplies upon arriving at a new location, WCK teams needed to know which local grocery stores had the most stock available.  “We were working with World Central Kitchen as a side project. What we built was an app that supposed to help locate resources during disasters. So we already had the base done.” The app in question allows the user to select their location and the products they are after. It then provides information on where you can get each item, and what their nutritional values are, with the aim of improving turnaround time for volunteers.  One of the original Data Scientists, Nupur Neti, explained how they built the platform: “We used a combination of R and Python to build the back-end processing and R Shiny to build the web application. We also included Google APIs that took your location and could find the closest store to you. Then, once you have the product and the sizes, we had an internal ranking algorithm which could rank the products selected based on optimisation, originally were based on nutritional value.”  The team figured that the same technology could help in the current situation, ranking based on stock levels rather than nutritional value. With an updated app, Iyer notes “People won’t have to go miles and stand in lines where they are not socially distancing. They’ll know to visit a local grocery store that does have what they need in stock, that they’ve probably not even thought of before.” However, creating an updated version presented its own challenges. Whereas the WCK app utilised static Data, this version has to rely on real-time Data. Unfortunately this isn’t as easy to come by, as Iyer knows too well:  “When we were building this for the nutrition app we reached out to groceries stores and got some responses for static Data. Now, we know there is real-time Data on stock levels because they’re scanning products in and out. Where is that inventory though? We don’t know.” After putting an article out asking for help finding live Data, crowdsourcing app OurStreets got in touch. They, like Data Society, were looking to help people find groceries in short supply. But, with a robust front and back-end in place, the app already live, and submissions flying in across the States, they were looking for a Data Science team who could make something of their findings.  “We have the opportunity,” says Iyer “to take the conceptual ideas behind our app and work with OurStreets robust framework to create a tool that could be used nationwide.” Before visiting a store, app users select what they are looking for. This allows them to check off what the store has against their expectations, as well as uploading a picture of what is available. They can also report on whether the store is effectively practising social distancing. Neti explains, that this Data holds lots of possibilities for their Data Science team: “Once we take their Data, our system will clean any submitted text using NLP and utilise image recognition on submitted pictures using Deep Learning. This quality Data, paired with the Social Distancing information, will allow us to gain better insights into how and what people are shopping for. We’ll then be able to look at trends, see what people are shopping for and where. Ultimately, it will also allow us to make recommendations as to where people should then go if they are looking for a product.”  In addition to crowdsourced information, Data Society are still keen to get their hands on any real-time Data that supermarkets have to offer. If you know where they could get their hands on it, you can get in touch with their team.  Outside of their current projects, Iyer remains optimistic for the world when it emerges from the current situation: “Things will return to normal. As dark a time as this is, I think it’s going to exemplify why people need to use Artificial Intelligence and Data Science more. If this type of app is publicised during the Coronavirus, maybe more people will understand the power of what Data and Data Science can do and more companies that are slow adaptors will see this and see how it could be helpful to their industry.”   If you want to make the world a better place using Data, we may have a role for you, including a number of remote opportunities. Or, if you’re looking to expand and build out your team with the best minds in Data, get in touch with one of expert consultants who will be able to advise on the best remote and long-term processes. 

What defines a Data Architect?

Data Analyst. Data Wrangler. Data Architect? If you like pulling together threads of a company’s Data into one cohesive point, you may want to consider a Data Architect role. But what exactly is a Data Architect and how does it differ from a Data Engineer? Data Architect vs. Data Engineer As businesses continue to combine their Data and business strategies into one, they are beginning to understand to the need for a variety of Data Analysts. But as important as it is to have someone build your platform and begin pipeline processes, there is also need for someone with vision. Someone who can see patterns and designs. Someone who has end-to-end vision and can see how the patterns flow through your processes. This is your Data Architect. Data Engineers, on the other hand, lay the foundation for your Data platform. They draft the blueprint. After all, you can’t build a house without a blueprint first, right? The Data Engineer is at the beginning of the process, so the rest of the team can do their parts. But it’s the Data Architect who pulls it all together. THE ROLE OF THE DATA ARCHITECT  If you’re considering your next career move and wondering if Data Architecture is for you, here are some typical requirements. A typical Data Architect will: Meet with stakeholders to understand business needs and translate them into technical requirements using ETL techniques to develop Data ArchitectureUnderstand their full Data lifecycle to provide technical architecture leadershipDesign a real-time data pipeline ecosystem and how to make it scalable usingDevelop Big Data Architecture in an AWS environmentBe educated to a degree level in a numerate discipline (Mathematics, Statistics, Computer Science, Computer Engineering)• Have proven experience in a commercial environmentHave advanced Cloud Computing Ecosystem experience with AWS (GCP or Azure also considered)Have proven Big Data Ecosystem experienceHave proven Big Data Architecture experience in a commercial environment Have proven Data Engineering experience in a commercial environment Though the likes of Google, IBM, and others have ramped up their education efforts, and online courses traditional universities offer a variety of Data Science degrees, there is still a shortage of professionals in the industry. So can businesses simplify and automate processes without the right people in place? Businesses Step Up Their Data Strategies Though there are easier ways to get the information a business needs through rented predictive modelling or an already drafted Data Science model, it doesn’t give the true value of Data. Add in new regulations, requirements, and new Data which offer new insights, and the impact on business is profound.   It’s time for business to start ensuring that their Data teams are treated as critically as possible. Time to lay a path of progression, a pipeline, of systems and processes for the creation and production of Data. After all, simply optimising your Data will only get you so far. Enterprise-wide Data systems are more than wrangling and analysing Data.  Most importantly, businesses need to ensure they have the right people in place. They also need to understand what they need and why they need it. This is a key part of Data Strategy and with the right people in place, can put your business ahead of the competition.  Digging Deeper into Requirements for Top Talent While the standard requirements for a Data professional are to be educated to a degree level in things like Computer Science and Mathematics, technical skills, and experience within certain industries, for the natural progression from Data Analyst to Data Architect, there’s a bit more nuance to consider. Whether your business is just getting started in Data Science or you’re ready to start growing an existing team, there are some things you may want to focus on when looking for your Data Architect role. Define and determine how to keep projects streamlined with repeatable processes. Pivot between guiding team members through the pipeline and explaining insights to executives and stakeholders. Determine the right format for the right project. Determine when and when not to use automation to integrate Data. Visualise and extract models to predict future events and describe the process. In other words, be able to interpret Data to ensure reliability of the best approach. With the right talent in place, your teams can collaborate and build on their shared expertise to ensure Data is analysed and understood to the best benefit of your business. If you like solving puzzles, pulling disparate threads together into organised systems, and have experience as analysing and collecting Data, we may have a role for you.  Take a look at our latest opportunities or get in touch with one of our expert consultants to find out more.  

Recently Viewed jobs