Harnham
SEARCH
Shortlist

DATA ENGINEER - ONLINE

Oslo
750000kr - 850000kr per annum
Apply Save job

DATA ENGINEER - MARKETPLACE

ONLINE BUSINESS

OSLO

Are you a data engineer looking to work with the big data sets? Want to work with the latest technologies? If cloud-based tech, big data, and real-time processing excite you, then look no further. Great opportunity to work with Scala, Java, Python, AWS, and Azure.

THE COMPANY:

As a Data Engineer, you will be joining one of the top online businesses in Oslo who serve millions of customers every day. This company is one of the most data-driven companies in the Nordics - and has an incredible data and analytics environment with employees from all over the world.

THE ROLE:

As a Data Engineer, you can expect to sit within a team of senior data engineers and technical leads, offering next-level mentorship. You will work hands-on within this team to build large-scale batch and real-time data pipelines with data processing frameworks.

Specifically, you can expect to be involved in the following:

  • Designing, building, and implementing both real-time and batch pipelines
  • Support rapid discovery and prototyping in Scala, Python, and Java
  • Implement cutting-edge cloud technologies
  • Help define the development environment (code reviews, testing, etc)

YOUR SKILLS AND EXPERIENCE:

A successful Data Engineer will have the following skills, attributes, and experience.

  • Educated to a degree level in a relevant subject area
  • Hands-on commercial experience with big data technologies (etc Kafka, Spark, Hadoop, Storm, Cassandra)
  • Detailed knowledge of SQL based ETL pipelines and proven hands-on commercial experience
  • Strong Python, Scala, Java or JS programming skills, having utilized these in a commercial environment
  • Expert in data modeling, data access, and data storage

HOW TO APPLY:

Please register your interest by sending your CV to Elise Myhren via the Apply link on this page.

KEYWORDS:

Data Engineer, Oslo, Norway, Online, Data Pipelines, ETL, SQL, Scala, Python, Java, GCP, Spark, Scalding, Scio, Storm

Apply Save job
Send similar jobs by email
VAC-18631/EM
Oslo
750000kr - 850000kr per annum
  1. Permanent
  2. Big Data

Similar Jobs

Senior Big Data Engineer - CPG eCommerce
Salary

US$120000 - US$130000 per annum + Additional Benefits

Location

Cincinnati, Ohio

Description

My client in Ohio are looking for big data engineering experts looking to join a learning-based cutting edge environment to grow technically!

ADD TO SHORTLIST
Director of Data Engineering
Salary

US$180000 - US$200000 per annum + Additional Benefits

Location

New York

Description

A NYC healthcare leader is looking to bring in a Director of Data Engineering to join their team!

ADD TO SHORTLIST
Data Engineer - Azure (Contract)
Salary

£500 - £600 per day

Location

London

Description

Data Engineer 6 month contract (Outside IR35) Azure, Python, Spark Remote/London

ADD TO SHORTLIST
DATA ENGINEER
Salary

€40000 - €50000 per annum + Additional benefits

Location

Amsterdam, North Holland

Description

A start-up is looking data engineer to join their team to transform the experience of online supermarket shopping, who will be building ETL pipelines.

ADD TO SHORTLIST

Harnham blog & news

With over 10 years experience working solely in the Data & Analytics sector our consultants are able to offer detailed insights into the industry.

Visit our Blogs & News portal or check out our recent posts below.

The Search For Toilet Paper: A Q&A With The Data Society

We recently spoke Nisha Iyer, Head of Data Science, and Nupur Neti, a Data Scientist from Data Society.  Founded in 2014, Data Society consult and offer tailored Data Science training for businesses and organisations across the US. With an adaptable back-end model, they create training programs that are not only tailored when it comes to content, but also incorporate a company’s own Data to create real-life situations to work with.  However, recently they’ve been looking into another area: toilet paper.  Following mass, ill-informed, stock-piling as countries began to go into lockdown, toilet paper became one of a number of items that were suddenly unavailable. And, with a global pandemic declared, Data Society were one of a number of Data Science organisations who were looking to help anyway they could.  “When this Pandemic hit, we began thinking how could we help?” says Iyer. “There’s a lot of ways Data Scientists could get involved with this but our first thought was about how people were freaking out about toilet paper. That was the base of how we started, as kind of a joke. But then we realised we already had an app in place that could help.” The app in question began life as a project for the World Central Kitchen (WCK), a non-profit who help support communities after natural disasters occur.  With the need to go out and get nutritionally viable supplies upon arriving at a new location, WCK teams needed to know which local grocery stores had the most stock available.  “We were working with World Central Kitchen as a side project. What we built was an app that supposed to help locate resources during disasters. So we already had the base done.” The app in question allows the user to select their location and the products they are after. It then provides information on where you can get each item, and what their nutritional values are, with the aim of improving turnaround time for volunteers.  One of the original Data Scientists, Nupur Neti, explained how they built the platform: “We used a combination of R and Python to build the back-end processing and R Shiny to build the web application. We also included Google APIs that took your location and could find the closest store to you. Then, once you have the product and the sizes, we had an internal ranking algorithm which could rank the products selected based on optimisation, originally were based on nutritional value.”  The team figured that the same technology could help in the current situation, ranking based on stock levels rather than nutritional value. With an updated app, Iyer notes “People won’t have to go miles and stand in lines where they are not socially distancing. They’ll know to visit a local grocery store that does have what they need in stock, that they’ve probably not even thought of before.” However, creating an updated version presented its own challenges. Whereas the WCK app utilised static Data, this version has to rely on real-time Data. Unfortunately this isn’t as easy to come by, as Iyer knows too well:  “When we were building this for the nutrition app we reached out to groceries stores and got some responses for static Data. Now, we know there is real-time Data on stock levels because they’re scanning products in and out. Where is that inventory though? We don’t know.” After putting an article out asking for help finding live Data, crowdsourcing app OurStreets got in touch. They, like Data Society, were looking to help people find groceries in short supply. But, with a robust front and back-end in place, the app already live, and submissions flying in across the States, they were looking for a Data Science team who could make something of their findings.  “We have the opportunity,” says Iyer “to take the conceptual ideas behind our app and work with OurStreets robust framework to create a tool that could be used nationwide.” Before visiting a store, app users select what they are looking for. This allows them to check off what the store has against their expectations, as well as uploading a picture of what is available. They can also report on whether the store is effectively practising social distancing. Neti explains, that this Data holds lots of possibilities for their Data Science team: “Once we take their Data, our system will clean any submitted text using NLP and utilise image recognition on submitted pictures using Deep Learning. This quality Data, paired with the Social Distancing information, will allow us to gain better insights into how and what people are shopping for. We’ll then be able to look at trends, see what people are shopping for and where. Ultimately, it will also allow us to make recommendations as to where people should then go if they are looking for a product.”  In addition to crowdsourced information, Data Society are still keen to get their hands on any real-time Data that supermarkets have to offer. If you know where they could get their hands on it, you can get in touch with their team.  Outside of their current projects, Iyer remains optimistic for the world when it emerges from the current situation: “Things will return to normal. As dark a time as this is, I think it’s going to exemplify why people need to use Artificial Intelligence and Data Science more. If this type of app is publicised during the Coronavirus, maybe more people will understand the power of what Data and Data Science can do and more companies that are slow adaptors will see this and see how it could be helpful to their industry.”   If you want to make the world a better place using Data, we may have a role for you, including a number of remote opportunities. Or, if you’re looking to expand and build out your team with the best minds in Data, get in touch with one of expert consultants who will be able to advise on the best remote and long-term processes. 

Nick Mandella

Nick Mandella

14. April 2020

News

What Does The Fourth Industrial Revolution Look Like?

We’re in the next stage of the fourth Industrial Revolution and technologies continue to merge. No longer is advancement as simple as adding “tech” to the end of a word - sorry Fintech, InsurTech, HRTech, and the rest. Now technologies stand together as each becomes a separate piece of how tech operates in the business world. AI and IoT have merged to become AIoT. Data is as much commodity as it is information to fuel business growth. Computer Vision partnered with AI is teaching computers to convert their ones and zeros to images humans take for granted.   In a word, it’s a transformative time for every industry and every industry is taking advantage of the benefits in one way or another. Smart manufacturing. Human Resources. Marketing. Even insurance has joined the party. But, with so many advancements, we thought we’d take a look at just the tip of the iceberg, starting with A, B, and C.  AI Meets IoT  We’ve all heard how AI and the Internet of Things (wearable and smart devices etc.) are being used in the Health sector. With the kind of real-time Data available, patients, insurers, and medical professionals can map out health plans based on wearable devices to track patient health and encourage preventative care.  Indeed, one insurance company is embracing these Data trends to ramp up the speed and efficiency of their data. Using Machine Learning and IoT sensors to develop an AI-based solution, customer information is used to match clients with the right policies tailored to their needs.  Car insurance is another industry to benefit. Insurers are able to collect real-time driving data which they can analyse to determine risk or offer discounted policies for good driving. This kind of information can also be used to revisit and reconstruct accident scenes to figure out what happened and who’s at fault.  Big Data, Big Money We’ve all heard the phrase ‘Data Is The New Oil’ by now, which I’m sure we can all agree, just means Data is a resource everybody wants and is willing to pay a lot for. But the differences between Data and Oil are two-fold; Data has the potential to be infinite, and it tells us about what oil cannot; the human experience.  Cloud technologies, edge hardware, and the IoT have helped shape the digitisation of objects, people, and organisations. From sensors to wearable devices, more and more data is being collected, allowing us to be more connected than ever before. It’s also providing more information to the tech giants than ever before. For example, Amazon’s Ring doorbell is logging every motion around it and can pinpoint the time to millisecond.   Add these technologies to Natural Language Processing (NLP) and watch the world around us draw value from and understand our Data like never before. The wave of Big Data value shows no signs of slowing down. Computer Vision in Business In the last few years, Computer Vision has been making great strides in the business world. Yet the Data required for processing power and memory can still be impacted by image quality. The opportunities  are alive with possibility and, from small businesses to enterprise solutions, Computer Vision has seen a variety of industries finding practical business uses.  Below are just a few additional areas Computer Vision is making its mark. Facial Recognition – providing surveillance and security systems in such areas as police work, payment portals, and retail stores.Digital Marketing – sorting and analysing online images to target ad campaigns to the right audiences.Financial Institutions –preventing fraud, allowing mobile deposits, analysing handwriting, and beyond. With the global market for fourth industry technologies predicted to be between $17.4 billion and $48.32 billion by 2023, now is the time to find your focus within the industry.  Ready to take the next step in your career? Whether you’re interested in AI, Big Data and Analytics, Computer Vision or more, we may have a role for you. Take a look at our current opportunities or get in touch with one of our expert consultants to find out more.  

Krishen Patel

Krishen Patel

05. March 2020

News

Recently Viewed jobs