Lead Data Engineer

London
£55000 - £75000 per annum + Additional Benefits

Senior Data Engineer

London

Up to £75,000

The Company

Join a small start up within one of the largest retailers in the UK, taking full ownership of your technology stack. Work with your own AWS account, each team having its own profit and loss and will work autonomously.

The Role

In this role you will build the next generation of data lakes across the whole of the company, allowing every team to go in and pull out data in its raw form. Working with either Scala, Java or Python you will build an infrastructure to revolutionise the company. You will:

  • Build and design large scale, real time data pipelines using Apache Spark and Kafka
  • Programme perfectly in Java, Scala or Python and go on to implement this in cloud technologies e.g. AWS
  • Work in a collaborative and agile environment within a team of cross functional engineers

Key Skills and Requirements

  • Experience working with large data at scale
  • Worked within an agile environment
  • Experience with either Scala, Python or Java

HOW TO APPLY

Interested? Please register your interest by submitting your CV directly by applying to this advert.

Send similar jobs by email
49349
London
£55000 - £75000 per annum + Additional Benefits
  1. Permanent
  2. Software Engineer

Similar Jobs

Salary

Up to £80000 per annum + Flexible Working + Numerous Benefits

Location

Hertfordshire

Description

Head of DevOps position at on of the most technically orientated businesses in the UK.

Salary

US$100000 - US$130000 per year

Location

New York

Description

Software Engineer II - Fintech

Salary

£60000 - £65000 per annum

Location

London

Description

Are you a Java enthusiast who is looking to grow their skill set while adding value and previous expertise to an Engineering team?

Salary

£70000 - £90000 per annum + 10% bonus

Location

London

Description

Your role will be largely management based, with plenty of opportunity to get involved in high-level architectural decisions and solution-based discussions.

Salary

£50000 - £60000 per annum + Wide Ranging Benefits Package

Location

Hertfordshire

Description

Senior DevOps Engineer position for a global icon, world renowned for their exceptional products and culture.

Harnham blog & news

With over 10 years experience working solely in the Data & Analytics sector our consultants are able to offer detailed insights into the industry.

Visit our Blogs & News portal or check out our recent posts below.

How Big Data Is Impacting Logistics

How Big Data is Impacting Logistics

As Big Data can reveal patterns, trends and associations relating to human behaviour and interactions, it’s no surprise that Data & Analytics are changing the way that the supply chain sector operates today.  From informing and predicting buying trends to streamlining order processing and logistics, technological innovations are impacting the industry, boosting efficiency and improving supply chain management.  Analysing behavioural patterns Using pattern recognition systems, Artificial Intelligence is able to analyse Big Data. During this process, Artificial Intelligence defines and identifies external influences which may affect the process of operations (such as customer purchasing choices) using Machine Learning algorithms. From the Data collected, Artificial Intelligence is able to determine information or characteristics which can inform us of repetitive behaviour or predict statistically probable actions.  Consequently, organisation and planning can be undertaken with ease to improve the efficiency of the supply chain. For example, ordering a calculated amount of stock in preparation for a busy season can be made using much more accurate predictions - contributing to less over-stocking and potentially more profit. As a result, analysing behavioural patterns facilitates better management and administration, with a knock-on effect for improving processes.  Streamlining operations  Using image recognition technology, Artificial Intelligence enables quicker processes that are ideally suited for warehouses and stock control applications. Additionally, transcribing voice to text applications mean stock can be identified and processed quickly to reach its destination, reducing the human resource time required and minimising human error.  Artificial intelligence has also changed the way we use our inventory systems. Using natural language interaction, enterprises have the capability to generate reports on sales, meaning businesses can quickly identify stock concerns and replenish accordingly. Intelligence can even communicate these reports, so Data reliably reaches the next person in the supply chain, expanding capabilities for efficient operations to a level that humans physically cannot attain. It’s no surprise that when it comes to warehousing and packaging operations Artificial Intelligence can revolutionise the efficiency of current systems. With image recognition now capable of detecting which brands and logos are visible on cardboard boxes of all sizes, monitoring shelf space is now possible on a real-time basis. In turn, Artificial Intelligence is able to offer short term insights that would have previously been restricted to broad annual time frames for consumers and management alike.  Forecasting  Many companies manually undertake forecasting predictions using excel spreadsheets that are then subject to communication and data from other departments. Using this method, there’s ample room for human error as forecasting cannot be uniform across all regions in national or global companies. This can create impactful mistakes which have the potential to make predictions increasingly inaccurate.  Using intelligent stock management systems, Machine Learning algorithms can predict when stock replenishment will be required in warehouse environments. When combined with trend prediction technology, warehouses will effectively be capable enough to almost run themselves  negating the risk of human error and wasted time. Automating the forecasting process decreases cycle time, while providing early warning signals for unexpected issues, leaving businesses better prepared for most eventualities that may not have been spotted by the human eye.  Big Data is continuing to transform the world of logistics, and utilising it in the best way possible is essential to meeting customer demands and exercising agile supply chain management.  If you’re interested in utilising Artificial Intelligence and Machine Learning to help improve processes, Harnham may be able to help. Take a look at our latest opportunities or get in touch with one of our expert consultants to find out more.  Author Bio: Alex Jones is a content creator for Kendon Packaging. Now one of Britain's leading packaging companies, Kendon Packaging has been supporting businesses nationwide since the 1930s.

Where Tech Meets Tradition

Where Tech Meets Tradition

If you’re lamenting the decline of handmade traditional products, cast your cares aside. There’s a new Sheriff in town and its name is, Tech. Just a generation ago, children would leave the farm or the family business, go to school, and then move on to make their place in the world doing their own thing. Away from family.  Today, the landscape has changed and those who have left are coming home. But this time, they’re bringing technology with them to help make things more efficient and more productive. Is Tech-Assisted Still Handmade? In a word, yes. Artists still make things “from scratch”, except now technologies allow them to not only see their vision in real-time, but their customers, too. Have you ever wondered what the image in your head might look like on paper or in metal? What about the design of prosthetic arms and healthcare devices by 3D printers? You’re still designing, creating.  But just like any new technology, there’s still a learning curve. Even for cutting-edge craftspeople who find that sometimes, the line between craftsmanship and high-tech creativity may be a bit of a blur. Not to mention the expense for either the equipment required or being able to offer art using traditional tools at technology-assisted prices. Somewhere between the two, there is a trade-off. It’s up to the individual to determine where and what that trade-off is. Life in the Creative Economy One of Banksy’s paintings shredded itself upon purchase at an auction recently. AI is making music and writing books. Augmented Reality, Virtual Reality, and Blockchain all have their place in the creative economy from immersive entertainment to efficient manufacturing processes. Each of these touches the way we live now. In a joint study between McKinsey and the World Economic Forum, 'Creative Disruption: The impact of emerging technologies on the creative economy', the organisations broke down the various technologies used in the creative economy and how they’re driving change. For example: AI is being used to distill user preferences when it comes to curating movies and music. The Associated Press has used AI to free up reporters’ time and the Washington Post has created a tool to help it generate up to 70 articles a month, many stories of which they wouldn’t have otherwise dedicated staff.Machine Learning has begun to create original content. Virtual Reality and Augmented Reality have come together as a new medium to help move people to get up, get active, and go play whether it’s a stroll through a virtual art gallery or watching your children play at the playground.  Where else might immersive media play out? Content today could help tell humanitarian stories or offer work-place diversity training. But back to the artisan handicrafts.  Artistry with technology Whilst publishing firms may be looking to use AI to redefine the creative economy, they are not alone. Other artists utilising these technologies include:  SculptorsDigital artistsPaintersJewellery makersBourbon distillers America’s oldest distiller has gotten on the technology bandwagon and while there is no rushing good Bourbon, but you can manage the process more efficiently. They’ve even taken things a step further and have created an app for aficionados to follow along in the process. Talk about crafted and curated for individual tastes and transparency. It may seem almost self-explanatory to note how other artisans are using technology. But what about distilleries? What are they doing? They’re creating efficiency by: Adding IoT sensors for Data Analytics collection Adding RFID tags to their barrels Creating experimental ageing warehouses (AR, anyone?) to refine their craft. Don’t worry, though. These changes won’t affect the spirit itself. After all, according to Mr. Wheatley, Master Distiller, “There’s no way to cheat mother nature or father time.” Ultimately, the idea is to not only understand the history behind the process, but to make it more efficient and repeatable. A way to preserve the processes of the past while using the advances of the present with an eye to the future. If you’re interested in using Data & Analytics to drive creativity, we may have a role for you. Take a look at our latest opportunities or get in touch with one of our expect consultants to find out more. 

Recently Viewed jobs