Harnham
SEARCH
Shortlist

Data Engineer

Manchester, Greater Manchester
£40000 - £50000 per annum
Apply Save job

DATA ENGINEER

£40,000 - £50,000

MANCHESTER

Are you a data engineer who wants to make an impact in the world of sports? This exciting opportunity allows you to do just that whilst being hands on in building out a new data platform in Azure and Spark.

THE COMPANY:

This role is within a Sports devoted company based in Manchester. They have a strong analytics function and a large data platform used by multiple teams. The data platform has previously been a BI infrastructure but is now based around Spark, Data Bricks.

THE ROLE:

The successful candidate will sit in the Data & Analytics department and deal with large scale data in an advanced sports offering. You will be working on the development of the new Data Bricks platform.

In particular, this role will involve:

  • The maintenance and development of the Data Platform using Data Bricks and Azure.
  • Gathering large amounts of data for analytics.
  • Programming in Python.
  • Using a software engineering mindset for writing pipelines rather than scripting.

YOUR SKILLS & EXPERIENCE:

  • Relevant tech stack including Python for programming, Apache Spark or Data bricks.
  • Commercial experience with cloud technologies, ideally Azure - data factory.
  • An interest in sports.
  • Software engineering skill set.
  • Knowledge of BI tools (ideally Tableau and Power BI).

THE BENEFITS:

The selected candidate will receive a salary between £40,000 - £50,000 depending on their experience and requirements.

HOW TO APPLY:

Please register your interest by sending your CV to Holly Neeves via the Apply link on this page.

Apply Save job
Send similar jobs by email
96212/HN
Manchester, Greater Manchester
£40000 - £50000 per annum
  1. Permanent
  2. Big Data

Similar Jobs

Data Engineer
Salary

£60000 - £70000 per annum

Location

London

Description

An opportunity for a mid-senior level Data Engineer to join a large corporation.

ADD TO SHORTLIST
Senior Big Data Engineer - CPG eCommerce
Salary

US$120000 - US$130000 per annum + Additional Benefits

Location

Cincinnati, Ohio

Description

My client in Ohio are looking for big data engineering experts looking to join a learning-based cutting edge environment to grow technically!

ADD TO SHORTLIST
Salesforce Solutions Engineer
Salary

US$150000 - US$160000 per annum

Location

New York

Description

Looking for a Salesforce Developer to join a Data Intelligence consulting firm that provides innovative solutions to meet business goals.

ADD TO SHORTLIST
Senior Data Engineer
Salary

£75000 - £85000 per annum

Location

London

Description

I am now recruiting for a Senior Data Engineer with experience in client facing positions, mentoring and leadership experience and with a broad tech stack.

ADD TO SHORTLIST

Harnham blog & news

With over 10 years experience working solely in the Data & Analytics sector our consultants are able to offer detailed insights into the industry.

Visit our Blogs & News portal or check out our recent posts below.

Using Data Ethically To Guide Digital Transformation

Over the past few years, the uptick in the number of companies putting more budget behind digital transformation has been significant. However, since the start of 2020 and the outbreak of the coronavirus pandemic, this number has accelerated on an unprecedented scale. Companies have been forced to re-evaluate  their systems and services to make them more efficient, effective and financially viable in order to stay competitive in this time of crisis. These changes help to support internal operational agility and learn about customers' needs and wants to create a much more personalised customer experience.  However, despite the vast amount of good these systems can do for companies' offerings, a lot of them, such as AI and machine learning, are inherently data driven. Therefore, these systems run a high risk of breaching ethical conducts, such as privacy and security leaks or serious issues with bias, if not created, developed and managed properly.  So, what can businesses do to ensure their digital transformation efforts are implemented in the most ethical way possible? Implement ways to reduce bias From Twitter opting to show a white person in a photo instead of a black person, soap dispensers not recognising black hands and women being perpetually rejected for financial loans; digital transformation tools, such as AI, have proven over the years to be inherently biased.  Of course, a computer cannot be decisive about gender or race, this problem of inequality from computer algorithms stems from the humans behind the screen. Despite the advancements made with Diversity and Inclusion efforts across all industries, Data & Analytics is still a predominantly white and male industry. Only 22 per cent of AI specialists are women, and an even lower number represent the BAME communities. Within Google, the world’s largest technology organisation, only 2.5 per cent of its employees are black, and a similar story can be seen at Facebook and Microsoft, where only 4 per cent of employees are black.  So, where our systems are being run by a group of people who are not representative of our diverse society, it should come as no surprise that our machines and algorithms are not representative either.  For businesses looking to implement AI and machine learning into their digital transformation moving forward, it is important you do so in a way that is truly reflective of a fair society. This can be achieved by encouraging a more diverse hiring process when looking for developers of AI systems, implementing fairness tests and always keeping your end user in mind, considering how the workings of your system may affect them.  Transparency Capturing Data is crucial for businesses when they are looking to implement or update digital transformation tools. Not only can this data show them the best ways to service customers’ needs and wants, but it can also show them where there are potential holes and issues in their current business models.  However, due to many mismanagements in past cases, such as Cambridge Analytica, customers have become increasingly worried about sharing their data with businesses in fear of personal data, such as credit card details or home addresses, being leaked. In 2018, Europe devised a new law known as the General Data Protection Regulation, or GDPR, to help minimise the risk of data breaches. Nevertheless, this still hasn’t stopped all businesses from collecting or sharing data illegally, which in turn, has damaged the trustworthiness of even the most law-abiding businesses who need to collect relevant consumer data.  Transparency is key to successful data collection for digital transformation. Your priority should be to always think about the end user and the impact poorly managed data may have on them. Explain methods for data collection clearly, ensure you can provide a clear end-to-end map of how their data is being used and always follow the law in order to keep your consumers, current and potential, safe from harm.  Make sure there is a process for accountability  Digital tools are usually brought in to replace a human being with qualifications and a wealth of experience. If this human being were to make a mistake in their line of work, then they would be held accountable and appropriate action would be taken. This process would then restore trust between business and consumer and things would carry on as usual.  But what happens if a machine makes an error, who is accountable?  Unfortunately, it has been the case that businesses choose to implement digital transformation tools in order to avoid corporate responsibility. This attitude will only cause, potentially lethal, harm to a business's reputation.  If you choose to implement digital tools, ensure you have a valid process for accountability which creates trust between yourself and your consumers and is representative of and fair to every group in society you’re potentially addressing.  Businesses must be aware of the potential ethical risks that come with badly managed digital transformation and the effects this may have on their brands reputation. Before implementing any technology, ensure you can, and will, do so in a transparent, trustworthy, fair, representative and law-abiding way.  If you’re in the world of Data & Analytics and looking to take a step up or find the next member of your team, we can help. Take a look at our latest opportunities or get in touch with one of our expert consultants to find out more.

Henry Rodrigues

Henry Rodrigues

21. January 2021

News

The Dialogue: Keeping Data Secure While Working Flexibly

Last week, Peter Schroeter and Ryan Collins, head of DevSecOps at Upvest, and co-founder of RapidGroup, discussed how to keep Data secure while working flexibly.  Ryan brings to the table 12 years of experience from Server Administrator to CTO roles as well as experience as a Contractor. And as a business owner himself, he sees where the shortage falls and perhaps a way to fill the gap. Security is in the Spotlight Security is a priority for many businesses today. Avoiding negative PR has caused an internal shift in which companies take more care with their Data. There is also a Catch-22. In order to provide higher levels of security, businesses are slowing down their developers and software engineers, and taking more engineering time which in turn costs more money. Security before cost is becoming the new reality. The contradiction of deployment, project run time, and budgets are only part of the bigger picture. Compromise is key and follow these three tenets. Don’t leave anything open to vulnerability.Focus on auditability.Offer more training for Software Engineers. The Security-Focused Skills Gap How can you push security forward in a meaningful way when you can’t find the people you need with relevant experience? There’s a big gap in the market right now for security-oriented DevOps engineers. The skillsets many businesses are looking for include: Google Platform AWS and possibly Azure with a modern suite of tools with Teraform.Ability to float to the development side and work in SRE to ensure things are stable and scalable.DevOps Engineers with experience in the GO stack are especially hard to find. Companies want to go with what they know. However, there is a shift toward a more remote-friendly and contractor culture. When you can’t find the talent you need, sometimes it’s best to bite the bullet, and consider a Contractor. The Case to Hire a Contractor If this year has taught us anything, it’s that we don’t have to be confined to an office or to even one location. And yes, while there is an element of risk to being a Contractor, there are benefits to both sides. Contractors are compensated higher because businesses have lower HR costs, less tax regulations, payroll, and reporting to do.  Though there is some risk, there is always work for because so many companies want to secure their data. There’s always something to be optimised, always something which needs attention. You won’t be without work for long, if you have the skills. Disconnect Between Capability and Desire in the Market Some candidates have mentioned they have 60% of the skills required, but not enough project-based experience. How do you reconcile the disconnect?  It’s hard to specialise in DevOps, DevSecOps, and similar roles because it’s about automation, you have to be that "Swiss Army Knife", you have to live in the middle. You have to know how to get into the code, you have to know how to get in and do the CI workflows, etc. It’s almost Developer plus value-added skills or security engineer plus value-added skills. Remote Working Habits You Need Now Have a separated space set up for work.If you have a mix of people in office and those working remote, you have to make sure they’re communicating with each other.Rolling coffee breaks within Google Meet or something similar.Have a task management and time tracking system used by employees both on and offline.Build culture by picking at random two people and putting them together for a half hour to have a conversation that isn’t about work.  Startup vs Legacy Hiring Don’t box yourself in, but understand startups have a unique set of skills they need and most often without the budget to train someone. Whereas legacy businesses more often than not have the budget to train someone in the skillset they need. However, it’s important to note, the tech stack itself doesn’t really change. Best Practices for Async-Comm Teams Remember not everyone works in the same time zone. Don’t expect and immediate answer, and if you need an immediate answer, pick up the phone. Make sure your team has remote tools such as Slack or Google Meet and is doing most of their communication this way, even if they are in an office. If you don’t, your remote workers could be missing key information. The Future of DevOps and Data Security Many businesses may see a shift toward a more open working environment which is a good balance for what works best for talent and is good for productivity as well. Ultimately, we’re all solving the same challenges What is most important when it comes to keeping Data secure? Most important is getting people, systems, and education in place to do something in the first place. In other words, build from concept rather than moving too fast and breaking too many things. How Can Prospective Candidates Prepare? Focus on continuous learning and getting your skillsets like automation tools. If you really want to get involved with security side and SRE, you really have to get involved with the development side, too. Using the modern tech stacks like the GOLANG, the RUST, the SWIFT on the mobile side, and there’s always new pieces to the puzzle.There’s room for all types of relationships. Whether you’re looking for a long-term role, a short-term role, or something in between, DevOps is a never-ending project, so continuous learning is key for both candidate and company. You can watch the full conversation below:

Ali Mirza

03. December 2020

News

Recently Viewed jobs