Senior Data Engineer – Analytics Consultancy
Edinburgh / £55000 - £60000
£55000 - £60000
Senior Data Engineer
Hybrid - 1x per month in office, Edinburgh
A great opportunity for a Data Engineer looking to take the next step up in their career in an independent analytics consultancy.
The company is an independent analytics consultancy and works with a range of high-profile clients, including major banks, airlines, and retailers. They help their client utilise data to accelerate growth, increase the efficiency of their sales and marketing, and improve their customer experience.
They have a small workforce overall. This role will join a team of 10 Data Engineers and will see you developing Python-based pipelines for the Data Science team to create their marketing attribution and customer lifetime value models.
The role and responsibilities
- Manage projects and stakeholders
- Creating data pipelines for Data Science team
- Configuring and deploying marketing tools
Your skills and experience
- Experience with Python, SQL, and any Cloud platform
- Good communication skills
- Experience managing stakeholders
- £60, 000 salary
- Near-totally remote role, 1x per month in Edinburgh office
- Varied workload and opportunity to learn new data and tools
Data Engineer Or Software Engineer: What Does Your Business Need? | Harnham US Recruitment post
We are in a time in which what we do with Data matters. Over the last few years, we have seen a rapid rise in the number of Data Scientists and Machine Learning Engineers as businesses look to find deeper insights and improve their strategies. But, without proper access to the right Data that has been processed and massaged, Data Scientists and Machine Learning Engineers would be unable to do their job properly. So who are the people who work in the background and are responsible to make sure all of this works? The quick answer is Data Engineers!… or is it? In reality, there are two similar, yet different profiles who can help help a company achieve their Data-driven goals. Data Engineers When people think of Data Engineers, they think of people who make Data more accessible to others within an organization. Their responsibility is to make sure the end user of the Data, whether it be an Analyst, Data Scientist, or an executive, can get accurate Data from which the business can make insightful decisions. They are experts when it comes to data modeling, often working with SQL. Frequently, “modern” Data Engineers work with a number of tools including Spark, Kafka, and AWS (or any cloud provider), whilst some newer Databases/Data Warehouses include Mongo DB and Snowflake. Companies are choosing to leverage these technologies and update their stack because it allows Data teams to move at a much faster pace and be able to deliver results to their stakeholders. An enterprise looking for a Data Engineer will need someone to focus more on their Data Warehouse and utilize their strong knowledge of querying information, whilst constantly working to ingest/process Data. Data Engineers also focus more on Data Flow and knowing how each Data sets works in collaboration with one another. Software Engineers – DataSimilar to a Data Engineers, Software Engineers – Data ( who I will refer to as Software Data Engineers in this article) also build out Data Pipelines. These individuals might go by different names like Platform or Infrastructure Engineer. They have to be good with SQL and Data Modeling, working with similar technologies such as Spark, AWS, and Hadoop. What separates Software Data Engineers from Data Engineers is the necessity to look at things from a macro-level. They are responsible for building out the cluster manager and scheduler, the distributed cluster system, and implementing code to make things function faster and more efficiently. Software Data Engineers are also better programers. Frequently, they will work in Python, Java, Scala, and more recently, Golang. They also work with DevOps tools such as Docker, Kubernetes, or some sort of CI/CD tool like Jenkins. These skills are critical as Software Data Engineers are constantly testing and deploying new services to make systems more efficient. This is important to understand, especially when incorporating Data Science and Machine Learning teams. If Data Scientists or Machine Learning Engineers do not have a strong Software Engineers in place to build their platforms, the models they build won’t be fully maximized. They also have to be able to scale out systems as their platform grows in order to handle more Data, while finding ways to make improvements. Software Data Engineers will also be looking to work with Data Scientists and Machine Learning Engineers in order to understand the prerequisites of what is needed to support a Machine Learning model. Which is right for your business? If you are looking for someone who can focus extensively on pulling Data from a Data source or API, before transforming or “massaging” the Data, and then moving it elsewhere, then you are looking for a Data Engineer. Quality Data Engineers will be really good at querying Data and Data Modeling and will also be good at working with Data Warehouses and using visualization tools like Tableau or Looker. If you need someone who can wear multiple hats and build highly scalable and distributed systems, you are looking for a Software Data Engineer. It’s more common to see this role in smaller companies and teams, since Hiring Managers often need someone who can do multiple tasks due to budget constraints and the need for a leaner team. They will also be better coders and have some experience working with DevOps tools. Although they might be able to do more than a Data Engineer, Software Data Engineers may not be as strong when it comes to the nitty gritty parts of Data Engineering, in particular querying Data and working within a Data Warehouse. It is always a challenge knowing which type of job to recruit for. It is not uncommon to see job posts where companies advertise that they are looking for a Data Engineer, but in reality are looking for a Software Data Engineer or Machine Learning Platform Engineer. In order to bring the right candidates to your door, it is crucial to have an understanding of what responsibilities you are looking to be fulfilled.That’s not to say a Data Engineer can’t work with Docker or Kubernetes. Engineers are working in a time where they need to become proficient with multiple tools and be constantly honing their skills to keep up with the competition. However, it is this demand to keep up with the latest tech trends and choices that makes finding the right candidate difficult. Hiring Managers need to identify which skills are essential for the role from the start, and which can be easily picked up on the job. Hiring teams should focus on an individual’s past experience and the projects they have worked on, rather than looking at their previous job titles. If you’re looking to hire a Data Engineer or a Software Data Engineer, or to find a new role in this area, we may be able to help. Take a look at our latest opportunities or get in touch if you have any questions.
Keepers of the Data Kingdom: the Analytics Engineer | Harnham US Recruitment post
If it seems the Data world is drilling down further into niche specialities, you’re right. Considering the swathes of information sent and received on a day-by-day, minute-by-minute, and second-by-second basis, is it any wonder? The sheer volume, depending on your business and what you want to know, requires not just a Data team, but must now include someone with a particular skillset, including the tech-savvy analyst who can speak to the executive team.So, who holds it all together? These swathes of information. Who organizes the information in a cohesive order, so anyone with a map, can make their own analyses? Enter the Analytics Engineer.What Makes an Analytics Engineer an Analytics Engineer?Though it’s a rather new speciality within the Data Scientist scope—think Machine Learning Engineer, Software Engineer, Business Analyst, etc—at its core, the definition of an Analytics Engineer is this: “The analytics engineer sits at the intersection of the skill sets of data scientists, analysts, and data engineers. They bring a formal and rigorous software engineering practice to the efforts of analysts and data scientists, and they bring an analytical and business-outcomes mindset to the efforts of data engineering.” Michael Kaminsky, consultant, and former Director of Analytics at Harry’s.In other words, analytics engineers, using best software engineering practices transform data through testing and documentation so that data analysts begin with cleaner data to analyze. As technically savvy as the engineer must be, they must also be able to explain to stakeholders what they’re looking at so they can formulate their own insights. Five Roles and Responsibilities of the Analytics EngineerLike all new niche specialities, there are core responsibilities to consider as well as that of skillsets required to either study to become an Analytics Engineer or to discover if you’re one already. How? Consider the questions you ask, your studies within Data Science, Computer Science, Statistics, and Math, and your balance between technical skills and soft skills. Below are five things to consider when thinking about this role:Programming language experience. Experience with programming languages like R and Python along with strong SQL skills which are at the core of this role. DBT technology knowledge. As the driving force behind the rise of Analytics Engineer as a separate role, it’s imperative anyone interested in pursuing it should have a firm grasp of DBT — the Data Build Tool — that allows the implementation of analytics code using SQL. Data tracking expertise using Git. Data modelling. Clean, tested, and raw data which allow executives and analysts to view their Data, understand it within the database or its warehouse. Data transformation. Analytics Engineers determine what Data is most useful and transform it to ensure it fits related tasks. It’s part of building the foundational layer so businesses can answer their own questions. Key Changes Leading to the Shift in Data RolesWith every technological advancement their comes new players to the field. The difference is here is that the job description already existed. We were only missing a title. But from the traditional Data team to the modern Data team, there are a few key changes that point directly to the rise of this niche field. Cloud warehouses (like Snowflake, Redshift, BigQuery) and the arrival of the DBT the foundational layer which can be built on top of modern data warehouses are the first two that come to mind. Then, the Software-as-a-Service (SaaS) tools like Stitch and Hevo are capable of integrating Data from a variety of sources, and the introduction of tools like Mode and Looker allows anyone interested in drawing insight from Data to do so on their own.Who Needs an Analytics Engineer? Small or Large Businesses?The short answer is it depends. But the general rule follows that while both large and small companies can benefit from having this professional on their staff, there are different things to consider. For example, a small business may be able to find what they need in a single individual. The Analytics Engineer is something of a jack-of-all-trades. Larger businesses, on the other hand, may already have a Data team in place. In this case, an Analytics Engineer adds to your team, something like an additional set of eyes increasing insight drawn from those large swathes of Data we spoke about earlier.So, what’s next for the role of Analytics Engineer? Who knows? The roles of any Data industry professional is constantly evolving. If you’re interested in Analytics Engineering, Machine Learning, Data Science, or Business Intelligence just to name a few, Harnham may have a role for you. Check out our latest Data & Analytics Engineering jobs or contact one of our expert consultants to learn more. For our West Coast Team, contact us at (415) 614 – 4999 or send an email to firstname.lastname@example.org. For our Arizona Team, contact us at (602) 562 7011 or send an email to email@example.com. For our Mid-West and East Coast teams contact us at (212) 796-6070 or send an email to firstname.lastname@example.org.
What’s Hot in NYC’s Data Market? Modern Analytics Engineering is on the Rise
New York has always set the stage for what’s next. When it comes to the latest in the tech stack, it’s modern Analytics Engineering is the latest addition to the Data and Analytics industry. The role of Analytics Engineer is one of the newer positions in the world of Data, and in NYC, a hub of media, advertising, and e-commerce – it’s emerging as one of the most in-demand markets in New York and beyond.
Why You Might Need an Analytics Engineer
Data-driven businesses interested in building value for their customers often turn to a mix of Analytics and Data Modelling Engineer. The Data Engineer creates the infrastructure, platform development, and Data movement for the purpose of Machine Learning and Analytics downstream. Ultimately, the Analytics Engineer role is quite similar to the typical Data Engineer but differs in that it doesn’t involve platform development or infrastructure the same way.
Analytics Engineering is a relatively new term within the last five years and are coming into this field from a variety of backgrounds. But the most in-demand background moving into this role is Data Engineering. Why? For the most part, it’s those individuals who can not only script in Python but also do Python programming on the backend.
Key Aspects of this Role:
- Warehouse architecture (e.g., Snowflake, Redshift, BigQuery), and Data Modeling with a popular and relatively new tool dbt (originally Fishtown Analytics), for use by Analysts.
- ETL Development
- Data visualization
- Other tech such as Fivetran, Stitch, and Python
With SQL and Data modelling being the real meat and potatoes of the position, people often move into an Analytics Engineering position that requires little Python experience – however, the salary you can expect if Engineering or Data Science experience and proficiency in Python is substantially higher. It poses an interesting opportunity for Analysts, Data modellers, and Data visualization folks interested in learning a modern engineering stack to make a transition into a more technical, and higher-paying role.
Why You May Want to Consider an Analytics Engineering Role
People move into this role from careers as Analysts, Data Scientists, Data Engineers, and even Software Engineers, a unique career progression in this industry. For the already heavily technical professionals – this is a role that provides both engineering challenges and the chance to work close to the business. Wherever you are on your career journey, Analytics Engineer is a great opportunity from a career growth perspective and can help get you where you want to go. You’re no cog in the wheel here. As an Analytics Engineer, you can help drive decisions that make an impact for your company.
Analytics Engineers on Your Team Can Drive Value for Your Business
Though this position is relatively new in the grand scheme of technological advances to help drive business, it is in demand and growing exponentially. So, it’s important to know if you’re business needs someone to fill this role, you need to know what you’re looking for. For companies, whose main objective is making Data-driven decisions regarding customer retention, marketing campaign conversion, supply chain analytics, etc.
The role of the Analytics Engineer can be a perfect addition to both managing large amounts of Data coming into the businesses and helping drive value.
Take a look at our latest Analytics Engineer jobs here or get in touch with one of our expert consultants to find out more:
For our West Coast Team, contact us at (415) 614 – 4999 or send an email to email@example.com.
For our Arizona Team, contact us at (602) 562 7011 or send an email to firstname.lastname@example.org.
For our Mid-West and East Coast teams contact us at (212) 796-6070 or send an email to email@example.com.
CAN’T FIND THE RIGHT OPPORTUNITY?
If you can’t see what you’re looking for right now, send us your CV anyway – we’re always getting fresh new roles through the door.