Senior Data Engineer/Machine Learning Engineer
Birmingham, West Midlands / £70000 - £90000
£70000 - £90000
Birmingham, West Midlands
Senior Data Engineer/Machine Learning Engineer
Hybrid - 1x/2x per month in office, London
An interesting opportunity for a Data Engineer with Machine Learning experience to join the Data team of a popular family-advice website.
The company is a website offering support and advice to expecting and young parents. The company also partners with governmental bodies to offer support to underprivileged or vulnerable families.
The data team is made up of Data Scientists, Insights Analysts, and Developers. This is a standalone Data Engineering role, and will be a hybrid between Machine Learning Engineering and classic Data Engineering.
The role and responsibilities
- Productionising Machine Learning algorithms
- Building out APIs to connect data sources
- Maintaining AWS/GCP architecture to keep the platform robust
- Building out a data lake to improve analytics function
Your skills and experience
- Experience with AWS/GCP and a thorough understanding of AWS/GCP architecture
- Experience with Python and SQL
- Prior experience with Machine Learning or exposure to Data Science
- Experience with APIs
- Up to £90, 000 salary
- Leisure, retail and wellness perks
- Flexible working policy
Data Engineer Or Software Engineer: What Does Your Business Need? | Harnham US Recruitment post
We are in a time in which what we do with Data matters. Over the last few years, we have seen a rapid rise in the number of Data Scientists and Machine Learning Engineers as businesses look to find deeper insights and improve their strategies. But, without proper access to the right Data that has been processed and massaged, Data Scientists and Machine Learning Engineers would be unable to do their job properly. So who are the people who work in the background and are responsible to make sure all of this works? The quick answer is Data Engineers!… or is it? In reality, there are two similar, yet different profiles who can help help a company achieve their Data-driven goals. Data Engineers When people think of Data Engineers, they think of people who make Data more accessible to others within an organization. Their responsibility is to make sure the end user of the Data, whether it be an Analyst, Data Scientist, or an executive, can get accurate Data from which the business can make insightful decisions. They are experts when it comes to data modeling, often working with SQL. Frequently, “modern” Data Engineers work with a number of tools including Spark, Kafka, and AWS (or any cloud provider), whilst some newer Databases/Data Warehouses include Mongo DB and Snowflake. Companies are choosing to leverage these technologies and update their stack because it allows Data teams to move at a much faster pace and be able to deliver results to their stakeholders. An enterprise looking for a Data Engineer will need someone to focus more on their Data Warehouse and utilize their strong knowledge of querying information, whilst constantly working to ingest/process Data. Data Engineers also focus more on Data Flow and knowing how each Data sets works in collaboration with one another. Software Engineers – DataSimilar to a Data Engineers, Software Engineers – Data ( who I will refer to as Software Data Engineers in this article) also build out Data Pipelines. These individuals might go by different names like Platform or Infrastructure Engineer. They have to be good with SQL and Data Modeling, working with similar technologies such as Spark, AWS, and Hadoop. What separates Software Data Engineers from Data Engineers is the necessity to look at things from a macro-level. They are responsible for building out the cluster manager and scheduler, the distributed cluster system, and implementing code to make things function faster and more efficiently. Software Data Engineers are also better programers. Frequently, they will work in Python, Java, Scala, and more recently, Golang. They also work with DevOps tools such as Docker, Kubernetes, or some sort of CI/CD tool like Jenkins. These skills are critical as Software Data Engineers are constantly testing and deploying new services to make systems more efficient. This is important to understand, especially when incorporating Data Science and Machine Learning teams. If Data Scientists or Machine Learning Engineers do not have a strong Software Engineers in place to build their platforms, the models they build won’t be fully maximized. They also have to be able to scale out systems as their platform grows in order to handle more Data, while finding ways to make improvements. Software Data Engineers will also be looking to work with Data Scientists and Machine Learning Engineers in order to understand the prerequisites of what is needed to support a Machine Learning model. Which is right for your business? If you are looking for someone who can focus extensively on pulling Data from a Data source or API, before transforming or “massaging” the Data, and then moving it elsewhere, then you are looking for a Data Engineer. Quality Data Engineers will be really good at querying Data and Data Modeling and will also be good at working with Data Warehouses and using visualization tools like Tableau or Looker. If you need someone who can wear multiple hats and build highly scalable and distributed systems, you are looking for a Software Data Engineer. It’s more common to see this role in smaller companies and teams, since Hiring Managers often need someone who can do multiple tasks due to budget constraints and the need for a leaner team. They will also be better coders and have some experience working with DevOps tools. Although they might be able to do more than a Data Engineer, Software Data Engineers may not be as strong when it comes to the nitty gritty parts of Data Engineering, in particular querying Data and working within a Data Warehouse. It is always a challenge knowing which type of job to recruit for. It is not uncommon to see job posts where companies advertise that they are looking for a Data Engineer, but in reality are looking for a Software Data Engineer or Machine Learning Platform Engineer. In order to bring the right candidates to your door, it is crucial to have an understanding of what responsibilities you are looking to be fulfilled.That’s not to say a Data Engineer can’t work with Docker or Kubernetes. Engineers are working in a time where they need to become proficient with multiple tools and be constantly honing their skills to keep up with the competition. However, it is this demand to keep up with the latest tech trends and choices that makes finding the right candidate difficult. Hiring Managers need to identify which skills are essential for the role from the start, and which can be easily picked up on the job. Hiring teams should focus on an individual’s past experience and the projects they have worked on, rather than looking at their previous job titles. If you’re looking to hire a Data Engineer or a Software Data Engineer, or to find a new role in this area, we may be able to help. Take a look at our latest opportunities or get in touch if you have any questions.
Data Science Interview Questions: What The Experts Say | Harnham Recruitment post
Our friends at Data Science Dojo have compiled a list of 101 actual Data Science interview questions that have been asked between 2016-2019 at some of the largest recruiters in the Data Science industry – Amazon, Microsoft, Facebook, Google, Netflix, Expedia, etc. Data Science is an interdisciplinary field and sits at the intersection of computer science, statistics/mathematics, and domain knowledge. To be able to perform well, one needs to have a good foundation in not one but multiple fields, and it reflects in the interview. They’ve divided the questions into six categories: Machine LearningData AnalysisStatistics, Probability, and MathematicsProgrammingSQLExperiential/Behavioural QuestionsOnce you’ve gone through all the questions, you should have a good understanding of how well you’re prepared for your next Data Science interview.
Machine LearningAs one will expect, Data Science interviews focus heavily on questions that help the company test your concepts, applications, and experience on machine learning. Each question included in this category has been recently asked in one or more actual Data Science interviews at companies such as Amazon, Google, Microsoft, etc. These questions will give you a good sense of what sub-topics appear more often than others. You should also pay close attention to the way these questions are phrased in an interview. Explain Logistic Regression and its assumptions.Explain Linear Regression and its assumptions.How do you split your data between training and validation?Describe Binary Classification.Explain the working of decision trees.What are different metrics to classify a dataset?What’s the role of a cost function?What’s the difference between convex and non-convex cost function?Why is it important to know bias-variance trade off while modeling?Why is regularisation used in machine learning models? What are the differences between L1 and L2 regularisation?What’s the problem of exploding gradients in machine learning?Is it necessary to use activation functions in neural networks?In what aspects is a box plot different from a histogram?What is cross validation? Why is it used?Can you explain the concept of false positive and false negative?Explain how SVM works.While working at Facebook, you’re asked to implement some new features. What type of experiment would you run to implement these features?What techniques can be used to evaluate a Machine Learning model?Why is overfitting a problem in machine learning models? What steps can you take to avoid it?Describe a way to detect anomalies in a given dataset.What are the Naive Bayes fundamentals?What is AUC – ROC Curve?What is K-means?How does the Gradient Boosting algorithm work?Explain advantages and drawbacks of Support Vector Machines (SVM).What is the difference between bagging and boosting?Before building any model, why do we need the feature selection/engineering step?How to deal with unbalanced binary classification?What is the ROC curve and the meaning of sensitivity, specificity, confusion matrix?Why is dimensionality reduction important?What are hyperparameters, how to tune them, how to test and know if they worked for the particular problem?How will you decide whether a customer will buy a product today or not given the income of the customer, location where the customer lives, profession, and gender? Define a machine learning algorithm for this.How will you inspect missing data and when are they important for your analysis?How will you design the heatmap for Uber drivers to provide recommendation on where to wait for passengers? How would you approach this?What are time series forecasting techniques?How does a logistic regression model know what the coefficients are?Explain Principle Component Analysis (PCA) and it’s assumptions.Formulate Latent Semantic Analysis (LSA) and Latent Dirichlet Allocation (LDA) techniques.What are neural networks used for?40. Why is gradient checking important?Is random weight assignment better than assigning same weights to the units in the hidden layer?How to find the F1 score after a model is trained?How many topic modeling techniques do you know of? Explain them briefly.How does a neural network with one layer and one input and output compare to a logistic regression?Why Rectified Linear Unit/ReLU is a good activation function?When using the Gaussian mixture model, how do you know it’s applicable?If a Product Manager says that they want to double the number of ads in Facebook’s Newsfeed, how would you figure out if this is a good idea or not?What do you know about LSTM?Explain the difference between generative and discriminative algorithms.Can you explain what MapReduce is and how it works? If the model isn’t perfect, how would you like to select the threshold so that the model outputs 1 or 0 for label?Are boosting algorithms better than decision trees? If yes, why?What do you think are the important factors in the algorithm Uber uses to assign rides to drivers?How does speech synthesis works?
Data AnalysisMachine Learning concepts are not the only area in which you’ll be tested in the interview. Data pre-processing and data exploration are other areas where you can always expect a few questions. We’re grouping all such questions under this category. Data Analysis is the process of evaluating data using analytical and statistical tools to discover useful insights. Once again, all these questions have been recently asked in one or more actual Data Science interviews at the companies listed above. What are the core steps of the data analysis process?How do you detect if a new observation is an outlier?Facebook wants to analyse why the “likes per user and minutes spent on a platform are increasing, but total number of users are decreasing”. How can they do that?If you have a chance to add something to Facebook then how would you measure its success?If you are working at Facebook and you want to detect bogus/fake accounts. How will you go about that?What are anomaly detection methods?How do you solve for multicollinearity?How to optimise marketing spend between various marketing channels?What metrics would you use to track whether Uber’s strategy of using paid advertising to acquire customers works?What are the core steps for data preprocessing before applying machine learning algorithms?How do you inspect missing data?How does caching work and how do you use it in Data Science?
Statistics, Probability and MathematicsAs we’ve already mentioned, Data Science builds its foundation on statistics and probability concepts. Having a strong foundation in statistics and probability concepts is a requirement for Data Science, and these topics are always brought up in data science interviews. Here is a list of statistics and probability questions that have been asked in actual Data Science interviews.How would you select a representative sample of search queries from 5 million queries?Discuss how to randomly select a sample from a product user population.What is the importance of Markov Chains in Data Science?How do you prove that males are on average taller than females by knowing just gender or height.What is the difference between Maximum Likelihood Estimation (MLE) and Maximum A Posteriori (MAP)?What does P-Value mean?Define Central Limit Theorem (CLT) and it’s application?There are six marbles in a bag, one is white. You reach in the bag 100 times. After drawing a marble, it is placed back in the bag. What is the probability of drawing the white marble at least once?Explain Euclidean distance.Define variance.How will you cut a circular cake into eight equal pieces?What is the law of large numbers?How do you weigh nine marbles three times on a balance scale to select the heaviest one?You call three random friends who live in Seattle and ask each independently if it’s raining. Each of your friends has a 2/3 chance of telling you the truth and a 1/3 chance of lying. All three say “yes”. What’s the probability it’s actually raining?Explain a probability distribution that is not normal and how to apply that?You have two dice. What is the probability of getting at least one four? Also find out the probability of getting at least one four if you have n dice.Draw the curve log(x+10)
ProgrammingWhen you appear for a data science interview your interviewers are not expecting you to come up with a highly efficient code that takes the lowest resources on computer hardware and executes it quickly. However, they do expect you to be able to use R, Python, or SQL programming languages so that you can access the data sources and at least build prototypes for solutions.You should expect a few programming/coding questions in your data science interviews. You interviewer might want you to write a short piece of code on a whiteboard to assess how comfortable you are with coding, as well as get a feel for how many lines of codes you typically write in a given week. Here are some programming and coding questions that companies like Amazon, Google, and Microsoft have asked in their Data Science interviews. Write a function to check whether a particular word is a palindrome or not.Write a program to generate Fibonacci sequence.Explain about string parsing in R languageWrite a sorting algorithm for a numerical dataset in Python.Coding test: moving average Input 10, 20, 30, 10, … Output: 10, 15, 20, 17.5, …Write a Python code to return the count of words in a stringHow do you find percentile? Write the code for itWhat is the difference between – (i) Stack and Queue and (ii) Linked list and Array?
Structured Query Language (SQL)Real-world data is stored in databases and it ‘travels’ via queries. If there’s one language a Data Science professional must know, it’s SQL – or “Structured Query Language”. SQL is widely used across all job roles in Data Science and is often a ‘deal-breaker’. SQL questions are placed early on in the hiring process and used for screening. Here are some SQL questions that top companies have asked in their Data Science interviews. How would you handle NULLs when querying a data set?How will you explain JOIN function in SQL in the simplest possible way?Select all customers who purchased at least two items on two separate days from Amazon.What is the difference between DDL, DML, and DCL?96. Why is Database Normalisation Important?What is the difference between clustered and non-clustered index?
Situational/Behavioural QuestionsCapabilities don’t necessarily guarantee performance. It’s for this reason employers ask you situational or behavioural questions in order to assess how you would perform in a given situation. In some cases, a situational or behavioural question would force you to reflect on how you behaved and performed in a past situation. A situational question can help interviewers in assessing your role in a project you might have included in your resume, can reveal whether or not you’re a team player, or how you deal with pressure and failure. Situational questions are no less important than any of the technical questions, and it will always help to do some homework beforehand. Recall your experience and be prepared! Here are some situational/behavioural questions that large tech companies typically ask: What was the most challenging project you have worked on so far? Can you explain your learning outcomes?According to your judgement, does Data Science differ from Machine Learning?If you’re faced with Selection Bias, how will you avoid it?How would you describe Data Science to a Business Executive?
If you’re looking for new Data Science role, you can find our latest opportunities here. This article was written by Tooba Mukhtar and Rahim Rasool for Data Science Jojo. It has been republished with permission. You can view the original article, which includes answers to the above questions here.
Keepers of the Data Kingdom: the Analytics Engineer | Harnham US Recruitment post
If it seems the Data world is drilling down further into niche specialities, you’re right. Considering the swathes of information sent and received on a day-by-day, minute-by-minute, and second-by-second basis, is it any wonder? The sheer volume, depending on your business and what you want to know, requires not just a Data team, but must now include someone with a particular skillset, including the tech-savvy analyst who can speak to the executive team.So, who holds it all together? These swathes of information. Who organizes the information in a cohesive order, so anyone with a map, can make their own analyses? Enter the Analytics Engineer.What Makes an Analytics Engineer an Analytics Engineer?Though it’s a rather new speciality within the Data Scientist scope—think Machine Learning Engineer, Software Engineer, Business Analyst, etc—at its core, the definition of an Analytics Engineer is this: “The analytics engineer sits at the intersection of the skill sets of data scientists, analysts, and data engineers. They bring a formal and rigorous software engineering practice to the efforts of analysts and data scientists, and they bring an analytical and business-outcomes mindset to the efforts of data engineering.” Michael Kaminsky, consultant, and former Director of Analytics at Harry’s.In other words, analytics engineers, using best software engineering practices transform data through testing and documentation so that data analysts begin with cleaner data to analyze. As technically savvy as the engineer must be, they must also be able to explain to stakeholders what they’re looking at so they can formulate their own insights. Five Roles and Responsibilities of the Analytics EngineerLike all new niche specialities, there are core responsibilities to consider as well as that of skillsets required to either study to become an Analytics Engineer or to discover if you’re one already. How? Consider the questions you ask, your studies within Data Science, Computer Science, Statistics, and Math, and your balance between technical skills and soft skills. Below are five things to consider when thinking about this role:Programming language experience. Experience with programming languages like R and Python along with strong SQL skills which are at the core of this role. DBT technology knowledge. As the driving force behind the rise of Analytics Engineer as a separate role, it’s imperative anyone interested in pursuing it should have a firm grasp of DBT — the Data Build Tool — that allows the implementation of analytics code using SQL. Data tracking expertise using Git. Data modelling. Clean, tested, and raw data which allow executives and analysts to view their Data, understand it within the database or its warehouse. Data transformation. Analytics Engineers determine what Data is most useful and transform it to ensure it fits related tasks. It’s part of building the foundational layer so businesses can answer their own questions. Key Changes Leading to the Shift in Data RolesWith every technological advancement their comes new players to the field. The difference is here is that the job description already existed. We were only missing a title. But from the traditional Data team to the modern Data team, there are a few key changes that point directly to the rise of this niche field. Cloud warehouses (like Snowflake, Redshift, BigQuery) and the arrival of the DBT the foundational layer which can be built on top of modern data warehouses are the first two that come to mind. Then, the Software-as-a-Service (SaaS) tools like Stitch and Hevo are capable of integrating Data from a variety of sources, and the introduction of tools like Mode and Looker allows anyone interested in drawing insight from Data to do so on their own.Who Needs an Analytics Engineer? Small or Large Businesses?The short answer is it depends. But the general rule follows that while both large and small companies can benefit from having this professional on their staff, there are different things to consider. For example, a small business may be able to find what they need in a single individual. The Analytics Engineer is something of a jack-of-all-trades. Larger businesses, on the other hand, may already have a Data team in place. In this case, an Analytics Engineer adds to your team, something like an additional set of eyes increasing insight drawn from those large swathes of Data we spoke about earlier.So, what’s next for the role of Analytics Engineer? Who knows? The roles of any Data industry professional is constantly evolving. If you’re interested in Analytics Engineering, Machine Learning, Data Science, or Business Intelligence just to name a few, Harnham may have a role for you. Check out our latest Data & Analytics Engineering jobs or contact one of our expert consultants to learn more. For our West Coast Team, contact us at (415) 614 – 4999 or send an email to firstname.lastname@example.org. For our Arizona Team, contact us at (602) 562 7011 or send an email to email@example.com. For our Mid-West and East Coast teams contact us at (212) 796-6070 or send an email to firstname.lastname@example.org.
CAN’T FIND THE RIGHT OPPORTUNITY?
If you can’t see what you’re looking for right now, send us your CV anyway – we’re always getting fresh new roles through the door.