Model Validation Manager
Manchester, Greater Manchester / £60000 - £75000
£60000 - £75000
Manchester, Greater Manchester
MODEL VALIDATION MANAGER
This company is a leading international bank who have an established presence in the UK. They are seeking a Model Validation Manager to lead their internal validation team. This role will allow you to work across a range of retail and corporate models to improve and enhance these. This role offers an opportunity to work across a wide range of models in addition to excellent exposure and learning opportunities within the bank.
This position sits within the Internal Validation Team. You would be involved in:
- Reviewing a wide range of models from across the bank, including IRB, Climate, and Fraud models in addition to scorecards
- Creating challenger models to improve these and enhance performance
- Preparing reports for senior members and recommending how these can be optimised
- Liaising with senior stakeholders and presenting reviews clearly and concisely
- Managing the wider validation team and leading ad hoc projects
YOUR SKILLS AND EXPERIENCE
- Prior experience in statistical modelling, ideally in IRB, scorecards or other risk models
- Prior experience in the validation of credit risk models is highly desirable
- Good knowledge and experience in SAS ideally, or SQL/Python
- Strong communication skills and prior experience in reporting and presenting ideas
- An education to degree level, in a numerate discipline
SALARY AND BENEFITS
- Up to £75,000 base salary
- Pension contribution scheme
- Discretionary bonus
- 27 days holiday
- Car allowance
- Healthcare benefits
- Hybrid work model
HOW TO APPLY
Please register your interest by sending your CV to Rosie Walsh through the 'Apply' link
Data Science Interview Questions: What The Experts Say | Harnham Recruitment post
Our friends at Data Science Dojo have compiled a list of 101 actual Data Science interview questions that have been asked between 2016-2019 at some of the largest recruiters in the Data Science industry – Amazon, Microsoft, Facebook, Google, Netflix, Expedia, etc. Data Science is an interdisciplinary field and sits at the intersection of computer science, statistics/mathematics, and domain knowledge. To be able to perform well, one needs to have a good foundation in not one but multiple fields, and it reflects in the interview. They’ve divided the questions into six categories: Machine LearningData AnalysisStatistics, Probability, and MathematicsProgrammingSQLExperiential/Behavioural QuestionsOnce you’ve gone through all the questions, you should have a good understanding of how well you’re prepared for your next Data Science interview.
Machine LearningAs one will expect, Data Science interviews focus heavily on questions that help the company test your concepts, applications, and experience on machine learning. Each question included in this category has been recently asked in one or more actual Data Science interviews at companies such as Amazon, Google, Microsoft, etc. These questions will give you a good sense of what sub-topics appear more often than others. You should also pay close attention to the way these questions are phrased in an interview. Explain Logistic Regression and its assumptions.Explain Linear Regression and its assumptions.How do you split your data between training and validation?Describe Binary Classification.Explain the working of decision trees.What are different metrics to classify a dataset?What’s the role of a cost function?What’s the difference between convex and non-convex cost function?Why is it important to know bias-variance trade off while modeling?Why is regularisation used in machine learning models? What are the differences between L1 and L2 regularisation?What’s the problem of exploding gradients in machine learning?Is it necessary to use activation functions in neural networks?In what aspects is a box plot different from a histogram?What is cross validation? Why is it used?Can you explain the concept of false positive and false negative?Explain how SVM works.While working at Facebook, you’re asked to implement some new features. What type of experiment would you run to implement these features?What techniques can be used to evaluate a Machine Learning model?Why is overfitting a problem in machine learning models? What steps can you take to avoid it?Describe a way to detect anomalies in a given dataset.What are the Naive Bayes fundamentals?What is AUC – ROC Curve?What is K-means?How does the Gradient Boosting algorithm work?Explain advantages and drawbacks of Support Vector Machines (SVM).What is the difference between bagging and boosting?Before building any model, why do we need the feature selection/engineering step?How to deal with unbalanced binary classification?What is the ROC curve and the meaning of sensitivity, specificity, confusion matrix?Why is dimensionality reduction important?What are hyperparameters, how to tune them, how to test and know if they worked for the particular problem?How will you decide whether a customer will buy a product today or not given the income of the customer, location where the customer lives, profession, and gender? Define a machine learning algorithm for this.How will you inspect missing data and when are they important for your analysis?How will you design the heatmap for Uber drivers to provide recommendation on where to wait for passengers? How would you approach this?What are time series forecasting techniques?How does a logistic regression model know what the coefficients are?Explain Principle Component Analysis (PCA) and it’s assumptions.Formulate Latent Semantic Analysis (LSA) and Latent Dirichlet Allocation (LDA) techniques.What are neural networks used for?40. Why is gradient checking important?Is random weight assignment better than assigning same weights to the units in the hidden layer?How to find the F1 score after a model is trained?How many topic modeling techniques do you know of? Explain them briefly.How does a neural network with one layer and one input and output compare to a logistic regression?Why Rectified Linear Unit/ReLU is a good activation function?When using the Gaussian mixture model, how do you know it’s applicable?If a Product Manager says that they want to double the number of ads in Facebook’s Newsfeed, how would you figure out if this is a good idea or not?What do you know about LSTM?Explain the difference between generative and discriminative algorithms.Can you explain what MapReduce is and how it works? If the model isn’t perfect, how would you like to select the threshold so that the model outputs 1 or 0 for label?Are boosting algorithms better than decision trees? If yes, why?What do you think are the important factors in the algorithm Uber uses to assign rides to drivers?How does speech synthesis works?
Data AnalysisMachine Learning concepts are not the only area in which you’ll be tested in the interview. Data pre-processing and data exploration are other areas where you can always expect a few questions. We’re grouping all such questions under this category. Data Analysis is the process of evaluating data using analytical and statistical tools to discover useful insights. Once again, all these questions have been recently asked in one or more actual Data Science interviews at the companies listed above. What are the core steps of the data analysis process?How do you detect if a new observation is an outlier?Facebook wants to analyse why the “likes per user and minutes spent on a platform are increasing, but total number of users are decreasing”. How can they do that?If you have a chance to add something to Facebook then how would you measure its success?If you are working at Facebook and you want to detect bogus/fake accounts. How will you go about that?What are anomaly detection methods?How do you solve for multicollinearity?How to optimise marketing spend between various marketing channels?What metrics would you use to track whether Uber’s strategy of using paid advertising to acquire customers works?What are the core steps for data preprocessing before applying machine learning algorithms?How do you inspect missing data?How does caching work and how do you use it in Data Science?
Statistics, Probability and MathematicsAs we’ve already mentioned, Data Science builds its foundation on statistics and probability concepts. Having a strong foundation in statistics and probability concepts is a requirement for Data Science, and these topics are always brought up in data science interviews. Here is a list of statistics and probability questions that have been asked in actual Data Science interviews.How would you select a representative sample of search queries from 5 million queries?Discuss how to randomly select a sample from a product user population.What is the importance of Markov Chains in Data Science?How do you prove that males are on average taller than females by knowing just gender or height.What is the difference between Maximum Likelihood Estimation (MLE) and Maximum A Posteriori (MAP)?What does P-Value mean?Define Central Limit Theorem (CLT) and it’s application?There are six marbles in a bag, one is white. You reach in the bag 100 times. After drawing a marble, it is placed back in the bag. What is the probability of drawing the white marble at least once?Explain Euclidean distance.Define variance.How will you cut a circular cake into eight equal pieces?What is the law of large numbers?How do you weigh nine marbles three times on a balance scale to select the heaviest one?You call three random friends who live in Seattle and ask each independently if it’s raining. Each of your friends has a 2/3 chance of telling you the truth and a 1/3 chance of lying. All three say “yes”. What’s the probability it’s actually raining?Explain a probability distribution that is not normal and how to apply that?You have two dice. What is the probability of getting at least one four? Also find out the probability of getting at least one four if you have n dice.Draw the curve log(x+10)
ProgrammingWhen you appear for a data science interview your interviewers are not expecting you to come up with a highly efficient code that takes the lowest resources on computer hardware and executes it quickly. However, they do expect you to be able to use R, Python, or SQL programming languages so that you can access the data sources and at least build prototypes for solutions.You should expect a few programming/coding questions in your data science interviews. You interviewer might want you to write a short piece of code on a whiteboard to assess how comfortable you are with coding, as well as get a feel for how many lines of codes you typically write in a given week. Here are some programming and coding questions that companies like Amazon, Google, and Microsoft have asked in their Data Science interviews. Write a function to check whether a particular word is a palindrome or not.Write a program to generate Fibonacci sequence.Explain about string parsing in R languageWrite a sorting algorithm for a numerical dataset in Python.Coding test: moving average Input 10, 20, 30, 10, … Output: 10, 15, 20, 17.5, …Write a Python code to return the count of words in a stringHow do you find percentile? Write the code for itWhat is the difference between – (i) Stack and Queue and (ii) Linked list and Array?
Structured Query Language (SQL)Real-world data is stored in databases and it ‘travels’ via queries. If there’s one language a Data Science professional must know, it’s SQL – or “Structured Query Language”. SQL is widely used across all job roles in Data Science and is often a ‘deal-breaker’. SQL questions are placed early on in the hiring process and used for screening. Here are some SQL questions that top companies have asked in their Data Science interviews. How would you handle NULLs when querying a data set?How will you explain JOIN function in SQL in the simplest possible way?Select all customers who purchased at least two items on two separate days from Amazon.What is the difference between DDL, DML, and DCL?96. Why is Database Normalisation Important?What is the difference between clustered and non-clustered index?
Situational/Behavioural QuestionsCapabilities don’t necessarily guarantee performance. It’s for this reason employers ask you situational or behavioural questions in order to assess how you would perform in a given situation. In some cases, a situational or behavioural question would force you to reflect on how you behaved and performed in a past situation. A situational question can help interviewers in assessing your role in a project you might have included in your resume, can reveal whether or not you’re a team player, or how you deal with pressure and failure. Situational questions are no less important than any of the technical questions, and it will always help to do some homework beforehand. Recall your experience and be prepared! Here are some situational/behavioural questions that large tech companies typically ask: What was the most challenging project you have worked on so far? Can you explain your learning outcomes?According to your judgement, does Data Science differ from Machine Learning?If you’re faced with Selection Bias, how will you avoid it?How would you describe Data Science to a Business Executive?
If you’re looking for new Data Science role, you can find our latest opportunities here. This article was written by Tooba Mukhtar and Rahim Rasool for Data Science Jojo. It has been republished with permission. You can view the original article, which includes answers to the above questions here.
Ten Tips for Writing the Perfect Data & Analytics CV | Harnham Recruitment post
It’s no secret that jobs within the Data & Analytics market are more competitive than ever and with some jobs having hundreds of applicants (if not more), having a CV that stands out is more important than ever. It’s well known that many Hiring Managers spend a short amount of time reviewing a candidate, so you need to consider what they can do to have the best impact. We’ve seen it all over the years, from resumes sorely lacking detail through to those that have almost every accomplishment written over too many pages – so we’ve complied a list of the 10 things that could help you create a resume that makes an impact, complete with top tips from our team of experienced recruiters.1. Keep it Simple All of our recruiters are unanimous in suggesting to candidates that the perfect CV length is no more than two pages, or one for a graduate or more junior candidate. Sam, our Corporate Accounts manager suggests that candidates keep it simple:“In analytics, it’s all about the detail and less about how fun your CV looks. My best piece of advice would be to keep it to two pages, use the same font without boxes or pictures, and bold titles for the company and role. It sounds pretty simple but it’s really effective and often what our clients seem to be drawn to the most”. 2. Consider the audience & avoid jargon Before your CV gets to the Hiring Manager, it may be screened by an HR or recruitment professional so it’s crucial to ensure that your CV is understandable enough that every person reviewing it could gauge your fit. Whilst showing your technical ability is important, ensure that you save yourself from anything excessively technical meaning only the Hiring Manager could understand what you have been doing. 3. Showcase your technical skills There is, of course, a need to showcase your technical skills. However, you should avoid a long list of technologies, instead clarify your years of experience and competence with each of the tools. Within the Data & Analytics market specifically, clarifying the tools that you used to analyse or model is very important and writing those within your work experience can be very helpful. Wesley, who heads up our French team, explained where candidates can often go wrong: “Candidates often write technical languages on their CV in long lists and forget to make them come to life. My clients are looking for them to give examples of how and when they have used the listed tools and languages”4. Consider the impact of your workJust writing words such as ‘leadership’ or ‘collaboration’ can often easily be over-looked. It’s important that you are able to showcase the impact that you work has beyond the traditionally technical. Think about how you can showcase the projects that you have lead or contributed to and what impact it had on the business. Often people forget the CV isn’t about listing your duties, it’s about listening your accomplishments. Ewan, our Nordics Senior Manager brings this to life: “I would always tell someone that whenever you are stating something you did in a job you always follow up with the result of that. For example, ‘I implemented an Acquisition Credit Risk Strategy from start to finish’ – but then adding, ‘which meant that we saw an uplift of 15% of credit card use’”. Joe, New York Senior Manager, concurs: “Actionable insights are important, results driven candidates are what our clients are looking for. So instead of ‘Implemented A/B Testing’, I’d get my candidates to make that more commercial, such as ‘Implemented A/B test that result in 80% increase in conversion’”. 5. Use your Personal Summary A personal summary is effective when it comes to technical positions, as some people can often overlook them. Use this to summarise your experience and progression as well as indicate the type of role and opportunity you are looking for. If this is highly tailored to the role you are applying for, it can have an extremely positive impact. For example: ‘Highly accomplished Data Scientist, with proven experience in both retail and banking environments. Prior experience managing a team of five, and proven ability in both a strategic and hands on capabilities. Proven skills in Machine Learning and Statistical Modelling with advanced knowledge of Python, R and Hadoop. Seeking Data Science Manager role in a fast-paced organisation with data-centric thinking at it’s heart’. 6. Consider what work and non-work experience is relevant If you’ve been working in the commercial technical sphere for more than five years, it’s likely that your part time work experience during university or the non-technical roles that you took before you moved into your space are no longer as relevant. Ensure you are using your space to offer the Hiring Manager recent, relevant and commercially focused information. However, do not leave gaps just because you took a role that didn’t relate to your chosen field, you don’t need to describe what you did but have the job title, company and dates to ensure you are highlighting a clear history of your experience. It’s important to note that you are more than just your work experience as well, Principal Consultant Conor advises candidates to talk about more than just their work accomplishments:“Listing non work achievements can help make the CV stand out. If someone has a broad range of achievements and proven drive outside of work, they will probably be good at their job too. Plus, it’s a differentiating point. My clients have found interesting talking points with people who have excelled in sports, instruments, languages and more specifically for the Analytics community – things like maths and Rubik’s cube competitions”. 7. Don’t forget your education For most technical roles, education is an important factor. Ensure that you include your degree and university/college clearly as well as the technical exposure you had within this. If you did not undertake a traditionally technical subject, make sure you highlight further courses and qualifications that you have completed near this section to highlight to the Hiring Manager that you have the relevant level of technical competence for the role. 8. Don’t include exaggerated statementsIt goes without saying that if you are going to detail your experience with a certain technical tool or software that you could be asked to evidence it. Saying your proficient in R when you’ve done a few courses on it won’t go over well, especially if there are technical tests involved in the interview process. At the same time, don’t undervalue your expertise in certain areas either, your strengths are what the Hiring Managers is looking for. 9. Don’t get too creativeUnless you’re in a creative role it’s unlikely that the Hiring Manager will be looking for something unique when it comes to the CV. In fact, very few people can pull of an overly flashy CV, most of them being those that work specifically in design. When in doubt, stick to standard templates and muted tones. 10. Tailor, Tailor, Tailor! Time is of the essence and when it comes to reviewing CVs and you don’t have long to make an impact. Make sure to customise your resume using keywords and phrases that match the job description (if they match your own, of course). For example, if the role is looking for a Business Intelligence Analyst with proven skills in Tableau you would not just claim, “experience in Data Visualisation”, you’d list the software name, “experience in Tableau based Data Visualisation”. Although every job description is different, all it takes is a few small tweaks to ensure your maximising your skillset. If you’re looking for your next Data & Analytics role or are seeking the best candidates on the market, we may be able to help. Take a look at our latest opportunities or get in touch with one of our expert consultants to find out more.
Unicorn Operations: An Intro to MLOps | Harnham US Recruitment post
If the wheels of technology seem to be spinning faster than ever before, they are. But instead of separate circles, it’s more like the magic rings trick in which the magician links them all together to perform. In layman’s terms, that would be Machine Learning (ML) Ops. Rather than the unicorn employee – everything a company desires in an individual rolled into one – this unicorn practice employs and is built upon three technologies that are used to operationalize businesses to keep those wheels turning. If DevOps was the sheriff of 2021, then ML Ops is the new sheriff in town.What is Machine Learning Operations (ML Ops)?It seems self-explanatory as a combination of machine learning and operations, but the technology behind it delves a bit deeper. It is a collaboration of engineering practices designed by Data Scientists, Operations, and Data Engineers using machine learning, model development, deployment, and data management to wrangle the enormous amounts of Data businesses now must navigate to gain actionable insights for their business.But where ML Ops shines is in its deployment capabilities. After all, once the business makes their decision based on actionable insights from the Data, then the next step is to implement it and put it into play. It is the de facto operations product lifecycle, the wheel that keeps the business moving forward. Ultimately, its goal is to automate the Machine Learning lifecycle from modelling to implementation to retraining once new Data gets into the mix. Because there will always be new Data and the world is not one-size-fits-all.4 ML Ops Pipeline StepsOne of the elements of MLOps lifecycle systems automation is that of continuous learning and retraining. Think of it like this. If you know the movie War Games, J.O.S.H.U.A knows how to play the games, but he doesn’t understand them; the whys and the hows. When it comes to the end of the movie and the game (spoiler alert), he has ‘learned’ human behavior and must make the decision to stop the game. Granted, he was an 80s realized version of early Artificial Intelligence, but ultimately, he was a machine who learned. Machine Learning can expand its knowledge base to integrate Data and model validation, that is part of the draw of ML Ops. Not only can Data and Operations professionals address complexities of deployments but they can create predefined steps to emulate and consider factors such as company size, project, and Machine Learning capabilities and complexities.Data Ingestion – Takes Data in and determines through which pipeline it should continue. What Data will be used in training? Which for validation sets? And which should be combined into one multi-streamed dataset.Data Validation – Once the Data has been taken in, the role of Data validation is to see if there are any anomalies. This focus not only lets you know how your Data has changed over time.Data Preparation – Data is cleaned and prepared to fit into the right format so your model can follow it. At this stage, also, Data may be combined with domain knowledge featuring engineering to build new features and solutions.Model Training – At the core of the pipeline is Model training which uses ingested Data to help launch trainings in sequence or parallel to determine what might be needed for a production model as well. There are three ways to launch such models and they include:Embedded in an appOn and IoT deviceIn a specialized dedicated web service using remote procedure call (RP), for example.Where Do We Go From Here?MLOps isn’t the only technology predicated on learning and driving an automated pipeline that will free up Data professionals to focus on the bigger picture. Though built on DevOps principles and in collaboration with a long unsiloed team of both technical and non-technical professionals, ML Ops has grown in popularity over the years and shows no sign of slowing down any time soon. As Open-Source networks emerge and ML Ops teams get created to help navigate this new technology, it will only continue to grow and reach new heights. Businesses who understand and develop these strategies now will have their foot firmly in the future and head and shoulders above their competition.If you’re interested in Digital Analytics, Machine Learning, or Robotics just to name a few, Harnham may have a role for you. Check out our latest MLOps jobs or contact one of our expert consultants to learn more. For our West Coast Team, contact us at (415) 614 – 4999 or send an email to firstname.lastname@example.org. For our Arizona Team, contact us at (602) 562 7011 or send an email to email@example.com. For our Mid-West and East Coast teams contact us at (212) 796-6070 or send an email to firstname.lastname@example.org.
CAN’T FIND THE RIGHT OPPORTUNITY?
If you can’t see what you’re looking for right now, send us your CV anyway – we’re always getting fresh new roles through the door.