With over 10 years experience working solely in the Data & Analytics sector our consultants are able to offer detailed insights into the industry.
Visit our Blogs & News portal or check out our recent posts below.
Josh joined Harnham in August of 2017 working in the growing Data and Technology team specialising in Data Engineering recruitment. As a Senior Recruitment Consultant Josh has helped to grow Harnham’s Data Engineering network whilst gaining a detailed understanding of the market.
With over 10 years experience working solely in the Data & Analytics sector our consultants are able to offer detailed insights into the industry.
Visit our Blogs & News portal or check out our recent posts below.
Data Engineers are the architects of Data. They lay the foundation businesses use to collect, gather, store, and make Data usable. Each iteration of the Data as it moves along the pipeline is cleaned and analysed to be used by Data professionals for their reports and Machine Learning models. A ROLE IN HIGH DEMAND Even as businesses reopen, reassess, and for some, remain remote, the demand for Data Engineers is high. Computer applications, Data modelling, prediction modelling, Machine Learning, and more need Data professionals to lay the groundwork to help businesses benefit in today’s Data-driven culture. The word gets thrown around a bit, but when the majority of business has moved online, Data-driven is the name of the game. Having a Data plan, a Data team, and all aligned with your business strategy is imperative to the way business is done today. This type of innovation can offer insight for better business decisions, enhance customer engagement, and improve customer retention without missing a beat. Without Data Engineers, Data Scientists can’t do their jobs. Understanding the amount of Data, the speed at which is delivered, and its variety need Engineers to create reliable and efficient systems. Like many Data professional jobs, even still in 2020, Data Engineers are in high demand. Yet a skills shortage remains. This has created an emerging field of professionals from other backgrounds who are looking to take on the role of Data Engineer and fill the gap. Whether by necessity or design, these individuals build and manage pipelines, automate projects, and see their projects through to the end result. CAREER OPPORTUNITIES OUTSIDE THE NORM As this growing trend emerges, it has created career opportunities for those with experience outside the normal channels of Data Engineering study. While it might involve individuals from backgrounds such as software Engineering, Databases, or something similarly IT-related, some businesses are upskilling their employees with talent. Rapid growth, reskilling, upskilling, and ever-constant changes still leave businesses with a shortage of Data Engineers to meet the demand. It’s critical to fill the gap for success. According to LinkedIn’s 2020 Emerging Jobs Report, Data Engineering is listed in the top 10 of jobs experiencing growth. THREE STEPS TOWARDS BECOMING A DATA ENGINEER This is a vital role in today’s organisations. So, if you’re in the tech industry and want to take a deeper dive into Data as a Data Engineer, what steps can you take? This is a time like no other. There’s time to assess your goals, take online classes, and get hands on with projects. Though having a base of computer science, mathematics, or business-related degree is always a good start. Be well-versed in such popular programming languages such as SQL, Python, R, Hadoop, Spark, and Amazon Web Services (AWS).Prepare for an entry-level role once you have your bachelor’s degree.Consider additional education to stay ahead of the curve. This can include not only professional certifications, but higher education degrees as well. The more experience, hands-on as well as academic, you have the more in demand you’ll be as a Data Engineer. Data scientists might be the rockstars of Data, but Data Engineers set the stage. As business processes have shifted online, looking for your next job has become more daunting than ever before. If you’re looking for your next opportunity in Data, take a look at our current jobs or get in touch with one of our expert consultants to find out more.
25. June 2020
Data Analyst. Data Wrangler. Data Architect? If you like pulling together threads of a company’s Data into one cohesive point, you may want to consider a Data Architect role. But what exactly is a Data Architect and how does it differ from a Data Engineer? Data Architect vs. Data Engineer As businesses continue to combine their Data and business strategies into one, they are beginning to understand to the need for a variety of Data Analysts. But as important as it is to have someone build your platform and begin pipeline processes, there is also need for someone with vision. Someone who can see patterns and designs. Someone who has end-to-end vision and can see how the patterns flow through your processes. This is your Data Architect. Data Engineers, on the other hand, lay the foundation for your Data platform. They draft the blueprint. After all, you can’t build a house without a blueprint first, right? The Data Engineer is at the beginning of the process, so the rest of the team can do their parts. But it’s the Data Architect who pulls it all together. THE ROLE OF THE DATA ARCHITECT If you’re considering your next career move and wondering if Data Architecture is for you, here are some typical requirements. A typical Data Architect will: Meet with stakeholders to understand business needs and translate them into technical requirements using ETL techniques to develop Data ArchitectureUnderstand their full Data lifecycle to provide technical architecture leadershipDesign a real-time data pipeline ecosystem and how to make it scalable usingDevelop Big Data Architecture in an AWS environmentBe educated to a degree level in a numerate discipline (Mathematics, Statistics, Computer Science, Computer Engineering)• Have proven experience in a commercial environmentHave advanced Cloud Computing Ecosystem experience with AWS (GCP or Azure also considered)Have proven Big Data Ecosystem experienceHave proven Big Data Architecture experience in a commercial environment Have proven Data Engineering experience in a commercial environment Though the likes of Google, IBM, and others have ramped up their education efforts, and online courses traditional universities offer a variety of Data Science degrees, there is still a shortage of professionals in the industry. So can businesses simplify and automate processes without the right people in place? Businesses Step Up Their Data Strategies Though there are easier ways to get the information a business needs through rented predictive modelling or an already drafted Data Science model, it doesn’t give the true value of Data. Add in new regulations, requirements, and new Data which offer new insights, and the impact on business is profound. It’s time for business to start ensuring that their Data teams are treated as critically as possible. Time to lay a path of progression, a pipeline, of systems and processes for the creation and production of Data. After all, simply optimising your Data will only get you so far. Enterprise-wide Data systems are more than wrangling and analysing Data. Most importantly, businesses need to ensure they have the right people in place. They also need to understand what they need and why they need it. This is a key part of Data Strategy and with the right people in place, can put your business ahead of the competition. Digging Deeper into Requirements for Top Talent While the standard requirements for a Data professional are to be educated to a degree level in things like Computer Science and Mathematics, technical skills, and experience within certain industries, for the natural progression from Data Analyst to Data Architect, there’s a bit more nuance to consider. Whether your business is just getting started in Data Science or you’re ready to start growing an existing team, there are some things you may want to focus on when looking for your Data Architect role. Define and determine how to keep projects streamlined with repeatable processes. Pivot between guiding team members through the pipeline and explaining insights to executives and stakeholders. Determine the right format for the right project. Determine when and when not to use automation to integrate Data. Visualise and extract models to predict future events and describe the process. In other words, be able to interpret Data to ensure reliability of the best approach. With the right talent in place, your teams can collaborate and build on their shared expertise to ensure Data is analysed and understood to the best benefit of your business. If you like solving puzzles, pulling disparate threads together into organised systems, and have experience as analysing and collecting Data, we may have a role for you. Take a look at our latest opportunities or get in touch with one of our expert consultants to find out more.
20. February 2020
Y2K was nearly 20 years ago. Remember when we were all worried about the massive changes that could mean; the preparations we made getting ready for the turn of the century? Ten years later, we scoffed at our worries and hopped on the Data bandwagon…some of us. Others are still trying to catch up but in recent years, most businesses have realised it isn’t a matter of “if” you should have a Data strategy and begin to build your team, it’s a matter of “if you don’t do it, you’ll be left behind.” As the year and the decade come to a close, we thought we’d take a look back and see some of the trends which have shaped a decade of digital transformation. And like everyone who takes a moment to look back and reflect, in our next article, we’ll take a look forward and see what surprises 2020 has in store. Data Trends Then and Now Still reeling from the financial crisis of 2008-2009, budget concerns were top of mind for many. The takeaway? Plan, and be flexible. Other trends which began in 2010 still exist today, but the vocabulary has changed. And there are further changes still which impact our technologies today and in ways we may not have realized. Train and Retrain becomes Upskill and Reskill. In 2010, organisations were advised to train, and cross train their staff. Not much has changed in ten years as it’s just as important now. Only the vocabulary has changed. Now it’s upskill and reskill those employees with the skill and inclination to pivot into more Data-centric roles within your company.Colocation Concerns Give Rise to the Cloud. Astronomical real estate costs for Data centre space and colocation prices drove businesses to find another way to store and manage their Data. As Cloud Computing spread, it allowed companies to avoid costly IT infrastructures. Not only did this save money, but it also gave businesses the flexibility they needed. In addition to the benefits of enterprise level organisations, cloud computing levels the playing field for smaller businesses to get in on the game.Virtual in the Palm of Your Hand. Smartphones and apps offer project management of our businesses and personal lives from “what’s for dinner?” to “let’s schedule our next meeting.” Our smartphones are a one-stop shop for phone calls, text messages, video conferences, scheduling, communication with remote teams, online banking, bill pay, and more.Eco-friendly is not an option, it’s an imperative. Carbon-emissions and reduction plans were already abuzz within companies. Today, Data has evolved from LEED green building certification to massive advances and predictions on the climate crisis. Standards are set.Blockchain finds friends in finance, and beyond. Though it debuted in 2008 in the finance industry, it was quickly snapped up in every industry from manufacturing to retail to shipping; any business requiring a more organised supply chain. Rise of Automation and Artificial Intelligence (AI) offers benefits beyond basic tasks. While this evokes fears for many in the workforce, there are benefits which is what’s driving things forward. While this is intended to streamline processes and avoid health risks in dangerous places like factories, there is still some cause for concern. However, some studies suggest people are happy to allow computers to take on mundane, routine, and menial tasks, freeing humans to think more creatively. Getting Social Goes Online. Though platforms like Facebook and MySpace (yeah, remember MySpace?) were already available in 2010, the plethora of platforms today was a glimmer in our smartphones’ eye. No longer relegated to youth culture, social media has become one of the most important ways for leaders and corporations to communicate with people. The Information Age has morphed into the 'Data Decade', with improvements across Data and Analytics, AI, and Machine Learning just to name a few. It’s enhancements within these spectrums which allow Data professionals to search and sort more quickly to provide the most useful Insights for their enterprises. It’s estimated that in the next couple of years, 90% of companies will list information as critical and Analytics as essential to their business strategy. If you’re interested in Marketing & Insights, Robots and Automation, Big Data and Digital or Web Analytics, we may have a role for you. Take a look at our latest opportunities or get in touch with one of our expert consultants to find out more.
03. December 2019
2018 has seen Big Data & Analytics come to the forefront on the public’s attention like never before. A series of scandals, new laws, and technological developments have opened up fresh conversations about who has access to our data, and what privacy really means in the 21st Century. In a year with a lot of news, it’s no surprise that some of the biggest stories have had a major impact on the Data & Analytics marketplace. As 2018 comes to a close, we’ve pulled together five of the biggest stories that have not only had a huge impact this year, but will continue to have repercussions in 2019 and beyond. #5. Apple Become the World’s First $1 Trillion Company At the beginning of August, Apple became the first company to be valued at $1 Trillion. A result of the launch of their premium iPhone X, they beat rivals Microsoft, Amazon and Alphabet to the milestone. Initial fears that the death of Steve Jobs in 2011 would stall the company’s growth proved to be unfounded, highlighting that product and brand still play the largest role in consumer loyalty. This achievement has raised the bar for what a tech company can achieve, and expect to see numerous others attempting to reach this level over the next decade. For an idea of what they’ll have to achieve, however, take a look at the New York Times’ visualisation of what a $1 Trillion value really means. #4. Google Walk Out Over Women’s Rights Following the #MeToo movement coming to precedence in 2017, businesses are now being properly scrutinised for their treatment of women. From the gender pay gap, to cases of sexual harassment, people are demanding transparency and accountability. Within Data & Analytics, the protests at Google were the leading example. Allegations surrounding the company’s handling of claims of sexual misconduct led to staff around the world walking out. Looking for several key changes, in particular the end of forced arbitration, employees highlighted Google’s key mission statement of ‘Don’t Be Evil’. Diversity and equality will continue to take centre stage in the years to come, with smaller businesses likely to face similar amounts of scrutiny. We’ll be releasing our report on the state of Diversity in Data & Analytics in early 2019, so come back soon to get your copy. #3. The Crypto Crash Having peaked at $19,783.06 in December ’17, 2018 saw Bitcoin, and numerous other cryptocurrencies, finally crash. Whilst this had been predicted for a while, it looks as though it may take some time for any of the currencies to gather any new momentum and regain stability. Tough new restrictions in China, one of the biggest countries for crypto, as well as ICO Ad bans on Twitter, Facebook and Instagram will limit the number of new and returning buyers. Furthermore, initial moves into the mainstream, such as Barclay’s crypto trading project, appear to have stalled. In contrast to the past few years, the future of crypto is no longer looking so bright. #2. GDPR Comes Into Play Anyone who works with any form of data couldn’t miss the introduction of GDPR, as it became enforced in April this year. A complete rewrite of the rules for data protection, we’re only beginning to see its true impact, as the first UK enforcement finally arrives. Many industries are already feeling a more specific impact, however. In particular, those working in Ad Tech have found the new regulations to be frustratingly limiting to their capabilities. Despite these issues, this is far from the end of GDPR, as both the US and India look to introduce similar regulations in the not-too-distant future. #1. THE CAMBRIDGE ANALYTICA SCANDAL The biggest Data & Analytics story of the year is, undoubtedly, the Cambridge Analytica scandal. A watershed moment in the public’s perception of how their data is used, concern grew from privacy issues to potential large-scale election rigging. The resulting chaos has seen an immense amount of pressure on Facebook and, in particular, Mark Zuckerberg, who has been called in front of numerous governments. Whilst the outcomes don’t appear to have ultimately been too dire for Facebook as a business, the consequences of the scandal will continue to be felt for a long time to come. Data breaches now regularly make headline news and the way we scrutinise how companies use our data is forever changed. If you’re looking to make a big impact in 2019 and beyond, we may have a role for you. Check out our latest roles or get in touch with one of our specialist consultants.
20. December 2018
Summer is here and over half will be planning their holiday via mobile phone. Heathrow Airport, one of the busiest airports in the world, will be filled to the brim with eager travelers. Hotels and cruise ships will be bursting at the seams. But, will the people traveling today, be the same people traveling next summer? How will the travel organisations know? They’ll need data analytics to help them make sense of all their data with advice and direction on how best to utilise nuggets of actionable insights. It wasn’t that long ago, at the busiest time of the year, Heathrow, as well as most of the airports around Europe, shut down. Why? Because no had planned for something of that magnitude – the snowstorm of the century some said. Tangible evidence of planning for every scenario, no matter how unlikely. What Would a Data Professional Do? Data professionals ask questions. They create scenarios. In the above instance, what might a Data Engineer have asked? In a data-driven world, could the airport shutdowns have been prevented? Would scenarios and algorithms have helped to scale corrections faster? These are just a few of the questions a data professional might ask. But, in order to provide solutions, first you need a builder – a data engineer. You don’t always have to build from the ground up, though. Sometimes, reconstruction and refurbishment are just what’s needed to bring a project back to its former glory. Digital Transformation in Travel As travel companies prepare for the upcoming Digital Travel Summit at the end of this month, they’ve released a report targeting certain areas on which to focus. Not the least of which is the need for stronger data analytics with less than half reporting they have a plan in place but are looking to improve it. One of the main areas of focus according to the report is lack of programming skills coupled with a hesitant approach by management. The complexity and difficulty of trying to analyse and personalise the sheer amount of consumer data available is also a factor. And without the right tools, it’s even more difficult. Knowing what data is useful and what is isn’t is a major challenge, as well as the ability to track large amounts of data in real time can be a daunting task. These challenges and more, offer the perfect opportunity for a seasoned Data Engineer to step in, and take on a leadership role. Take the Next the Step Are you a Data Engineer ready to take your career to the next level? Are you a wizard at big data technologies with a leadership bent? Then we may have a role for you. A leading e-commerce company is looking to transform their data infrastructure (Python, AWS, Airflow) by hiring a senior data engineer. Check out our other current vacancies or contact Joshua Carter, Recruitment Consultant with a Data Engineering focus at +44 20 8408 6070 or email firstname.lastname@example.org to learn more.
07. June 2018
Do you like breaking things down to see how they work? Do you want to build something that helps solves problems and can make lives better? Are you a puzzle solver curious about the world around you with a knack for mathematics? Do you prefer to work behind the scenes or front of stage? If you want to be the person behind the curtain, then this is your year. The year of the Data Engineer is here. In last week’s article, we talked about Data Engineer as the unsung hero of the data science world and briefly touched on route to the role of engineer. Though experience supersedes education, you do need the basics – a bachelor’s degree in computer science, data science, applied math, physics, statistics, software/computer engineering which can lead to a Master’s in Data Engineering and to cement your knowledge – fellowships and professional organisations are now available around the world. In today’s article, we’ll cover a few options. Lay Your Educational Foundation Computer Science, Data Science, and Engineering programs abound in university today, but no school can really teach big data skills. It’s too focused. Most schools today offer general purpose tech education with a focus on web development or backend systems. And here begins that Catch-22. Though experience supersedes education, you still need a framework from which to build. More often than not, if you type Data Engineer into Google looking for education programs, you’ll get undergrad opportunities for Data Science. However, that’s not to say a Bachelor’s in Data Science can’t lead you to a Master’s in Data Engineering. So, how do you get from point A to point B? Here are a few suggestions: Beef up your skills with specific certifications for the languages businesses need – Scala, Python, and Java Take courses in data engineering technology: Hadoop, Spark, AWS, GCP, Azure etc. Join a professional organization for Data Engineers such as The Data Warehousing Institute (TDWI) or the Institution of Engineering and Technology (IET) – here you’ll find articles, resources, and a network of mentors ready to offer advice and suggestions. Apply for a fellowship with ASI Data Science – an 8-10 week intensive project with one of their partner companies to solve real-world business problems using Data Science or Data Engineering skills. If you’re a postgrad or higher, this a perfect opportunity to build your portfolio. Boost Your Data Engineering Resume with These Tips In the world of data engineering, it’s important to highlight the details. Be specific: Companies will be more interested in interviewing you if you can clearly outline why/what you have used different technology for. Keep this punchy and concise, and outline your in-put with said technologiesOutline projects you’ve worked on Detail the technologies you’ve used David Bianco, a Data Engineer with Urthecast, offers the following advice to data engineering students. Be fluent in the languages and tools you use to get the job Understand the concepts behind what you’re doing Get involved with a community – meetup.com, hackathons, and other groups in your area are great places to get started. If you’re interested in switching your career to Big Data, check out Jessen Anderson’s new e-book, The Ultimate Guide to Switching Careers to Big Data -- Upgrading Your Skills for the Big Data Revolution. Your Turn: Route to the Role of Data Engineer Our data driven world moves at lightning speed and it can be hard to keep up. If you’re a Data Engineer, we want to hear your story. What was your route to the role? What kind of cross-training programs might businesses and schools employ for future Data Engineers? What other backgrounds are we overlooking as businesses seek to find and engage this most critical role within their data science teams? What can we, as recruiters do to engage qualified candidates ready for their next role in the world of data and analytics?
21. May 2018
Before you can build a house, you need a blueprint of its design and schematics. When you begin construction, you must first lay the foundation upon which it will be built. Tangible products taken step-by-step to create first a house and then a home. However, in the world of data science, companies seem to have skipped the blueprints and foundational aspects and gone straight for the aesthetics. But, how do you decorate a house before it’s built? A house without a foundation becomes a house of cards and the same is true of data analysis. Before the data scientists can process and analyse data, first must come the engineers. The Data Engineers who lay the digital foundation and set the parameters, who create the data lakes and platforms, so the data analysts have something to make sense of. As high as the demand is for data scientists, the demand and the need, is even greater for data engineers, yet a shortage remains. Where are the Data Engineers? Data engineering jobs outnumber data scientist jobs nearly four to one according to a quick search on job boards such as Glassdoor and Indeed. Yet, the complex technical nature of data engineering to support data scientists takes more than a degreed education. Unlike data analysts, data scientists, and other data professionals who can land a mid-level job directly out of university, data engineers cannot. Ultimately, it takes between five to ten years for mid-level data engineers to gain enough experience for practical application. As such, systems do not yet exist in schools and universities to supplement data engineers undergraduate or postgraduate degrees in preparation for real life work experience in the field. However, once the experience is gained, it can take a company who has hired a data engineer up to two years to catch up with its competition. With the pace of change in the tech world, this can be detrimental to both the business and the data science teams. Therein lies the Catch-22, data engineers must have experience before they can be hired, but there is no way to learn outside of hands-on, real life application. Why You Need to Add a Data Engineer to Your Data Science Team A data science team is not complete without a data engineer. Why? Because just like building a house, grand schemes and ideas to solve complex business problems, must first have a foundation. Data engineers are that foundational support of experts who design, build, and maintain data-based systems and organizational operations. Not only do data engineers lay the foundation upon which data can be built, analysed, and ultimately translated to business professionals, it must also be timely. Timely data leads to more data and better predictions. Data engineers are not completely siloed from data science teams, they are also responsible for deploying the code and models that are written by data scientists. For more on the reasons data engineering is more important than data science for companies today, check out this article from Captech Consulting. Data Science Team Seeks Data Engineer Companies know data drives business and they know the importance of data professionals. However, they may mistakenly assume either that their data teams can pick up engineering experience as they work their way through a project or they simply assume the titles are interchangeable. In the world of data engineering, there is no entry level job. Experience trumps education in this field. Like the once siloed data science team now integrated across the business with sales, marketing, and advertising departments, so must the role of data engineer be integrated. This is not a marriage of convenience, but of necessity in order to stay ahead of the competition. Together, your fully integrated data teams – data engineering and data science now on equal footing - will be able to help your business reach better predictions faster, making you a voice of authority in your discipline. Your Turn: Route to the Role of Data Engineer The route to the role of Data Engineer may seem daunting with the catch-22 that experience supersedes education. So, in the spirit of collaboration, we thought we’d ask for your thoughts and opinions on a few items of interest such as how we can educate aspiring data engineers and get them into companies faster. What kind of cross-training programs might businesses and schools employ to fill the shortage? What other backgrounds are we overlooking as businesses seek to find and engage this most critical role within their data science teams? According to the website Datanami, 2018 will be the year of the data engineer. If this is you, then we may have a role for you.
21. May 2018