Harnham

Big Data Trends to Keep Your Eye on



big data processing

Big data trends to keep your eye on


Big data just keeps getting bigger, and 2016 has been no exception. Organizations are increasingly realizing that data can reveal major insights and lead to better-informed decisions and business advantages. As we edge closer to the end of the year, we thought it was time to take stock of the emerging trends that are only set to get bigger.

More scrutiny on security


The last few years have showcased how data leaks and hacks can have a disastrous effect on organizations and consumers. The scandalous Ashley Madison hack highlighted the need for ultra-secure systems when dealing with delicate consumer information, while the TalkTalk data hack showed the risk of hacks on high-profile businesses and the need for enhanced password security measures from consumers.

As a consequence of a string of high-profile data breaches, we’ve seen many organizations place increased scrutiny on how data is protected and used. As the new EU General Data Protection Regulations come into play for many businesses, we’ll likely see more spending on data security measures and preparation for further enhancements to tighten up sensitive data. With the EU Data Protection Regulation coming into effect across the EU, it’s likely the protection of and access to big data will become more critical than ever.

Big data in healthcare


The healthcare field has seen enormous benefits from the development of big data in recent years. 2016 has seen more medical information become digitized, loaded up to the cloud and analyzed in more technological ways than ever. The emergence of big data in the medical world could see more information and facts readily available to complement the opinions of doctors, bringing about a technologically-driven movement in how patient data is processed and reviewed. Big data also allows medical and test data to be shared more freely and quickly all over the world, allowing for more rapid and precise establishment of correlations in aggregated data.

Smart data for the masses


2016 was predicted by many to be the year that big data became smarter but not necessarily bigger. Instead of looking to gather as much information as they possibly can, analysts and scientists have instead been focusing on data that provides more insight than before, including features such as time, location and links between data sets.

The use of data has also extended beyond major organizations with dedicated big data programs through to smaller start-ups and even consumers who look to collect data to do things better, faster and smarter. If you use devices to track your activity levels, sleep patterns or even where and on what you spend your money, you’re part of the new data revolution.

What’s next?


If you’re interested in contributing to the future of big data, we might have the role for you. We specialize in Data and Analytics recruitment and always have a wide range of vacancies at both junior and senior level. Take a look at our current vacancies or contact us to find out more.

SHARE THIS PAGE

 

Harnham blog & news

With over 10 years experience working solely in the Data & Analytics sector our consultants are able to offer detailed insights into the industry.

Visit our News & Blogs portal or check out our recent posts below.

How to lead a Data team

How To Lead A Data Team

Dream teams from sports to business are an ideal everyone aspires to live up to. But what is it every basketball or football dynasty has which makes them a dream team? What is it that brings individuals together to overcome odds, set examples, find solutions, and create the next best thing? Good management.  The need for good management is no different in the Data Science world. Yet according to our latest Salary Guide, poor management is one of the top five reasons Data professionals leave companies. So, let’s take a look at what poor management is, what causes it, and how businesses can better retain Data talent. What’s Your Data Science Strategy? Most businesses know they need a Data team. They may also assume that a Data Scientist who performed well can lead a Data team. But that isn’t necessarily the case. Managers have to know things like P&L statements, how to build a business case, make market assessments, and how to deal with people. And that’s just for a start.  The leader of a Data team has a number of other factors to consider as well such as Data Governance, MDM, compliance, legal issues around the use of algorithms, and the list goes on. At the same time, they also need to be managing their team with trust, authenticity, and candor. The list of responsibilities can be daunting and if someone is given too much too soon and without support, it can be a recipe for disaster. Other businesses might believe that a top performing Data Scientist would make a good manager. Yet these are two different fields. Or you might look at it this way. If you are willing to upskill a top performing Data professional and train them in managerial skills, giving them the education and support they need, that is one solution. Another solution is to create a Data Science strategy which brings in people with business backgrounds. Data Science is a diverse field and people come from a number of backgrounds not just Computer Science or Biostatistics, for example.  Now that you’ve seen what might cause a manager to fail, let’s take a look at a few tips to help you succeed. Seven Tips for Managing a Data Team Managing a team is about being able to hire, retain, and develop great talent. But if the manager has no management training, well, that’s how things tend to fall apart. Here a few tips to consider to help ensure you and your team work together to become the dream team of your organization: Build trust by caring about your team. Help define their role within the organization. Ensure projects are exciting and that they’re not being asked to do project with vague guidelines or unrealistic timeframes.Be open and candid. Remember, Data Scientists are trained in how to gather, collect, and analyze information. If anyone can see right through a façade, it will be these Data professionals. Have those “tough” conversations throughout every stage of the hiring, onboarding, and day-to-day, so that no one is caught unaware.Offer consistent feedback. And ask for it for yourself as well from your team.Ensure your team understands the business goals behind their projects. Let them in on the bigger picture. Think long-term recruitment for a permanent role, not short-term. If you have an urgent project, consider contracting it out. Prioritize diversity to include academic discipline and professional experience. Does the way this person view the world expand the knowledge of your team’s knowledge? Dream teams don’t always have to agree. Sometimes, the best solutions are found when there are other opinions. Finding the perfect, “Full Stack” Data Scientist or Data Engineer or Analyst is not impossible, and retaining them can be even easier. If you’ve done your job well, your team will trust you, have a balanced skillset, and understand how their work supports the organization and its goals. For more information on how to be a great manager, check out this article from HBR.  Ready for the next step?  Check out our current vacancies or contact one of our recruitment consultants to learn more. For our West Coast Team, call (415) 614 - 4999 or send an email to sanfraninfo@harnham.com.  For our Mid-West and East Coast Teams, call (212) 796 - 6070 or send an email to newyorkinfo@harnham.com.

Peterson Hypolite

Peterson Hypolite

11. July 2019

Battle Royale: Computational Biologists Vs Machine Learning Engineers

Battle Royale: Computational Biologists vs Machine Learning Engineers

From the first genome sequencing in the second revolution to Life Science Analytics as a growing field in the fourth industrial revolution, change has been both welcomed and fraught with fear. Everyone worries about robots, Artificial Intelligence, and in some cases even professionals who have stayed current by keeping up-to-date with trends. And it’s beginning to affect not only “office politics” within the tech space, but even interviewer and interviewee relationships. We’ve seen a growing trend of apprehension between Computational Biologists and Machine Learning Engineers. What could be the cause? Aren’t they each working toward a common goal? It seems the answer isn’t quite so cut and dry as we’d like it to be. Here are some thoughts on what could be driving this animosity. But first, a bit of background. So, What’s the Difference? Computational Biology and Machine Learning are two sides of the same coin; one sets the framework and the other applies what’s been learned. Both use statistical and computational methods to construct models from existing databases to create new Data. However, it is within the framework of biomedical problems as computational problems, that there seems to be a bit of a breakdown. It’s one thing to have all the information and all the Data, but its quite another to know how the Data might interact or affect the health and medications of people seeking help. This is the job of those in Life Science Analytics. Determine through Data what needs to be done, quickly, and efficiently, but at the same time, ensure the human element is still active.  A few examples of Computational Biology include concentrations, sequences, images and are used in such areas as Algorithmics, Robotics, and Machine Learning. The job of Machine Learning can help to classify spam emails, recognize human speech, and more. Here’s a good place to start if you’d like to take a deeper dive into the differences between the two or read this article about mindsets and misconceptions. Office Politics in the Tech Space Circling back to the concern between Computational Biologists and Data Scientists with a focus on Machine Learning. The latest around the water cooler within the tech space is that those with a biological background who understand Machine Learning are looked upon as dangerous to the status quo.  But, as many of our candidates know, it’s important to stay on the cutting edge and if that means, upskilling in Machine Learning so you have both the human element as well as the mathematical, robotic components, then that is more marketable than just having one or the other. The learning curve in biology training within the Life Sciences Analytics space means Computational Biologist with a Machine Learning skillset is best able to apply Data Science and computer science tools to more organic and biological datasets. Someone with just a computer science background may not have the depth of knowledge to understand how these models, systems, and data affect and impact medicine. Computational Biologists who are trained simultaneously in computer science and biology, and are a little heavier on the biology side, see Machine Learning Engineers as a threat because utilizing Machine Learning and other cutting-edge tools could mean their job is on the line. They worry their job will fall by the wayside. That when somebody proves Machine Learning is faster and more efficient the impetus might be why hire a Computational Biologist when a Machine Learning engineer will do? It’s like when a lot of people joke about how robots are going to take over the world and everybody will be out of a job. I think the worry with some folks on the Computational Biology side is that maybe they just aren’t up to date with their training or haven’t kept up with cutting edge of technology. With a Recruiter’s Eye While what I’ve seen agrees that, yes, Machine Learning is incredibly helpful and fast and you can get through so much more data. But its still that understanding of biology and chemistry that you will need because you need to be able to understand, for example, how these proteins are going to be reacting with one another or you need to understand how DNA and R&A work, how best to analyze, and what analyzing those things means. On the other hand, just because you know, “oh, this reaction comes out of it”, if you don’t know why that is or how that could impact a drug or a person, then you don’t really have anything to go on. There’s a caveat there. Though there may be concerns among Computational Biologists and Machine Learning Engineers, at both the upper and entry levels, it’s still the technical lead who will say, “we really do need somebody with a biological background because if we get all this Data and don’t really know what to do with it, then we’ll need to hire a Project Manager to converse between the two and that’s an inefficient use of time and resources”. What I hear most often is a company wants a Computational Biologist but they also want someone who knows Machine Learning. But they don’t want to compromise on either because they don’t understand there are limitations to things. We all want the unicorn employee, but we can’t make them fit into a box with too specific parameters. It’s a Fact of Life Any job, whether it’s in the tech industry, the food industry, Ad Optimization, or even recruitment, uses Machine Learning in one way or another. Yet compared to spaces which work on sequencing the human genome, it's amazing to see how far things have come. It used to take days to process DNA. Now you can spit in a tube and send it off to 23andMe to learn a little about your health. That’s what Machine Learning enables people to do. But it doesn’t mean Computational Biologists are going to fall by the wayside. It means there will be times you’ll have to liaise more between the two groups. It means you’ll be more marketable by adding Machine Learning to the work you’re already doing or taking some classes in Computational Science, for example, to keep your skills up to date. It’s a Transparency Issue Ultimately, it seems the heart of this apprehension comes down to a transparency issue. For example, let’s say companies begin to bring in AI people and suddenly the staff already in place begins to get worried about the security of their jobs. Even in an industry tense with skills gaps, the fear still abounds. In coming back to speak with the Hiring Manager, it became clear the animosity is even more prevalent than first imagined. So, it’s important to get input from within the company and develop a unified story, a unified message across departments, and especially within the Life Science Analytics and Data Science teams as well. In other words, “keep people in the loop.” If it’s happening to this company, it seems other companies may be facing this same issue. However, it’s not going away and is creating a kind of competition between the old guard and the up-and-coming startups. For example, any new company is going to want to integrate AI and will be asking the question how best to integrate it into their structure. They might also ask how best to optimize the ads coming through AI. This is just another way of how companies are catching up, but also how people are catching up to the companies.  Technology is coming whether you like it or not. So, if you want to stay marketable and work on really interesting projects, there’s always going to be the challenge of staying up-to-date and different companies attack this in different ways.  Stay open minded, keep an eye and an ear out for ways to stay on top of your game. Even just taking a few minutes to watch a YouTube video, listen to a TedTalk or a podcast, so you can talk about it and be informed. These are some really simple ways to stay on the cutting edge and help you figure out where you can grow and improve for better opportunities. Ready for the next step? Check out our current vacancies or contact one of our recruitment consultants to learn more. For our West Coast Team, call (415) 614 - 4999 or send an email to sanfraninfo@harnham.com.  For our Mid-West and East Coast Teams, call (212) 796 - 6070 or send an email to newyorkinfo@harnham.com.

Alyssa Liew

Alyssa Liew

04. July 2019

Data Engineers: The Workers Behind the Curtain

We’re halfway through the year and our salary guide is out. If you wonder where you are, where you’re going, and if you’re a business, how you’re going to get there. Well, then, you’ve come to the right place. Articles have touted Data Scientists as the “rock stars” of the 21st century, but even rock stars need their managers and roadies. Who else will build the stage and plan the tour? And in the world of data, it all begins with the Data Engineer, laying the groundwork, the foundation, and the framework. These are the stars behind the scenes who make it possible for Data and Data Scientists to be front and center. Send a Data Engineer Over As prevalent as Data has become in our lives and as its importance grows, there remains the challenge of Data Management. If you don’t know why something is built or how to navigate the structure, the Data you do receive may not make much sense. Your guide in this journey is the Data Engineer, one of the most important pieces of your Data Management puzzle.  These highly skilled and sought-after professionals should not be confused with a Software Engineer, though some elements may be transferable between the two. The building blocks to put massive amounts of Data into a scalable system both reliable and secure takes a unique set of skills.  Humans at the Helm as Skills Shift  As much as we depend on Data today to help determine actionable insights for our business and as much as we hear about the rise of machines in the form of Artificial Intelligence, Machine Learning, and Deep Learning, it is ultimately humans who are at the helm.  It is the people behind the curtain of Data who will build it, run it, and make it work. It is also people who are typically the biggest costs in a project. Finding the balance and ideal candidate, the right person with the right skills for the job, is critical to success. And if you’re starting from the ground up, Data Engineers who can work with the core tools of databases and Spark, for example, will see their opportunities grow.  In our Salary Guide for 2019, we learned one of the skills most sought after by companies today is knowing AWS/Azure and moving Data Lakes into the Cloud. Small businesses and startups are moving to the Cloud to help them scale their Data, but someone still needs to lay the groundwork, whether it’s for the small business or the public cloud providers. Data Engineers are in high demand and it doesn’t look as though things will be slowing down anytime soon. The field is slick with potential. The Time Has Come for Transparency Data is binary gold and, with enough of it, you can read or estimate the mind of your customer or you can wreak havoc on someone’s life. Just a year ago, the European Union put into place rules and regulations as well as financial consequences for poor Data Governance under the General Data and Protections Regulation Act (GDPR). Though the U.S. doesn’t yet have a similar law, there are still plenty of mandates to be aware of by states, unions, and countries.  One Final Thought As roles and technology evolves, it’s important for businesses, employees, and stakeholders to evolve as well. Whether that means making sure to implement practices for Data transparency or upskilling and reskilling your workforce to keep up or simply knowing the trends of forward-thinking companies to scale your own business. Data fuels digital innovation and organizations who are prepared to find solutions will benefit. Want to know what else is trending in big data? Here are a few trends in Big Data forward-looking organizations should look out for in this year and toward the next. Are you a business who knows you’re ready to scale up and hire a Data professional? We have a strong candidate pool and may have just the person you need to fill your role. Are you a candidate looking for a role in Big Data & Analytics? We specialize in junior and senior roles. Check out our current vacancies or contact one of our recruitment consultants to learn more.  For our West Coast Team, call (415) 614 - 4999 or send an email to sanfraninfo@harnham.com.   For our Mid-West and East Coast Teams, call (212) 796 - 6070 or send an email to newyorkinfo@harnham.com.

Kavya Kannan

Kavya Kannan

20. June 2019

A Data Engineer is a Unique Blend of Data Professional

From startup and small business to large enterprises, each type of business requires a unique blend of Data professional. Though in today’s world, much of the Data being gathered, catalogued, and analyzed happens both in the Cloud and on a hard drive, each type of business has a different need, budget, goals, and objectives. But there is one thing each and every business will have in common. At the heart of the Data team will be a Data Engineer. The Three Main Roles of a Data Engineer This is an analytics role in high demand. It is a growing and lucrative field with steps and stages for nearly every level of business and education experience. For example, a Data Scientist interested in stepping into a Data Engineer role might begin as a Generalist. In all, there are three main roles for each level and type of business – Generalist, Pipeline-Centric, and Data-Centric. Let’s take a quick look at each of the roles with an eye toward the type of person who might be the best fit: Generalist – Most often found on a small team, this type of Data Engineer is most likely the only Data-focused person in the company. They may have to do everything from build the system to analyze it, and while it carries its own unique set of skills, it doesn’t require heavy architecture knowledge as smaller companies may not yet be focusing on scale. In a nutshell, this might be a good entry point for a Data Scientist interested in upskilling and reskilling themselves to transition into a Data Engineering role.Pipeline-centric – This focus requires more in-depth knowledge working with more complex Data science needs. This type of role is found more often in mid-sized companies as they grow and incorporate a team of Data professionals to help analyze and offer actionable insight for the business. In a nutshell, this role creates a useful format for analysts to gather, collect, and analyze each bit of Data at each stage of development.Database-centric – This role is found most often in larger companies and deals not only with Data warehouses, but is focused on setting up analytics databases. Though there are some elements of the pipeline, this is more fine-tuned. In a nutshell, this role deals with many analysts across a wide distribution of databases. A Fine Balance Between Technical Skills, Soft Skills, and Business Acumen While it’s important for anyone filing this role to have deep knowledge of database design as well as a variety of programming languages, its equally important to understand company objectives. In other words, once the groundwork is laid and the datasets established, it’ll be important to explain what it is the business executives need to know to make the best decisions for their business.  Knowing how and what to communicate to executives, stakeholders, and your Data team also means understanding how to best retrieve and optimize the information for reporting. Depending on your organization’s size, you may need both a Data Analyst or Scientist and a Data Engineer. Though this is less likely in medium and larger enterprises. On the flip side, in order to understand the business’ needs, you’ll also need to be good at creating reliable pipelines, architecting systems and Data stores, and collaborating with your Data Science team to build the right solutions. Each of these skills are meant to help you understand concepts to build real-world systems no matter the size of your business. One Final Thought… Do you like to build things? Tweak systems? Take things apart and see how they work, then put them back together better and more efficient than before? Then Data Engineering might be for you. Are you a business who knows you’re ready to scale up and hire a Data professional? We have a strong candidate pool and may have just the person you need to fill your role. Are you a candidate looking for a role in big Data and analytics? We specialize in junior and senior roles. Check out our current vacancies or contact one of our recruitment consultants to learn more.  For our West Coast Team, call (415) 614 - 4999 or send an email to sanfraninfo@harnham.com.   For our Mid-West and East Coast Teams, call (212) 796 - 6070 or send an email to newyorkinfo@harnham.com.

Thomas Daughtrey

Thomas Daughtrey

11. June 2019

Recently Viewed jobs