Python Data Engineer

London
£600 - £700 per day

Data Engineer
£600- £700 p/d
3-month contract
Central London

As a Data Engineer, you will be building robust pipelines in Python using Spark and Airflow for a leadig fin tech.

THE COMPANY:

As a Data Engineer, you will be working for a trendy financial client who have a number of credit card products. They have partnerships with leading retailers and have a number of prime and sub-prime products. You will be working in an agile delivery team alongside other Data Engineers helping to build robust data pipelines. You will be based in modern central London office where you will be working in a close-knit team. As a Data Engineer, you will be working in an agile and collaborative environment.

THE ROLE:

As a Data Engineer, you will you will be heavily involved in not only ETL development using Python. Therefore as a Data Engineer you must be extremely skilled in Python and if you have exposure to Python libraries such a NumPy this is valuable. Alongside building robust Python code as a Data Engineer you will also be using Big Data technologies such as Spark so most have good commercial experience with Spark. You will be working in an AWS environment and therefore must be very familiar with this cloud platform as a Data Engineer. As a Data Engineer you should familiar with key AWS services such as S3, but commercial use of Airflow as a scheduling tool is desirable as you will be using this tool in your project. Not only Airflow but experience with tools such as Docker and CI/CD is desirable in building pipelines. As a Data Engineer you will also be liaising with business stakeholders and therefore must be a good communicator as there will be some client facing aspects of the role.

YOUR SKILLS AND EXPERIENCE:

The successful Data Engineer will have the following skills and experience:

  • Extensive experience with Python
  • Good use of Spark
  • Very good understanding of AWS including S3
  • Commercial use of Airflow
  • Understanding of tools such as Docker in CI/CD
  • Experience in a client facing role

THE BENEFITS:

  • A very competitive day rate

HOW TO APPLY:

Please register your interest by sending your CV to Anna Greenhill via the Apply link on this page.

Send similar jobs by email
69357/AG-2
London
£600 - £700 per day
  1. Contract
  2. Big Data

Similar Jobs

Salary

£45000 - £55000 per annum + + Bonus + Competitive Benefits

Location

London

Description

Are you looking to join one of the largest retailers, with one of the most visited websites in the UK as a Data Engineer?

Salary

€60867 - €71934 per annum + Bonus, Pension

Location

Amsterdam, North Holland

Description

A global retailer is looking for a skilled Data Engineer to join their Amsterdam office!

Salary

€50000 - €70000 per annum + Bonus, Pension, Travel Allowance

Location

Amsterdam, North Holland

Description

Do you have an interest in the media sector? Ever wanted to get behind the scenes and find out what makes it all tick? Click below to find out more!

Salary

€75000 - €85000 per annum + Bonus, Pension, Car/ Travel Allowance etc.

Location

Den Haag, South Holland

Description

Data Engineer looking to take the next step in your career? Get in touch if you want to become a leader and more!

Harnham blog & news

With over 10 years experience working solely in the Data & Analytics sector our consultants are able to offer detailed insights into the industry.

Visit our Blogs & News portal or check out our recent posts below.

Integrate Your Data And Business Strategies For Success

Why You Need To Integrate Your Data and Business Strategies

United we stand, together we fall. Not too put too fine a point to it, but how your business and data strategies align are integral to your business. Today’s world is about change, being able to pivot toward new strategies, and being open to trying new things. Consider this: the “mom-and-pop” shop is back and it is flourishing. Younger generations of farmers are returning to their family farms when they graduate and they’re bringing new knowledge with them. And the makerspace, freelance, and gig economies are thriving. These businesses are learning how to work with technology and align their Data Strategy with their Business strategy. Some legacy enterprises are taking notice. Others are missing the mark. Consumers may have changed how they want to shop and learn about services and products, but the services they want and expect haven’t changed that much which is why it’s more important than ever to “know your customer.”  3 Key Elements of Integrated Strategies While there are a number of things to take into consideration as you align your strategies, these three key elements can help get you started. 1. Understand the key elements of Business Strategy. 2. Apply innovation strategy to business objectives. 3. Determine key elements of your Data Strategy for use in a real-world scenario. Understand the key elements of business strategy  A business strategy encapsulates two main ideas; cost advantage versus competition. The cost advantage includes costs and other resources, identification and awareness of strengths, weaknesses, and competition. Competitive advantage happens when you’ve done your market research and can show what makes you different from any other provider with similar goods and services. This is the time you might perform a SWOT (strengths, weaknesses, opportunity, and threat) analysis of your business. It’s helpful to include your mission and vision statements, objectives, core values, risk tolerance, and understanding trends in your business. Apply Innovation Strategy to Business Objectives Ideas and innovation flow when you and your business understand your customers and are able to easily shift into new things. Think R&D into Bioinformatics, automated tasks into AI, or a platform such as streaming services to help sell services such as insurance. Laying the groundwork to apply innovation strategies to your business objectives follow these ideas: Identify your business objectives by asking questions.Assess the budget and personnel resources and develop a budget strategy.Test the market to determine what issues will or need to be solved and understand how this innovation will benefit your overall strategy. If you’re working on a Data initiative to integrate into your Business strategy, one of the key elements is to determine how those changes may affect your business. Determine Key Elements of Data Strategy for Use in Real-World Scenarios As you work on developing your Data Strategy, it’s important to consider all the elements required to ensure success. So, what do you need to take into consideration when working on this type of strategy? Here are some things to consider as you develop your framework. Determine your business needs and their current state.Determine what works and what can be improved upon if there is a technology improvement or process.Evaluate your Data from sales, profit, and evaluate your progress.}Develop an action plan. Many businesses don’t incorporate just one type of Data into their strategy. They consider the potential impact of technologies such as Machine Learning, Predictive and Data Analytics, and other Big Data Strategies to drive improvements when it comes to decision making. They understand these Data-driven insights can help them improve or solve their most critical problems. There is a caveat, however, and it is how you collect the information for real-world scenarios. Certain requirements are in place for a reason and they ensure only relevant Data is collected. This is done by formulating “predictive models” and necessary information to operate and determine whether your case will be something to be done over time or if it’s something brand new to consider when looking at real-time access. One Final Thought… Data-centric organisations have a distinct advantage over their competition. The information gained from collecting and analysing to understanding their customers can offer great insight as to what’s working and what isn’t. Integrating your Business Strategy with a Data Strategy can offer you a more well-rounded understanding of the customers you serve and can ultimately, help you to serve them better; now and in the future. Disruptive business models from this way of thinking can also foster growth and lead to innovative changes in your marketplace. If you want to be at the forefront of change we may have a role or candidate for you. Take a look at our latest opportunities or get in touch with one of our expert consultants to find out more.

How Big Data Is Impacting Logistics

How Big Data is Impacting Logistics

As Big Data can reveal patterns, trends and associations relating to human behaviour and interactions, it’s no surprise that Data & Analytics are changing the way that the supply chain sector operates today.  From informing and predicting buying trends to streamlining order processing and logistics, technological innovations are impacting the industry, boosting efficiency and improving supply chain management.  Analysing behavioural patterns Using pattern recognition systems, Artificial Intelligence is able to analyse Big Data. During this process, Artificial Intelligence defines and identifies external influences which may affect the process of operations (such as customer purchasing choices) using Machine Learning algorithms. From the Data collected, Artificial Intelligence is able to determine information or characteristics which can inform us of repetitive behaviour or predict statistically probable actions.  Consequently, organisation and planning can be undertaken with ease to improve the efficiency of the supply chain. For example, ordering a calculated amount of stock in preparation for a busy season can be made using much more accurate predictions - contributing to less over-stocking and potentially more profit. As a result, analysing behavioural patterns facilitates better management and administration, with a knock-on effect for improving processes.  Streamlining operations  Using image recognition technology, Artificial Intelligence enables quicker processes that are ideally suited for warehouses and stock control applications. Additionally, transcribing voice to text applications mean stock can be identified and processed quickly to reach its destination, reducing the human resource time required and minimising human error.  Artificial intelligence has also changed the way we use our inventory systems. Using natural language interaction, enterprises have the capability to generate reports on sales, meaning businesses can quickly identify stock concerns and replenish accordingly. Intelligence can even communicate these reports, so Data reliably reaches the next person in the supply chain, expanding capabilities for efficient operations to a level that humans physically cannot attain. It’s no surprise that when it comes to warehousing and packaging operations Artificial Intelligence can revolutionise the efficiency of current systems. With image recognition now capable of detecting which brands and logos are visible on cardboard boxes of all sizes, monitoring shelf space is now possible on a real-time basis. In turn, Artificial Intelligence is able to offer short term insights that would have previously been restricted to broad annual time frames for consumers and management alike.  Forecasting  Many companies manually undertake forecasting predictions using excel spreadsheets that are then subject to communication and data from other departments. Using this method, there’s ample room for human error as forecasting cannot be uniform across all regions in national or global companies. This can create impactful mistakes which have the potential to make predictions increasingly inaccurate.  Using intelligent stock management systems, Machine Learning algorithms can predict when stock replenishment will be required in warehouse environments. When combined with trend prediction technology, warehouses will effectively be capable enough to almost run themselves  negating the risk of human error and wasted time. Automating the forecasting process decreases cycle time, while providing early warning signals for unexpected issues, leaving businesses better prepared for most eventualities that may not have been spotted by the human eye.  Big Data is continuing to transform the world of logistics, and utilising it in the best way possible is essential to meeting customer demands and exercising agile supply chain management.  If you’re interested in utilising Artificial Intelligence and Machine Learning to help improve processes, Harnham may be able to help. Take a look at our latest opportunities or get in touch with one of our expert consultants to find out more.  Author Bio: Alex Jones is a content creator for Kendon Packaging. Now one of Britain's leading packaging companies, Kendon Packaging has been supporting businesses nationwide since the 1930s.

Recently Viewed jobs