Job Application

Lead Data Engineer (SC Cleared)

• 10 months +
• Hybrid-working (London, Westminster 2days pw and remote-working

Our client, a central government department are looking for a Data Engineer to join their data services team. - Data services are very much a priority as set out in the DDaT strategy and wider government agenda as well as one of the key pillars of work in delivering vision and mission in being more data driven.

Data Engineering and Architecture makes it easier to solve problems in executing complex datasets with reliability. Allowing the client to optimise data towards usability by transforming and modelling it to be usable in a way that can help Analysts visualise data for consumption.

Data can provide the client with the following:

• Help identify patterns and trends in data
• Stronger evidence base for policies and operations
• Better prediction of future trends
• Improved quality of decision making
• Faster operational response window
• Wider data availability and access
• Provide greater accountability
• Reduction in duplication and error

Key Skills/Experience:

The Lead Data Engineer will be required to work towards the following:

• Design AWS data ingestion and transformation pipelines based on needs driven by teams/SLT/Product Owners and user stories.
• Onboard new data sources to the data platform for highest priority reporting requirements
• Maintain and test pipelines for reliability and performance
• Ensure monitoring and scanning for pipelines conforms to vulnerability management processes
• Monitor and conduct production deployments/releases
• -Working collaboratively with Data Analysts to model data to deliver information requirements
• -Build the technical roadmaps including defining technical and non-functional stories
• -Assist with identifying and diagnosing incidents and problems
• -Review large design changes to ensure correct technical direction is being taken that aligns with long term vision
• -Providing technical input/pair-programming difficult problems where necessary
• -Keep applications up to date in line with Infrastructure and Platform Support teams requirements, mitigating vulnerabilities and reducing risks
• -Identify opportunities to integrate an Analytic Software (eg Matomo) with applications so the business can see how users use the products, where we can get the most value and where efficiencies can be made
• -Identify opportunities and make recommendations to improve functionality, applications, saving running costs, reducing tech debt etc.
• -Ensure governance around data handling, ingestion, storage and retention
• -Ensure best practices around data management and analysis
• -Identify cost optimisation opportunities across the DO Data Analytics platform
• -Identify and make recommendations to improve processes
• -Define and support data quality standards
• -Ensure process for receiving analysis requests is understood and communicated
• -Performance optimisation of data platform, its components and applications
• -Ensure requirements for new services are defined and planned
• -Create designs for onboarding new data sources on to the data platform

Travel Info

• Working at home with 1-2 days per week either working at the permanent location (2MS) or at an additional location with expensed travel.