Senior Data Engineer
Barclays
Date: 1 day ago
City: London
Contract type: Full time

We are seeking a Senior Data Engineer to design and implement a data platform for managing large-scale binary datasets with high-performance access and historical queries. You will build a robust lakehouse using Delta Lake and HDFS/S3, optimizing for scalability and frequent updates. The ideal candidate has strong expertise in Apache Spark, Delta Lake, and big data systems, with a focus on performance and reliability in an on-premises as well as AWS environment. The candidate will also work on building an access layer that works across unstructured data and structured data across multiple stores.
Key Skills Required For This Role Include
This role is based out of our London office.
Purpose of the role
To build and maintain the systems that collect, store, process, and analyse data, such as data pipelines, data warehouses and data lakes to ensure that all data is accurate, accessible, and secure.
Accountabilities
Key Skills Required For This Role Include
- Hands-on development experience with either Java or Python
- Experience in Big Data Technologies (Hadoop, Spark, Kafka, Streaming)
- Deep understanding of AWS services to architect and deploy cloud-native big data solutions
- Financial knowledge
- Real-time data processing
- Databricks, Snowflake, Dashboarding
- Soft-skills such as strong communication, stakeholder management
This role is based out of our London office.
Purpose of the role
To build and maintain the systems that collect, store, process, and analyse data, such as data pipelines, data warehouses and data lakes to ensure that all data is accurate, accessible, and secure.
Accountabilities
- Build and maintenance of data architectures pipelines that enable the transfer and processing of durable, complete and consistent data.
- Design and implementation of data warehoused and data lakes that manage the appropriate data volumes and velocity and adhere to the required security measures.
- Development of processing and analysis algorithms fit for the intended data complexity and volumes.
- Collaboration with data scientist to build and deploy machine learning models.
- To contribute or set strategy, drive requirements and make recommendations for change. Plan resources, budgets, and policies; manage and maintain policies/ processes; deliver continuous improvements and escalate breaches of policies/procedures..
- If managing a team, they define jobs and responsibilities, planning for the department’s future needs and operations, counselling employees on performance and contributing to employee pay decisions/changes. They may also lead a number of specialists to influence the operations of a department, in alignment with strategic as well as tactical priorities, while balancing short and long term goals and ensuring that budgets and schedules meet corporate requirements..
- If the position has leadership responsibilities, People Leaders are expected to demonstrate a clear set of leadership behaviours to create an environment for colleagues to thrive and deliver to a consistently excellent standard. The four LEAD behaviours are: L – Listen and be authentic, E – Energise and inspire, A – Align across the enterprise, D – Develop others..
- OR for an individual contributor, they will be a subject matter expert within own discipline and will guide technical direction. They will lead collaborative, multi-year assignments and guide team members through structured assignments, identify the need for the inclusion of other areas of specialisation to complete assignments. They will train, guide and coach less experienced specialists and provide information affecting long term profits, organisational risks and strategic decisions..
- Advise key stakeholders, including functional leadership teams and senior management on functional and cross functional areas of impact and alignment.
- Manage and mitigate risks through assessment, in support of the control and governance agenda.
- Demonstrate leadership and accountability for managing risk and strengthening controls in relation to the work your team does.
- Demonstrate comprehensive understanding of the organisation functions to contribute to achieving the goals of the business.
- Collaborate with other areas of work, for business aligned support areas to keep up to speed with business activity and the business strategies.
- Create solutions based on sophisticated analytical thought comparing and selecting complex alternatives. In-depth analysis with interpretative thinking will be required to define problems and develop innovative solutions.
- Adopt and include the outcomes of extensive research in problem solving processes.
- Seek out, build and maintain trusting relationships and partnerships with internal and external stakeholders in order to accomplish key business objectives, using influencing and negotiating skills to achieve outcomes.
How to apply
To apply for this job you need to authorize on our website. If you don't have an account yet, please register.
Post a resumeSimilar jobs
Estimator
Dragados UK & Ireland,
London
1 day ago
Job AdvertJob SummaryAs an Estimator at Dragados, you will prepare detailed and accurate cost estimates for direct work packages and indirect costs (preliminaries) for major infrastructure projects. You will also contribute significantly to the commercial submission process by developing pricing models, coordinating with internal teams, and factoring in construction methodology, project constraints, and risk elements into the overall estimate.Key ResponsibilitiesSupport...

Procurement Business Support Lead
BDO UK,
London
1 week ago
Ideas | People | TrustWe’re BDO. An accountancy and business advisory firm, providing the advice and solutions entrepreneurial organisations need to navigate today’s changing world.We work with the companies that are Britain’s economic engine – ambitious, entrepreneurially-spirited and high‑growth businesses that fuel the economy - and directly advise the owners and management teams that lead them.We’ll broaden your horizons At...

Senior Vice President, Regulatory Consulting
Kroll,
London
1 week ago
Kroll’s Regulatory Consulting team helps firms deliver on a wide range of professional service engagements. These include matters relating to financial crime, compliance risk management, regulatory readiness, compliance monitoring and regulatory reporting.At Kroll, your work will help deliver clarity to our clients’ most complex governance, risk, and transparency challenges.Job SummaryWe are searching for an experienced and proficient individual with sound...
