Please note that this role requires security clearance at SC level. You must be SC cleared to be considered for the role.
Tasks and Responsibilities:
Design:
- Define Data Platform technical architecture by analyzing the requirements.
- Define technical design for ingestion, visualization and integration.
- Analyze the data mapping and data models and come up with ingestion level data mapping.
- Coordinate and support dev and testing team.
- Data migration design.
- Define and execute data migration.
- Implement POC with limited developer support.
- Provide expert advice to product teams on best way to use existing services and utilities from the repository.
- Design and lead the Azure DevOps implementation for data platform.
- Define logical and physical data models.
Lead:
- Lead the development team in best practice DevOps processes.
- Engage in and improve the whole lifecycle of services—from inception and design, through deployment, operation and refinement.
- Coaching and mentoring the product team engineers and developers on DevOps practices.
- Collaborate with the shared platform teams/architecture team and a wider community internally and externally.
Operational Reliability:
- Act as the technical point of contact on the platform for the cloud service providers (e.g. Azure) to ensure the Platform is operational and is able to meet the service levels of the products using the platform.
- Support services before they go live through activities such as system design consulting, developing software platforms and frameworks, capacity planning and launch reviews.
- Maintain services once they are live by measuring and monitoring availability, latency and overall system health.
- Scale systems sustainably through mechanisms like automation, and evolve systems by pushing for changes that improve reliability and velocity.
- Lead incident response and resolve issues on the platform.
- Continuously improve our security, failover, resilience and disaster recovery mechanisms.
- Work with solution architect and Lead Developers on the adoption of new technologies into the estate, ensuring that these are shared with other areas and that they align with organizational standards.
- Support the team in identifying issues with code/deliverables.
- Champion and drive through alerting and monitoring requirements for the platform.
- Identify and execute pro-active actions to ensure continued stability and performance of the platform.
Our ideal candidate:
Strong experience in Designing and delivering Azure based data platform solutions, technologies including:
- Azure Data Bricks, Azure Synapse Analytics.
- Azure MS SQL managed service/Azure PostgreSQL.
- Azure ADF and Data Bricks, Azure Functions, App Service, Logic App, AKS, Azure App Service, Web App.
- Good knowledge in real-time streaming applications preferably with experience in Kafka Real-Time Messaging or Azure Stream Analytics/Event Hub.
- Spark processing and performance tuning.
- File formats partitioning for e.g. Parquet, JSON, XML, CSV.
- Azure DevOps, GIT hub actions.
- Hands-on experience in at least one of Python with knowledge of the others.
- Experience in Data modeling.
- Experience of synchronous and asynchronous interface approaches.
- Experience of designing and developing systems using microservices architectural patterns.
- DevOps experience in implementing development, testing, release and deployment processes using DevOps processes.
- Knowledge in data modeling (3NF/Dimensional modeling/Data Vault2).
- Work experience in agile delivery.
- Able to provide comprehensive documentation.
- Able to set and manage realistic expectations for timescales, costs, benefits and measures for success.
- Able to lead, undertake and interpret technical analysis.
Autonomy:
Works under broad direction. Work is often self-initiated. Is fully responsible for meeting allocated technical and/or project/supervisory objectives. Establishes milestones and has a significant role in the assignment of tasks and/or responsibilities.
Influence:
Influences organization, customers, suppliers, partners and peers on the contribution of own specialism. Builds appropriate and effective business relationships. Makes decisions, which affect the success of assigned work, i.e. results, deadlines and budget. Has significant influence over the allocation and management of resources appropriate to given assignments.
Complexity:
Performs an extensive range and variety of complex technical and/or professional work activities. Undertakes work, which requires the application of fundamental principles in a wide and often unpredictable range of contexts. Understands the relationship between own specialism and wider customer/organizational requirements.
Nice to Have:
- Experience with integration and implementation of data cataloging tool like Azure Purview, Unity Catalog.
- Experience in implementing and integrating visualizations tools like Power BI/Tableau etc.
- Snowflake knowledge.