Azure Data Engineer

Job Description…

As a Senior Azure Data Engineer, you will be a key member of our data engineering team responsible for designing, implementing, and optimizing data pipelines and solutions on the Azure cloud platform. You will play a crucial role in building and maintaining data infrastructure, ensuring data availability, reliability, and performance. This position requires a strong background in Azure data services, ETL processes, and a commitment to best practices in data engineering

Roles & Responsibilities :

The role must execute the following core responsibilities:

• • Design, develop, and maintain end-to-end data pipelines, leveraging Azure Data Factory, Azure Databricks, and other Azure data services.
  • Implement efficient data extraction, transformation, and loading (ETL) processes.
  • Experience in building end to end architecture for Data Lakes, Data Warehouses and Data Marts
  • Collaborate with data architects and other stakeholders to design scalable and robust data solutions.
  • Ensure data integration across various source systems and data stores.
  • Perform data transformation, cleansing, and enrichment to support analytics and reporting needs.
  • Optimize data processing for performance and scalability.
  • Leverage Azure cloud services, such as Azure SQL Data Warehouse, Azure Synapse Analytics, and Azure Data Lake Storage, to build and manage data solutions.
  • Implement best practices for Azure resource management and cost optimization.
  • Implement data governance policies and ensure data security and compliance with industry standards.
  • Manage access controls, encryption, and auditing.
  • Monitor data pipelines and infrastructure performance, identifying and resolving bottlenecks.
  • Continuously optimize data processes for efficiency.
  • Maintain comprehensive documentation of data pipelines, architecture, and configurations.
  • Share knowledge and mentor junior team members.

Qualification :

• • Bachelor’s or master’s degree in computer science, Information Technology, or a related field.
  • 5 to 10 years of experience in data engineering with a focus on Azure cloud technologies.
  • Expert in Azure Data Factory, Azure Databricks, Azure SQL Data Warehouse, and Azure Data Lake Storage.
  • Strong SQL and scripting skills for data manipulation and automation.
  • Knowledge of data warehousing concepts, data modeling, and data integration patterns.
  • Expertise in Python & PySpark development, DWH, Data Integration, Cloud, Architecture, Design, Data Modelling, Cloud data warehouse like Azure Synapse, Snowflake analytical warehouse
  • Familiarity with data governance and compliance standards.
  • Excellent problem-solving and troubleshooting skills.
  • Strong communication and collaboration skills.

Certifications :

• • Microsoft Certified: Azure Data Engineer Associate
  • Microsoft Certified: Azure Solutions Architect Expert