Vacancy caducado!
- We are looking for Senior Azure Data Engineer to build Data pipelines to work with one of the leading Retail providers in the US. The ideal candidate must possess strong background on Data Engineering development technologies & extensive experience in Azure. The candidate must possess excellent written and verbal communication skills with the ability to collaborate effectively with domain experts and technical experts in the team.
- Experience working with Azure Data Lake Storage Gen2 (ADLS).
- Experience with Azure Data Factory (ADF) and ETL concepts.
- Creating and defining; -Pipelines and Datasets.
- Understanding of variables and parameters and their use in creating dynamic pipelines.
- ADF orchestration concepts.
- Scheduling and triggering ADF jobs.
- Defining data mappings and complex transformation logic within Mapping Flows.
- Understanding of Linked Services.
- Experience with Databricks in Azure.
- Understanding of using Pyspark and Spark-SQL to perform key activities;
- Create data frames from a variety of sources.
- Perform transformations, filtering, data cleansing activities.
- Experience writing PowerShell or another comparable scripting language.
Vacancy caducado!