Data Engineer II

Data Engineer IITransform large, complex data into consumable business databases and applications for self-service analytics and reporting; perform data analysis using SQL and Spark SQL queries; develop ETL jobs using Databricks, Jupyter, and Pyspark; automate ETL jobs using AWS services, including S3, Athena, Lambda, RDS, Glue, and EMR; perform data quality checks and exploratory data analysis on large data sets using PySpark; create system architecture, design and specification using in-depth engineering skills and knowledge to solve difficult development problems and achieve engineering goals; determine and source appropriate data for a given analysis; work with data modelers/analysts to understand the business problems and create or augment data assets to support analysis; perform engineering tasks including database design, data manipulation, ETL, implementation, information storage and retrieval, data flow and analysis; resolve technical issues utilizing knowledge of policies and processes; identify and react to system notification and log to ensure quality standards for databases and applications; solve abstract problems beyond single development language or situation by reusing existing data file and flags; solve critical issues and present trends, aggregate, and quantify volume regarding specific data sources; participate in the implementation of solutions via data architecture, data engineering, or data manipulation within big data systems including Hadoop and SQL; collaborate with business owners and technical associates to ensure data collected and processed is both actionable and relevant to end goals; determine solutions for data storage and storage optimization and organization; determine table relations and how fields interact within tables to develop relational models; and collaborate with technology and platform management partners to optimize data sourcing and processing rules to ensure appropriate data quality. Position is eligible for 100% remote work.REQUIREMENTS: Bachelor's degree, or foreign equivalent, in Computer Science, Engineering, or related technical field, and one (1) year of experience performing data analysis using SQL and Spark SQL queries; developing ETL jobs using Databricks, Jupyter, and Pyspark; automating ETL jobs using AWS services, including S3, Athena, Lambda, RDS, Glue, and EMR; and performing data quality checks and exploratory data analysis on large data sets using PySpark.PLEASE SEND RESUMES TO [email protected] AND REFERENCE JOB ID # 9682 WHEN APPLYING FOR THIS POSITION.