senior data engineer.
beaverton , oregon (remote)
posted today
job detailssummary
$50 - $59.30 per hour
contract
bachelor degree
category computer and mathematical occupations
reference1073086
job detailsjob summary:As a Senior Data Engineer, you will play a critical role in designing, building, and maintaining our big data infrastructure, ensuring the scalability, reliability, and performance of our data systems. We are looking for a highly skilled Senior Data Engineer with a strong background in big data engineering, cloud computing, and software development. The ideal candidate will have a proven track record of designing and implementing scalable data solutions using AWS, Spark, and Python. The candidate should have hands-on experience with Databricks, optimizing Spark applications, and building ETL pipelines. Experience with CI/CD, unit testing, and big data problem-solving is a plus.Key Responsibilities:Design, build, and maintain large-scale data pipelines using AWS EMR, Spark, and PythonDevelop and optimize Spark applications and ETL pipelines for performance and scalabilityCollaborate with product managers and analysts to design and implement data models and data warehousing solutionsWork with cross-functional teams to integrate data systems with other applications and servicesEnsure data quality, integrity, and security across all data systemsDevelop and maintain unit test cases for data pipelines and applicationsImplement CI/CD pipelines for automated testing and deploymentCollaborate with the DevOps team to ensure seamless deployment of data applicationsStay up to date with industry trends and emerging technologies in big data and cloud computingRequirements:At least 5 years of experience in data engineering, big data, or a related fieldProficiency in Spark, including Spark Core, Spark SQL, and Spark StreamingExperience with AWS EMR, including cluster management and job optimizationstrong skills in Python, including data structures, algorithms, and software design patternsHands-on experience with Databricks, including Databricks Lakehouse (advantageous)Experience with optimizing Spark applications and ETL pipelines for performance and scalabilityGood understanding of data modeling, data warehousing, and data governanceExperience with CI/CD tools such as Jenkins, GitLab, or CircleCI (advantageous)strong understanding of software development principles, including unit testing and test-driven developmentAbility to design and implement scalable data solutions that meet business requirementsstrong problem-solving skills, with the ability to debug complex data issuesExcellent communication and collaboration skills, with the ability to work with cross-functional teamsNice to Have:Experience with Databricks LakehouseKnowledge of data engineering best practices and design patternsExperience with agile development methodologies, such as Scrum or Kanbanlocation: BEAVERTON, Oregonjob type: Contractsalary: $50.00 - 59.30 per hourwork hours: 8am to 4pmeducation: Bachelorsresponsibilities:Design, build, and maintain large-scale data pipelines using AWS EMR, Spark, and PythonDevelop and optimize Spark applications and ETL pipelines for performance and scalabilityCollaborate with product managers and analysts to design and implement data models and data warehousing solutionsWork with cross-functional teams to integrate data systems with other applications and servicesEnsure data quality, integrity, and security across all data systemsDevelop and maintain unit test cases for data pipelines and applicationsImplement CI/CD pipelines for automated testing and deploymentCollaborate with the DevOps team to ensure seamless deployment of data applicationsStay up to date with industry trends and emerging technologies in big data and cloud computingqualifications:
Experience level: Experienced
Minimum 5 years of experience
Education: Bachelors
skills:
Data AnalysisEqual Opportunity Employer: Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other legally protected group status.At Randstad Digital, we welcome people of all abilities and want to ensure that our hiring and interview process meets the needs of all applicants. If you require a reasonable accommodation to make your application or interview experience a great one, please contact [email protected] offered to a successful candidate will be based on several factors including the candidate's education, work experience, work location, specific job duties, certifications, etc. In addition, Randstad Digital offers a comprehensive benefits package, including health, an incentive and recognition program, and 401K contribution (all benefits are based on eligibility).This posting is open for thirty (30) days.