Vacancy caducado!
The Sr. Data Engineer's role is to design, develop, test, deploy and maintain software programs and applications. This includes automated unit testing, CI/CD, code reviews, and version control systems (GIT). The Sr. Data Engineer will also oversee system startups in a timely and accurate fashion and provide support to other departments where required.Day to Day
- Gain an understanding of the needs of internal Subject Matter Experts and collaborate with team members to generate autonomous reporting or dashboards
- Collaborate with analysts and designers in the development and initiation of new software programs and applications
- Oversee the quality assurance of program logic, data processing, and error resolution within a complicated and fast-moving system
- Influence the development of a complex system by presenting an awareness of programming and design alternative
- Direct research on emerging application development software products, languages, and standards in support of procurement and development efforts
- Integrate data sources with partner companies or 3rd party data vendors to help our business better understand its operations
- Interact regularly with managers, clients, vendors, and agencies to field queries and questions
- Research, document, and implement program requirements and specifications
- Consistently write, translate, and code software programs and applications according to specifications
- Run and monitor performance tests on new and existing software for the purposes of correcting mistakes, isolating areas for improvement, and general debugging
- Administer diagnostic analysis of test results and deliver solutions to critical areas
- Cultivate and disseminate knowledge of application development best practices
- Determine, develop, and maintain user manuals and guidelines
- Liaise with company's software and hardware suppliers for prompt rectification of any problems or emergencies
- Liaise with network administrators, systems analysts, programmers, and other software engineers to resolve any defects in products or company systems
- Ability to effectively prioritize and execute tasks in a high-pressure environment is crucial
- Proven record of deploying applications on premise or over the web
- Strong Data Lake and Data Warehousing experience
- Experience with IIoT systems
- Proven analytical and problem-solving abilities
- Adept at conducting research into software-related issues and products
- Knowledge of applicable data privacy practices and laws
- Certifications in Microsoft development technologies (.Net, Azure Cloud certifications such as Azure Data Platform, Databricks, Azure DevOps)
- Four-year college diploma or university degree in software engineering or computer science, and/or 5+ years equivalent work experience
- Hands-on expertise in many disparate technologies, typically ranging from front-end user interfaces through to back-end systems and all points in between
- Strong fluency in one or more object-oriented programming languages such as Python or Java
- Extensive ETL pipeline development experience with both streaming and batch processes
- Fluency in Linux (preferably Ubuntu or Debian) and Windows operating systems
- Databasing design and tuning experience in SQL (preferably PostgreSQL) and NoSQL
- Experience in working alongside Data Science teams developing applications
- Experience with Big Data technologies such as Apache Spark, Apache Flink
- Competent and proficient understanding of cloud infrastructure with bonus points for Azure system specifically
- Event-driven architecture and messaging frameworks (Pub/Sub, Kafka, and/or RabbitMQ)
- Experience with Microservices and related technologies - Jenkins, Docker, Kubernetes, Spring Boot
- Hands-on, professional experience integrating with both internal and external REST APIs
- Experience at working both independently and in a team-oriented, collaborative environment is essential