Overview
Embark on an exciting journey as a Data Engineer, where you will be at the forefront of designing, developing, and maintaining robust data pipelines and architectures that empower data-driven decision-making. This role offers the opportunity to work with cutting-edge big data technologies and cloud platforms, transforming complex data sets into actionable insights. Join a dynamic team committed to innovation, collaboration, and continuous learning as you help shape the future of data management and analytics.
Responsibilities
- Design, build, and optimize scalable data pipelines using ETL (Extract, Transform, Load) processes to ensure efficient data flow across systems.
- Develop and maintain data warehouses and data lakes utilizing platforms such as Azure Data Lake and Hadoop to support analytics initiatives.
- Collaborate with cross-functional teams to understand their data needs and deliver tailored solutions leveraging SQL, Python, Java, and shell scripting.
- Implement and manage big data frameworks including Apache Spark, Hive, and Hadoop for processing large datasets effectively.
- Integrate diverse data sources such as Oracle, Microsoft SQL Server, Linked Data, and RESTful APIs to create comprehensive datasets for analysis.
- Support model training and analysis efforts by preparing datasets and ensuring data quality through rigorous validation procedures.
- Maintain database design standards while ensuring compliance with security protocols in an Agile development environment.
Requirements
- Proven experience working with cloud platforms such as AWS or Azure, especially in managing data lakes and related services.
- Strong proficiency in SQL (including Microsoft SQL Server and Oracle), Python, Java, Bash (Unix shell), and Shell Scripting for automation and data processing tasks.
- Hands-on experience with big data technologies like Hadoop, Spark, Hive, and Talend for large-scale data processing.
- Familiarity with ETL tools such as Informatica or Talend to streamline data integration workflows.
- Knowledge of database design principles along with expertise in Data Warehouse architecture.
- Experience working with Looker or similar BI tools for analytics visualization.
- Understanding of RESTful APIs for integrating external systems and datasets.
- Ability to work within an Agile methodology while managing multiple priorities effectively.
- Strong analysis skills combined with attention to detail for ensuring high-quality data outputs. Join us in transforming raw data into powerful insights that drive strategic decisions! We’re dedicated to fostering an inclusive environment where innovation thrives—empowering you to grow your skills while making a meaningful impact through your work as a Data Engineer.
Pay: $60,000.00 - $80,000.00 per year
Benefits:
- Dental insurance
- Health insurance
- Vision insurance
Work Location: In person
