Job Overview
We are seeking a skilled Data Engineer to join our dynamic team. The ideal candidate will be responsible for designing, constructing, and maintaining scalable data pipelines and architectures that support our analytical and operational needs. This role requires a strong understanding of big data technologies, database design, and data warehousing principles. The Data Engineer will work closely with data scientists, analysts, and other stakeholders to ensure that data is accessible and usable for decision-making processes.
Duties
- Develop and maintain robust ETL processes to extract, transform, and load data from various sources into data warehouses.
- Design and implement scalable data architectures using technologies such as AWS, Azure Data Lake, Hadoop, and Apache Hive.
- Collaborate with cross-functional teams to gather requirements and translate them into technical specifications.
- Utilize programming languages such as Java, Python, and SQL to build data solutions that meet business needs.
- Implement analytics solutions using tools like Looker to visualize data insights effectively.
- Conduct model training and analysis to enhance the quality of data-driven decisions.
- Ensure the integrity of data through effective database design and management practices.
- Work in an Agile environment to deliver high-quality results within defined timelines.
- Utilize tools like Informatica, Talend, and Spark for efficient data processing and management.
- Write shell scripts (Bash/Unix) for automation of routine tasks.
Experience
- Proven experience in working with big data technologies such as Hadoop, Spark, and Apache Hive.
- Strong knowledge of cloud platforms including AWS and Azure Data Lake.
- Proficiency in database management systems like Microsoft SQL Server, Oracle, and NoSQL databases.
- Experience with RESTful APIs for integrating various systems.
- Familiarity with analytics tools such as Looker for reporting purposes.
- Solid understanding of Agile methodologies in software development.
- Excellent analysis skills with a focus on model training and performance optimization.
- Experience in shell scripting (Bash/Unix) is preferred; knowledge of VBA is a plus. We invite qualified candidates who are passionate about leveraging data to drive business success to apply for this exciting opportunity.
Job Type: Full-time
Pay: $106,439.09 - $128,184.72 per year
Work Location: In person
