Johnson Service Group (JSG) is currently looking for a Data Engineer (Databricks) with 3+ years of experience in Databricks, Azure Data Factory, SQL, and PySpark. This is a hybrid contract opportunity with a capital markets organization with offices across the continental United States, focused on developing, optimizing, and supporting enterprise data pipelines and cloud-based data solutions.
- 3+ years of experience in data engineering.
- Hands-on experience with Databricks for data engineering, including Delta Live Tables (DLT).
- Strong experience with Azure Data Factory (ADF) for data pipeline orchestration.
- Proficiency in SQL and PySpark for data transformation and processing.
- Experience with Azure Synapse Analytics, including Dedicated SQL Pools.
- Experience with Python for data engineering and pipeline development.
- Knowledge of Azure DevOps for development and deployment workflows.
- Experience with Azure Function Apps and Azure Logic Apps.
- Develop, enhance, and maintain data pipelines using existing ingestion frameworks and tools.
- Orchestrate and manage data pipelines using Azure Data Factory.
- Design and implement data transformations to parse, process, and load data into Enterprise Data Lake, Delta Lake, and Enterprise Data Warehouse (Synapse Analytics).
- Develop and maintain Databricks-based data processing solutions using DLT, Python, and PySpark.
- Perform unit testing and coordinate integration testing and User Acceptance Testing (UAT).
- Prepare and maintain technical documentation, including High-Level Designs (HLDs), Detailed Designs (DDs), and operational runbooks.
- Configure compute resources, implement data quality rules, and maintain pipeline reliability.
- Conduct performance tuning and optimization of data pipelines and processing workloads.
- Provide production support, troubleshoot operational issues, and resolve data pipeline failures.
- Data Engineering: Azure Databricks, Delta Live Tables (DLT)
- Data Integration: Azure Data Factory
- Data Warehousing: Azure Synapse Analytics (Dedicated SQL Pool)
- Programming: Python, PySpark, SQL
- DevOps: Azure DevOps
- Azure Services: Azure Function Apps, Azure Logic Apps
- Data Platforms: Enterprise Data Lake, Delta Lake, Enterprise Data Warehouse
Work Arrangement
- Hybrid contract opportunity requiring three days per week onsite.
- Office locations available across the continental United States.
- Capital markets industry environment.
Rate Range- $45-55 USD
Johnson Service Group (JSG) is an Equal Opportunity Employer. JSG provides equal employment opportunities to all applicants and employees without regard to race, color, religion, sex, age, sexual orientation, gender identity, national origin, disability, marital status, protected veteran status, or any other characteristic protected by law.
#LI-KS1
#D480