JOB DISCRIPTION:
Title: Data Engineer
Location: Dallas, TX (Hybrid)
We are seeking an experienced Data Engineer to design and maintain scalable data pipelines in Databricks. The role involves end-to-end ownership of data workflows, from ingestion to production, ensuring performance, quality, and reliability.
Responsibilities
- Build and optimize pipelines in Databricks (PySpark, SQL, Delta Lake).
- Ingest data from databases, APIs, PDFs, Excel, flat files, and web scraping.
- Implement data quality checks, validation, and testing frameworks.
- Deploy, monitor, and optimize pipelines in Databricks/AWS.
- Collaborate with data science/analytics teams and document data processes.
Qualifications
- 8+ years of Data Engineering experience.
- Hands-on expertise in Databricks (PySpark, SQL, Delta Lake).
- Strong experience with structured & unstructured data.
- Knowledge of data quality/testing tools (Great Expectations, Deequ, dbt tests).
- Proven ability to optimize large-scale Spark/Databricks workloads.
Email : shraddha.m@datasysamerica.com
Job Types: Full-time, Contract
Work Location: On the road
