Job Summary
We are looking for an experienced Senior ETL / Data Engineer with strong expertise in PySpark, modern cloud data platforms, NoSQL databases, Graph Databases, and large-scale ETL pipelines. The ideal candidate should have hands-on experience designing, developing, and optimizing scalable data processing solutions across cloud data warehouse and data lake environments.
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT data pipelines using PySpark and Python.
- Develop and optimize large-scale data processing jobs using Apache Spark / PySpark.
- Work with cloud data platforms such as Databricks, Snowflake, and Amazon Redshift.
- Build data ingestion and transformation pipelines from structured and unstructured data sources.
- Work with NoSQL databases for high-volume and distributed data workloads.
- Design and implement solutions using Graph Databases and graph-based data models.
- Perform data transformation, cleansing, validation, reconciliation, and quality checks.
- Optimize ETL pipelines, Spark jobs, queries, and data storage for performance and scalability.
- Implement data integration across databases, APIs, files, data lakes, and cloud platforms.
- Troubleshoot data pipeline failures and perform root-cause analysis.
- Collaborate with Data Architects, Data Scientists, BI teams, and application teams.
Required Skills
- 7+ years of experience in Data Engineering / ETL development.
- Strong hands-on experience with PySpark / Apache Spark.
- Experience with one or more cloud data platforms: Databricks, Snowflake, or Amazon Redshift.
- Strong experience in ETL/ELT pipeline development.
- Hands-on experience with NoSQL databases.
- Experience with Graph Databases and graph data modeling.
- Strong SQL and Python skills.
- Experience with data transformation, data integration, and performance optimization.
- Good understanding of data warehousing and data lake concepts.
- Experience working with large-scale datasets and distributed processing environments.
Good to Have
Databricks | Snowflake | Amazon Redshift | PySpark | Apache Spark | NoSQL | Graph Database | Python | SQL | ETL/ELT | Data Lake | Data Warehouse | Cloud Data Engineering
