Data Pipeline Developer Databricks Integration (Remote), Virginia

Join Sky Solutions, LLC - Pioneering Digital Transformation Excellence!

Welcome to Sky Solutions, LLC, a dynamic Digital Transformation Firm nestled in the vibrant heart of the Washington D.C. Metro Area. We specialize in empowering leading organizations in the Financial Services and Healthcare sectors, achieving remarkable business outcomes through cutting-edge Digital Process Automation (DPA) and innovative Low-Code/No-Code Application Development. Sky Solutions is a certified Small Disadvantaged Business (SDB) with Minority Owned certifications. Embark on our journey of innovation and transformation, where your skills align seamlessly with our commitment to excellence.

 

Position Title: Data Pipeline Developer

Duration: Longterm

Location: Remote

 

Note: U.S. citizenship is contractually required for this position.

 

Position Summary:

Build, test, and maintain scalable ETL/ELT pipelines that ingest, transform, and load IRS source systems into Databricks-based lake-house environments.

 

Core Duties:

Author PySpark / SQL ETL jobs and orchestrate via Databricks Workflows or Airflow.

Implement data-quality checks, validation rules, and error-handling according to IRS standards.

Package code into CI/CD pipelines (GitHub Actions, Jenkins) with automated unit & integration tests.

Support 24 × 7 O&M rotation for production incidents; document lessons learned.

 

Required Qualifications:

4 + years’ hands-on pipeline development; ≥ 2 projects of comparable scale loading into Databricks.

Proficiency in PySpark, delta-format, and object-store optimization.

Working knowledge of IRS / federal data security (PII, SBU) or similar compliance constraints.

Ability to obtain MBI and finish initial IRS training within 30 days.

 

Desired:

Experience with Informatica Intelligent Cloud Services (IICS) or AWS Glue.

Exposure to Immuta policy enforcement and fine-grained access controls.

 

Sky Solutions LLC does not discriminate based on race, color, religion, gender, sexual orientation, age, disability, or any other protected status. We welcome applications from all qualified candidates who share our vision and values.