Databricks Data Engineer

BCforward is currently seeking an Databricks Data Engineer in Pleasanton, CA, Onsite!

Position Title: Databricks Data Engineer
Location: Pleasanton, CA 4 days a week onsite
Anticipated Start Date: 09/01/2025
Please note this is the target date and is subject to change. BCforward will send official notice ahead of a confirmed start date.
Expected Duration: 5+ Months
Pay Range: $48hr-$50/hour
Please note that actual compensation may vary within this range due to factors such as location, experience, and job responsibilities, and does not encompass additional non-standard compensation (e.g., benefits, paid time off, per diem, etc.)

Job overview:
We are seeking a skilled Data Engineer with hands-on Databricks experience to design, build, and optimize large-scale data pipelines and analytics solutions. You will work with cross-functional teams to enable scalable data processing using the Databricks Lakehouse Platform on Azure.

Key Responsibilities:
* Design and implement ETL/ELT pipelines using Databricks, Delta Lake, and Apache Spark
* Collaborate with data scientists, analysts, and stakeholders to deliver clean, reliable, and well-modeled data
* Build and manage data workflows with Databricks Jobs, Notebooks, and Workflows
* Optimize Spark jobs for performance, reliability, and cost-efficiency
* Maintain and monitor data pipelines, ensuring availability and data quality
* Implement CI/CD practices for Databricks notebooks and infrastructure-as-code (e.g., Terraform, Databricks CLI)
* Document data pipelines, datasets, and operational processes
* Ensure compliance with data governance, privacy, and security policies

Qualifications:
* Bachelor's or Master's in Computer Science, Data Engineering, or a related field
* 5+ years of experience in data engineering or a similar role
* Strong hands-on experience with Databricks and Apache Spark (Python, Scala, or SQL)
* Proficiency with Delta Lake, Unity Catalog, and data lake architectures
* Experience with cloud platforms (Azure, AWS, or GCP), especially data services (e.g., S3, ADLS, BigQuery)
* Familiarity with CI/CD pipelines, version control (Git), and job orchestration tools (Airflow, DB Workflows)
* Strong understanding of data warehousing concepts, performance tuning, and big data processing

Preferred Skills:
* Experience with MLflow, Feature Store, or other machine learning tools in Databricks
* Knowledge of data governance tools like Unity Catalog or Purview
* Experience integrating BI tools (Power BI, Tableau) with Databricks
* Databricks certification(s) (Data Engineer Associate/Professional, Machine Learning, etc.)

Benefits:
BCforward offers all eligible employees a comprehensive benefits package including, but not limited to major medical, HSA, dental, vision, employer-provided group life, voluntary life insurance, short-term disability, long-term disability, and 401k.

About BCforward:
Founded in 1998 on the idea that industry leaders needed a professional service, and workforce management expert, to fuel the development and execution of core business and technology strategies, BCforward is a Black-owned firm providing unique solutions supporting value capture and digital product delivery needs for organizations around the world. Headquartered in Indianapolis, IN with an Offshore Development Center in Hyderabad, India, BCforward's 6,000 consultants support more than 225 clients globally.
BCforward champions the power of human potential to help companies transform, accelerate, and scale. Guided by our core values of People-Centric, Optimism, Excellence, Diversity, and Accountability, our professionals have helped our clients achieve their strategic goals for more than 25 years. Our strong culture and clear values have enabled BCforward to become a market leader and best in class place to work.
BCforward is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, or protected veteran status and will not be discriminated against based on disability.
To learn more about how BCforward collects and uses personal information as part of the recruiting process, view our Privacy Notice and CCPA Addendum. As part of the recruitment process, we may ask for you to disclose and provide us with various categories of personal information, including identifiers, professional information, commercial information, education information, and other related information. BCforward will only use this information to complete the recruitment process.
This posting is not an offer of employment. All applicants applying for positions in the United States must be legally authorized to work in the United States. The submission of intentionally false or fraudulent information in response to this posting may render the applicant ineligible for the position. Any subsequent offer of employment will be considered employment at-will regardless of the anticipated assignment duration.

Interested candidates please send resume in Word format Please reference job code 243239 when responding to this ad.