Databricks Data Engineer

ABBTECH Professional Resources, Inc.
Phoenix, AZ
**Location: Remote with quarterly travel to Gaithersburg, MD** **Clearance:** **Active** **Public Trust** Databricks Data Engineer to develop/support new/existing data pipelines, and data analytics environments in Azure cloud-based data lake. As a data engineer, you will translate business requirements to data engineering solutions to support an enterprise scale Microsoft Azure based data analytics platform. You will support continued maintenance of ETL operations and development of new pipelines ensuring data quality and data management. The ideal candidate will bring deep expertise in Databricks, a solid foundation in advanced AI technologies, and applies critical thinking to create innovative functions and solve technical issues by cross-functional team collaboration. Based in or willing to relocate to the Washington, DC or Indianapolis to support periodic on-site activities. In this role, you will: - Design, build, and optimize scalable data solutions using Databricks and Medallion Architecture. - Manage ingestion routines for processing multi-terabyte datasets efficiently for multiple projects simultaneously, where each project may have multiple Databricks workspaces. - Integrate data from various structured and unstructured sources to enable high-quality business insights. Proficiency in data analysis techniques for deriving insights from large datasets. - Implement effective data management strategies to ensure data integrity, availability, and accessibility. Identify opportunities for cost optimization in data storage, processing, and analytics operations. - Monitor and support user requests, addressing platform or performance issues, cluster stability, Spark optimization, and configuration management. - Collaborate with the team to enable advanced AI-driven analytics and data science workflows. - Integrate with various Azure services including Azure Functions, Storage Services, Data Factory, Log Analytics, and User Management for seamless data workflows. - Experience with the above Azure services is a plus. - Provision and manage infrastructure using Infrastructure-as-Code (IaC) - Apply best practices for data security, data governance, and compliance, ensuring support for federal regulations and public trust standards. - Proactively collaborate with technical and non-technical teams to gather requirements and translate business needs into data solutions. For this position, you must possess: - BS degree in Computer Science or related field and 3+ years or Master's degree with 2+ years of experience - 3+ years of experience developing and designing Ingestion flows (structured, streaming, and unstructured data) using cloud platform services with data quality - Databricks Data Engineer certification and 2+ years of experience maintaining Databricks platform and development in Spark - Ability to work directly with clients and act as front line support for requests coming in from clients. Clearly document and express the solution in form of architecture and interface diagrams. - Proficient at Python, Spark and R are essential. .NET based development is a plus. - Knowledge and experience with data governance, including metadata management, enterprise data catalog, design standards, data quality governance, and data security. - Experience with Agile process methodology, CI/CD automation, and cloud-based developments (Azure, AWS). - Not required, but additional education, certifications, and/or experience are a plus: Certifications in Azure cloud, Knowledge of FinOps principles and cost management **Position Details:** + Pay Rate / Range:_____$53-$63.34________ _The above salary range represents the range expected for the position; however, final salary offers are based on a number of factors such as the position's responsibilities; the candidate's experience, education, and skills; location; travel required; and current market conditions._ + Benefits (Regular, Full Time Employees): 1. Medical, Dental, and Vision offerings 2. Weekly Direct Deposit 3. Paid Holidays and Personal Time Off 4. 401(k) with match 5. Voluntary Life and AD&D, Short / Long Term Disability, plus other voluntary coverages 6. Pre-Paid Legal and Employee Assistance Programs 7. Northwest Federal Credit Union Membership 8. BB&T @ Work Program This program requires US Citizenship **_ABBTECH is an EOE/Minorities/Women/Disabled Individuals/Veterans_**
Posted 2025-11-12

Recommended Jobs

Physical Therapist - Up to 10K Sign on Bonus-OnSite

Foothills Sports Medicine Physical Therapy
Litchfield Park, AZ

Overview:    Congratulations on Winning Glassdoor's Best Places to Work! Physical Therapist – up to $10k Sign-on Bonus   We are looking for fun loving and constant learning Physical Therap…

View Details
Posted 2025-11-12

Operations Associate

Sephora
Glendale, AZ

Sephora is seeking an Operations Associate in Glendale, United States, to support store operations and ensure an exceptional client experience. Responsibilities include inventory management, visual me…

View Details
Posted 2025-10-03

Service Leader

Chipotle Mexican Grill
Gilbert, AZ

CULTIVATE A BETTER WORLD Food served fast does not have to be a typical fast-food experience. Chipotle has always done things differently, both in and out of our restaurants. We are changing the…

View Details
Posted 2025-11-13

Production Laborer

Advance Services
Goodyear, AZ

Job Description Job Description Certainly! Here's the revised job description with the words "DOT'S" and "Hershey's" removed: Production Laborer Schedule: Monday – Friday, 10:00 PM – 6:30…

View Details
Posted 2025-11-01

M&G Senior Commissioning Technician

ABB
Sedona, Yavapai County, AZ

At ABB, we help industries outrun - leaner and cleaner. Here, progress is an expectation - for you, your team, and the world. As a global market leader, we'll give you what you need to make it happen.…

View Details
Posted 2025-11-15

Sales Associate / Watch Technician

Fast Fix Jewelry & Watch Repair
Gilbert, AZ

Job Description Job Description We are seeking a motivated and detail-oriented Sales Associate / Watch Technician to join our team. This hybrid role blends front-line customer service with hand…

View Details
Posted 2025-09-18

Material Handler Loader

Arizona Steel & Ornamental Supply I
Tucson, AZ

Job Description Job Description As a material handler, you will primarily be taking care of customers by loading and unloading heavy material that has been ordered from the office. Other duties m…

View Details
Posted 2025-10-04

Full-Time and Part-Time Preschool Teachers-Mesa-Gilbert

KIDS CORNER PRESCHOOL AND CHI
Phoenix, AZ

Job Description Job Description Full-Time and Part-Time Competitive Pay! Our employee benefits for full-time employees, include 401(k) with employer matching, Paid-Time Off programs, employee b…

View Details
Posted 2025-09-14

Sr Software Engineer

Disney Entertainment and ESPN Product & Technology
Glendale, AZ

Job Summary: Disney Entertainment and ESPN Product & Technology Technology is at the heart of Disney’s past, present, and future. Disney Entertainment and ESPN Product & Technology is a global …

View Details
Posted 2025-11-13

Healthcare Architect Project Manager

Shepley Bulfinch
Phoenix, AZ

Come build something with Shepley Bulfinch! Shepley Bulfinch is seeking a well-rounded Healthcare Project Manager to join our Phoenix studio. The Healthcare Project Manager is creative and passiona…

View Details
Posted 2025-09-05