Databricks Data Engineer

ABBTECH Professional Resources, Inc.
Montgomery, AL
**Location: Remote with quarterly travel to Gaithersburg, MD** **Clearance:** **Active** **Public Trust** Databricks Data Engineer to develop/support new/existing data pipelines, and data analytics environments in Azure cloud-based data lake. As a data engineer, you will translate business requirements to data engineering solutions to support an enterprise scale Microsoft Azure based data analytics platform. You will support continued maintenance of ETL operations and development of new pipelines ensuring data quality and data management. The ideal candidate will bring deep expertise in Databricks, a solid foundation in advanced AI technologies, and applies critical thinking to create innovative functions and solve technical issues by cross-functional team collaboration. Based in or willing to relocate to the Washington, DC or Indianapolis to support periodic on-site activities. In this role, you will: - Design, build, and optimize scalable data solutions using Databricks and Medallion Architecture. - Manage ingestion routines for processing multi-terabyte datasets efficiently for multiple projects simultaneously, where each project may have multiple Databricks workspaces. - Integrate data from various structured and unstructured sources to enable high-quality business insights. Proficiency in data analysis techniques for deriving insights from large datasets. - Implement effective data management strategies to ensure data integrity, availability, and accessibility. Identify opportunities for cost optimization in data storage, processing, and analytics operations. - Monitor and support user requests, addressing platform or performance issues, cluster stability, Spark optimization, and configuration management. - Collaborate with the team to enable advanced AI-driven analytics and data science workflows. - Integrate with various Azure services including Azure Functions, Storage Services, Data Factory, Log Analytics, and User Management for seamless data workflows. - Experience with the above Azure services is a plus. - Provision and manage infrastructure using Infrastructure-as-Code (IaC) - Apply best practices for data security, data governance, and compliance, ensuring support for federal regulations and public trust standards. - Proactively collaborate with technical and non-technical teams to gather requirements and translate business needs into data solutions. For this position, you must possess: - BS degree in Computer Science or related field and 3+ years or Master's degree with 2+ years of experience - 3+ years of experience developing and designing Ingestion flows (structured, streaming, and unstructured data) using cloud platform services with data quality - Databricks Data Engineer certification and 2+ years of experience maintaining Databricks platform and development in Spark - Ability to work directly with clients and act as front line support for requests coming in from clients. Clearly document and express the solution in form of architecture and interface diagrams. - Proficient at Python, Spark and R are essential. .NET based development is a plus. - Knowledge and experience with data governance, including metadata management, enterprise data catalog, design standards, data quality governance, and data security. - Experience with Agile process methodology, CI/CD automation, and cloud-based developments (Azure, AWS). - Not required, but additional education, certifications, and/or experience are a plus: Certifications in Azure cloud, Knowledge of FinOps principles and cost management **Position Details:** + Pay Rate / Range:_____$53-$63.34________ _The above salary range represents the range expected for the position; however, final salary offers are based on a number of factors such as the position's responsibilities; the candidate's experience, education, and skills; location; travel required; and current market conditions._ + Benefits (Regular, Full Time Employees): 1. Medical, Dental, and Vision offerings 2. Weekly Direct Deposit 3. Paid Holidays and Personal Time Off 4. 401(k) with match 5. Voluntary Life and AD&D, Short / Long Term Disability, plus other voluntary coverages 6. Pre-Paid Legal and Employee Assistance Programs 7. Northwest Federal Credit Union Membership 8. BB&T @ Work Program This program requires US Citizenship **_ABBTECH is an EOE/Minorities/Women/Disabled Individuals/Veterans_**
Posted 2025-11-12

Recommended Jobs

Estimator - Site development

Noor Staffing Group
Huntsville, AL

We are partnered with a reputable Site Development Contractor located in the Huntsville area. Our client is seeking an experienced Civil Site Development. You'll drive project acquisition through str…

View Details
Posted 2025-10-07

Aircraft Structures Technician

Aerotek
Birmingham, AL

**Job Title: Aircraft Structures Technician** **Job Description** As a Structures Technician, you will perform heavy structural maintenance and modifications on Boeing B717 aircraft. This role involve…

View Details
Posted 2025-11-11

Mid-Level Systems Engineer

IERUS Technologies, Inc.
Huntsville, AL

Description: The primary responsibilities of this position are performing systems engineering using technical planning; system integration, verification and validation; cost and risk studies; and sup…

View Details
Posted 2025-09-12

Senior Peer Support Specialist

Veterans Recovery Resources
Mobile, AL

Job Description • Offer person-centered treatment planning with personally and culturally diverse clients. • Coordinate team of peer support specialists offering recovery services for adults,…

View Details
Posted 2025-11-05

Senior Software Engineer

Confluent
Montgomery, AL

**Location:** Remote, United States **Employment Type:** FullTime **Location Type:** Remote **Department** Engineering **Overview** We're not just building better tech. We're rewriting how data mo…

View Details
Posted 2025-11-12

Senior Sales Executive

Rustman Agency
Opelika, AL

Description Think “Senior” means you need years of experience? Think again. At The Rustman Agency, we're looking for ambitious, coachable go-getters ready to step into a high-impact sales role—no p…

View Details
Posted 2025-09-29

Travel RN - Dialysis - Nights - Tuscaloosa, AL

TMS
Tuscaloosa, AL

Job Description Job Description Location: Tuscaloosa, AL Shift: Nights - 7:00p-7:00a  Weekends: every other Holidays: as required Contract Length: 13 weeks Hours: 36 or 48 hou…

View Details
Posted 2025-11-08

HVAC I

ARAMARK
Mobile, AL

Job Description The HVAC Maintenance Worker is responsible for the operation, repair, maintenance, and installation of heating, ventilation, air conditioning, and refrigeration systems. Essential …

View Details
Posted 2025-11-09

Sr. Business Analyst (Remote)

GovCIO
Montgomery, AL

**Overview** GovCIO is currently seeking a Sr. Business Analyst to join a cross-functional team dedicated to delivering impactful features. This position is fully remote and available to candidates wi…

View Details
Posted 2025-11-01

Mission Specialist - Seeking Servers & Bartenders $$$

Aerial AI Solutions
Huntsville, AL

Job Description Job Description Seeking Bartenders and Servers For LIfe-Changing Opportunity! This is the breakthrough opportunity you’ve been waiting for. Join a fast-growing AI startup offeri…

View Details
Posted 2025-09-25