Bigdata/Spark Principal Architect

Job Title: BigData/Spark Principal Architect

Location: Remote, USA

About the Role:
We are seeking an experienced and hands-on BigData/Spark Principal Architect to join our team remotely across the USA. The ideal candidate will have deep expertise in Spark (DataFrame, Spark SQL), Databricks, AWS Glue, and streaming technologies like Kafka. This role demands strong design and architecture skills for big data systems, along with proven solutioning and consulting experience as a Subject Matter Expert (SME) in big data technologies. A background in media and entertainment domains, solution selling, and practice development is highly preferred.

Key Responsibilities:

• Design and develop scalable big data pipelines using Spark (batch and streaming) leveraging technologies such as Spark SQL, Databricks, PySpark/Scala Spark.
• Lead solutioning initiatives and provide expert consulting as a SME on big data architectures and technologies.
• Work with cloud platforms (AWS Glue or Azure) to handle big data workloads efficiently.
• Ensure data governance, data quality, and compliance best practices are met in pipeline architectures.
• Collaborate with cross-functional teams to align big data solutions with business goals.
• Continuously drive innovation in big data operations and architecture best practices.
• Mentor and upskill team members with evolving big data technologies and trends.

Must-Have Qualifications:

• Hands-on expertise with Spark DataFrame, Spark SQL, PySpark/Scala Spark, and Databricks for batch and streaming data pipelines.
• Proven track record as a solution architect and consultant on complex big data systems, serving as an SME.
• Strong cloud experience on AWS (Glue) or Azure for data workloads.
• Hands-on experience with streaming technologies such as Kafka or alternative streaming platforms.
• Solid understanding of ETL processes, data governance, data quality, and overall data operations.
• Background in media and entertainment industry, solution selling, and practice development is a big plus.
• Excellent learning ability and capability to cross-skill on emerging technologies.

Desired Skills:

• Expertise in design and architecture of big data systems.
• Strong problem-solving skills and ability to provide innovative solutions.
• Experience working in agile, collaborative environments.
• Effective communication skills for client-facing consultancy roles.

How to Apply:
If you're ready to grow your career in pharmaceutical manufacturing, apply directly via LinkedIn OR reach out through WhatsApp for quick screening:

#BigData #Spark #Databricks #PySpark #Scala #SparkSQL #DataEngineering #DataArchitect #PrincipalArchitect #SolutionArchitect #AWS #AWSGlue #Azure #Kafka #StreamingData #ETL #DataPipelines #DataGovernance #DataQuality #CloudComputing #MediaTech #EntertainmentTech #TechJobsUSA #RemoteJobsUSA #DataJobs #BigDataEngineering #USJobs #Ceipal