Mid-level Data Engineer
Minimum Qualifications:
- Masters in computer applications or equivalent OR bachelors degree in engineering or computer science or equivalent.
- Deep understanding of Hadoop and Spark Architecture and its working principle.
- Deep understanding of Data warehousing concepts.
- Ability to design and develop optimized Data pipelines for batch and real time data processing.
- Experience in data analytics and cleansing
- 5+ years of software development experience.
- 5+ years experience on Python or JavaHands-on experience on writing and understanding complex SQL (Hive/PySpark-data frames), optimizing joins while processing huge amount of data.
- 3+ years of hands-on experience of working with Map-Reduce, Hive, Spark (core, SQLand PySpark).
- Hands on Experience on Google Cloud Platform (BigQuery, DataProc, Cloud Composer)
- Hands on Experience in Airflow
- 3+ years of experience in UNIX shell scripting
- Should have experience in analysis, design, development, testing, and implementation of system applications.
- Ability to effectively communicate with internal and external business partners.
Additional requirements:
- Understanding of Distributed eco system.
- Experience in machine learning models, RAG, and NLP
- Experience in designing and building solutions using Kafka streams or queues.
- Experience with NoSQL i.e., HBase, Cassandra, Couchbase or MongoDB
- Experience with Data Visualization tools like PowerBI, Tableau, SiSense, Looker
- Ability to learn and apply new programming concepts.
- Knowledge of Financial reporting ecosystem will be a plus.
- Experience in leading teams of engineers and scrum teams
Recommended Jobs
Certified Special Education Teacher
Summary ...Summary ...Certified Teacher - Special Education - Competitive Pay - Multiple Openings. Kaleidoscope... ...and online related services for K-12 education staffing. We will act on y…
Business Development Manager [HT-964114]
CITYSCAPES ARCHITECTURAL INNOVATIONS BUSINESS DEVELOPMENT MANAGER WESTERN US REGION THE PERSON Do you thrive on building lasting relationships and making connections everywhere you go? …
Director, CFR Compliance, Commercial & Fraud
**Description** At American Express, our culture is built on a 175-year history of innovation, shared values and Leadership Behaviors, and an unwavering commitment to back our customers, communities, …
Specialty Beauty Advisor - Clinique
OVERVIEW Experience a place of energy, passion, and excitement. A place where the joy of discovery and uncommon artistry blend to create exhilarating buying experiences—for true beauty enthus…
Tax Manager
Job Description Job Description Tax Manager We are looking for an experienced Tax Manager specialized in Trusts & Estates! Responsibilities Review and prepare federal and state fiduciary i…
QA Manager
QA Developer The QA Developer is responsible for developing, planning and coordinating testing activities. The QA Developer will ensure quality issues and defects are appropriately identified, doc…
Experience Heartfelt Care Amidst Tucson’s Stunning Landscapes
Registered Nurse - Telemetry - Travel - (Tele RN) Join our Telemetry nursing team in sensational Tucson, where breathtaking vistas and vibrant sunsets await you! Experience the thrill of working with…
Phoenix, AZ - Interventional Cardiology - Partnership Track
Interviewing Interventional Cardiologists to join a physician-owned, independent multi-specialty group in Phoenix, Arizona. Practice Highlights: Join 11 Cardiologists and 8 APPs Physicia…
AI-Driven DevOps Automation Engineer - Oracle Health
**Job Description** **Who We Are** Automation and standardization are crucial for scaling cloud-based SaaS offerings to meet growing customer needs. The Cloud Infrastructure Automation Enablement team…
Enterprise Account Executive - Low Code Technology
Mendix - the leading low-code application development platform: The Mendix Platform uses visual modeling to abstract long-form coding out of application development. Our customers use Mendix to create…