Mid-level Data Engineer
Minimum Qualifications:
- Masters in computer applications or equivalent OR bachelors degree in engineering or computer science or equivalent.
- Deep understanding of Hadoop and Spark Architecture and its working principle.
- Deep understanding of Data warehousing concepts.
- Ability to design and develop optimized Data pipelines for batch and real time data processing.
- Experience in data analytics and cleansing
- 5+ years of software development experience.
- 5+ years experience on Python or JavaHands-on experience on writing and understanding complex SQL (Hive/PySpark-data frames), optimizing joins while processing huge amount of data.
- 3+ years of hands-on experience of working with Map-Reduce, Hive, Spark (core, SQLand PySpark).
- Hands on Experience on Google Cloud Platform (BigQuery, DataProc, Cloud Composer)
- Hands on Experience in Airflow
- 3+ years of experience in UNIX shell scripting
- Should have experience in analysis, design, development, testing, and implementation of system applications.
- Ability to effectively communicate with internal and external business partners.
Additional requirements:
- Understanding of Distributed eco system.
- Experience in machine learning models, RAG, and NLP
- Experience in designing and building solutions using Kafka streams or queues.
- Experience with NoSQL i.e., HBase, Cassandra, Couchbase or MongoDB
- Experience with Data Visualization tools like PowerBI, Tableau, SiSense, Looker
- Ability to learn and apply new programming concepts.
- Knowledge of Financial reporting ecosystem will be a plus.
- Experience in leading teams of engineers and scrum teams
Recommended Jobs
DENTAL RECEPTIONIST
DENTAL RECEPTIONIST - CHANDLER, AZ. MUST HAVE DENTAL EXPERIENCE AND BI-LINGUAL ENGLISH AND SPANISH We are a private dental practice located in the Chandler, Arizona area, seeking a Dental Front O…
Academic Cerebrovascular Neurologist
Why Mayo Clinic Mayo Clinic is top-ranked in more specialties than any other care provider according to U.S. News & World Report. As we work together to put the needs of the patient first, we ar…
Senior Water Resources Engineer
Why Black and Veatch At Black & Veatch, you have the opportunity to contribute your talent and perspective to tackling humanity’s greatest challenges in a flexible environment that empowers you to…
Maternal Fetal Medicine Physician
Maternal Fetal Medicine Physician TMC Health, Southern Arizona’s locally governed nonprofit health system is currently recruiting for a maternal fetal medicine physician to join our employed group…
Dental Assistant
Arizona Biltmore Dentistry, a leading dental practice in Scottsdale, AZ, is seeking a skilled and compassionate Dental Assistant to join our team. We are committed to providing comprehensive and pati…
Pest Control and Lawn Service Technician
Lawn and Pest Technician – Prime Pest and Lawn Tucson, AZ One of the fastest-growing pest control companies in the nation, Prime Pest and Lawn is on a mission to create the best Pest and Lawn…
General Manager/Operating Partner - Europe Trip, No Sundays
An established industry player is seeking a General Manager to lead its vibrant restaurant team. This role offers a unique opportunity to manage daily operations while fostering a positive work cultu…
Full Time Direct Care Provider (RBT/BT)
We are hiring for another Registered Behavior Technician. If you have experience as a Registered Behavior Technician, we would love to work with you! About our Crew: Creative Therapy Solution…
Junior Project Coordinator
Title : Junior Project Coordinator Department : Product Engineering Reports to : CEO Location : Scottsdale, Arizona About Wise Pelican Wise Pelican is hiring a dedicated Junior …
Senior Principal Software Engineer - DoD-Cleared, Onsite
A leading aerospace company in Tucson, AZ is seeking a Senior Principal Software Engineer. The role involves guiding teams in software development for test systems. Candidates need at least 10 years …