Lead Data Engineer Job at Argyllinfotech, San Jose, CA

YjUyL1dFL3hXZk94WVdVaGhEYTN2U295MFE9PQ==
  • Argyllinfotech
  • San Jose, CA

Job Description

Hello Everyone,

No GC,

Role: Senior Lead Data Engineer

Location: San Jose, CA 95125 Hybrid(Locals)

Key: Hadoop, Spark SQL strong

Key Responsibilities:

Architect, construct , test and maintain ELT pipelines using Hadoop and Spark SQL

- Manage and optimize the infrastructure ensuring reliability and performance

-Design and implement data models and schemas for enterprise data warehouse and data marts

-Write and maintain scripts in Python for various applications

-Develop, optimize, troubleshoot complex SQL queries and database structures for relational and non-relational databases

- Work closely with analysts and product managers to understand data requirements and deliver high-quality data

-Provide technical guidance and mentorship to other team members fostering best practices in data engineering

-Monitor system performance, identify bottlenecks, implement solutions to ensure high data availability

Required Skills:

-High proficiency in the Hadoop ecosystem (HDFS, MapReduce, Yarn) and Apache Spark, Scala for large scale data processing

-Good knowledge in Python for data manipulation and scripting

-Mastery of advanced SQL and deep experience working with relational databases and data warehouses

-Hands-on experience with workflow management tools like Apache Airflow

-Familiarity with Unix shell scripting for automation and system management

Job Tags

Local area,

Similar Jobs

XPO Logistics

Truck Driver - Home Daily - CDL A Job at XPO Logistics

 ...What youll need to succeed as a Truck Driver at XPO Minimum qualifications: Be at least 21 years of age Valid Class A commercial driver's license Safe driving record and history Currently possess or be willing to obtain hazardous materials, tank vehicle... 

Cleveland Clinic

Medical Technologist/MLT/Lab Technician Job at Cleveland Clinic

 ...specialists.We will hire our clinical laboratory testing personnel in three different job titles, based on educational background:+ Medical Technologist+ Med Lab Technician+ Lab Technician**A caregiver in this role works days, 7:00am-3:00pm or 3:00pm-11:30pm.**A caregiver... 

Public Health Solutions

Health Coach Job at Public Health Solutions

 ...Company Overview Health disparities among New Yorkers are large, persistent, and increasing. Public Health Solutions (PHS) exists to...  ...culturally relevant nutrition and physical activity and Health Coach support for pregnant and postpartum patients to improve maternal... 

NBC Universal

Lead Data Engineer Job at NBC Universal

JOB SUMMARY: Responsible for technical design, writing, support and operations of efficient, scalable, and extensible data integrations and ingestions. Implement effective solutions from the ground up and investigate anomalies in a mature running data platform and environment... 

Batter & Berries

Steward/ dishwasher Job at Batter & Berries

Wash plates, glassware, silverware pots and pans to uniform standards Repeatedly lift up to 50Lbs Prevent loss & breakage of plates, glassware and stemware Maintain clean & orderly work space Observe safety and sanitary standards Take out garbage Stand...