Engineering services,Fintech,Mobile Wallet,Banking,Telecom services and staff Augmentation is our key...
About the Company
A leading technology organization is expanding its data engineering team and is looking for a Senior Hadoop Developer. This company specializes in managing complex data environments and delivering data-driven solutions that power large-scale systems in production.
About the Role
This position is for an experienced professional with deep knowledge of big data ecosystems, SQL, and data warehousing. The Senior Hadoop Developer will be responsible for building and maintaining scalable data pipelines, optimizing data processes, and contributing to the architecture of data platforms. The role requires hands-on experience with Hadoop technologies, scripting, and performance tuning in production environments.
Key Responsibilities
-
Design, develop, and support big data solutions using Hadoop and related technologies.
-
Create and maintain ETL processes to support data integration and analytics.
-
Optimize complex SQL queries and ensure the performance of large-scale SQL systems.
-
Work with Hadoop ecosystem tools such as Hive, Spark, and Cloudera in a production environment.
-
Develop scripts using Python and Unix shell to automate workflows.
-
Use scheduling tools like Autosys for workflow management.
-
Participate in Agile development practices and team collaboration.
-
Support ongoing improvements in data architecture and pipeline efficiency.
Required Skills and Experience
-
Extensive experience with Hadoop technologies, including Cloudera, Hive, and Spark.
-
Strong knowledge of SQL and relational databases.
-
Proven ability to optimize and tune performance of complex SQL queries.
-
Experience working with big data systems in production environments.
-
Proficient in Python and Unix shell scripting.
-
Familiarity with job scheduling tools such as Autosys.
-
Understanding of Agile methodologies.
-
At least 4 years of experience in data warehousing architecture and a minimum of 4 years working with big data technologies.