Manager Data Engineering
Date: Sep 8, 2023
Location: Atlanta, GA, US, 30308
Company: Norfolk Southern Corp.
#LI-JK1
A resume helps you stand out to hiring managers and recruiters; your resume communicates your experience and your brand. While it is not required, we encourage you to include an up-to-date resume along with a completed job application to give you the best opportunity to be considered. A complete resume helps us to better understand your unique background, relevant experiences, and passions. We look forward to learning about you.
Job Description
The Manager Data Engineering will lead a team responsible for building various cutting-edge Big Data solutions and Big Data pipelines in collaboration with the Data Science, Business Intelligence (BI), software development, engineering, and business teams. They will be focused on driving part of the strategic horizontal Big Data initiatives that advance the skills and capabilities of the organization, while improving the level of enablement for partners and stakeholders. The candidate will participate in all phases of the Data Engineering life cycle and will independently and collaboratively write project requirements, architect solutions, perform data ingestion development, support duties, coach and mentor Data Engineers, and set the data vision for the organization.
Skills and Experience
Responsibilities
• Develop the vision and manage the execution of the Data Engineering roadmap by collaborating with cross-functional teams, products, analytics, engineering, and machine learning experts
• Oversee various parts of the data lifecycle from acquirement to insights, in particular: data ingestion, data warehouse, and data engineering
• Hire, coach, and mentor talented Data Engineers
• Lead Data Engineering team to use state-of-the-art Big Data tools and technologies to build scalable data architecture and performant data pipelines to acquire high-velocity real-time and batch data of different sizes and scales
• Design, implement, and support scalable data infrastructure solutions to integrate with multi-heterogeneous data sources, aggregate and retrieve Big Data in a fast and safe mode, curate data that can be used in BI reporting, analysis, machine learning models and ad-hoc data requests.
• Ensure appropriate levels of governance and testing procedures for developed solutions
• Stay up to date with technology, trends, and tools in Big Data
• Take part in development of data pipelines and solutions
• Optimize and tune data ingestion and data retrieval pipelines
• Implement the solutions in production environment and implement efficient and timely support models
• Manage multiple projects concurrently
Technical Skills
• Bachelor’s degree and at least 5 years of experience in Big Data, BI, and Advance Analytics areas with at least 7 years of experience in IT
• Advanced knowledge of Big Data, Data Engineering, and BI with hands-on and technical experience in tools/technologies like Apache Kafka, Spark, Hadoop, Hive, Sqoop, HBase, Spotfire, Tableau, etc.
• At least 5 years of experience in building data ingestion pipelines using: Spark and/or Spark-Streaming, etc.
• At least 5 years of experience directly managing people, including hiring, developing, motivating, and directing people as they work. Must have experience with leading deeply technical teams
• Experience with relational databases e.g. Teradata, Oracle, SQL Server, as well as NoSQL databases including HBase and Cassandra
• Experience with data engineering productivity toolchains and concepts (CI/CD, DevOps, GitHub, Jira, Confluence, Agile Scrum / Kanban, etc.)
• Hands-on coding experience in at least one modern programming language: Python, Scala, etc. Strong SQL and Python skills, including knowledge of Python libraries/frameworks.
• Knowledge of Unix/Linux platform and shell scripting
• Strong project management skills, including work prioritization, planning, and task delegation
• Ability to effectively communicate (both written and verbally) technical information at all levels of the organization
• Experience in successfully building and deploying a new data platform on Azure/ AWS
• Experience in Azure / AWS Serverless technologies, like, S3, Kinesis/MSK, lambda, and Glue
• Strong knowledge of Messaging Platforms like Kafka, Amazon MSK & TIBCO EMS or IBM MQ Series
• Experience with Databricks UI, Managing Databricks Notebooks, Delta Lake with Python, Delta Lake with Spark SQL, Delta Live Tables, Unity Catalog
• Experience with Cloudera/Hortonworks HDP and HDF platforms is a plus
Other Skills
• Planning and organizing
• Analytical thinking
• Communication skills
• Technical writing
Education
Bachelor’s Degree is required in Information Systems, Computer Science, Computer Information Systems or related field
Work Conditions
Remote, but Atlanta based. The ideal candidate must live in the Atlanta area.
Company Overview
Since 1827, Norfolk Southern Corporation (NYSE: NSC) and its predecessor companies have safely moved the goods and materials that drive the U.S. economy. Today, it operates a customer-centric and operations-driven freight transportation network. Committed to furthering sustainability, Norfolk Southern helps its customers avoid 15 million tons of yearly carbon emissions by shipping via rail. Its dedicated team members deliver more than 7 million carloads annually, from agriculture to consumer goods, and is the largest rail shipper of auto products and metals in North America. Norfolk Southern also has the most extensive intermodal network in the eastern U.S., serving a majority of the country’s population and manufacturing base, with connections to every major container port on the Atlantic coast as well as the Gulf of Mexico and Great Lakes. Learn more by visiting www.NorfolkSouthern.com.
At Norfolk Southern, we believe in celebrating our individuality. By leveraging the unique backgrounds and viewpoints of our employees, we can create a culture of innovation, respect, and inclusion. We know that employees thrive in a workplace where differing viewpoints, ideas, and experiences are freely shared and valued. As such, we encourage all employees to contribute their distinctive skills and capabilities to our organization.
Equal employment opportunities are available to all applicants regardless of race, color, religion, age, sex, national origin, disability status, genetic information, veteran status, sexual orientation, and gender identity. Together, we power progress.
Nearest Major Market: Atlanta