Share this Job

Sr. Big Data Engineer

Date: Jan 12, 2023

Location: Atlanta, GA, US, 30308

Company: Norfolk Southern Corp.



Norfolk Southern Corporation is Fortune-300 transportation company specializing in freight railroading. We operate approximately 21,000 route miles in 22 states and the District of Columbia, serve every major container port in the eastern United States, and provide efficient connections to other rail carriers. Norfolk Southern has the most extensive intermodal network in the East and is a major transporter of coal and industrial products.

Job Description

Norfolk Southern Corporation is currently seeking an experienced Sr Data Engineer – Big Data individual for their Midtown office in Atlanta, GA.  The successful candidate must have Big Data engineering experience and must demonstrate an affinity for working with others to create successful solutions.  Join a smart, highly skilled team with a passion for technology, where you will work on our state of the art Big Data Platforms (Cloudera).  They must be a very good communicator, both written and verbal, and have some experience working with business areas to translate their business data needs and data questions into project requirements.  The candidate will participate in all phases of the Data Engineering life cycle and will independently and collaboratively write project requirements, architect solutions and perform data ingestion development and support duties. 

Skills and Experiences


•    Architect, design, and build Big Data applications to support business strategies and deliver business value
•    Hands-on experience in development, leading code reviews and testing, and working on proofs of concept using latest big data technologies
•    Advanced proficiency in SQL, data ingestion frameworks, data modeling, and working with big data
•    Mentor and inspire Junior team members
•    Willingness to learn new technologies constantly, and work/guide other team members 
•    7+ years of overall IT experience
•    5+ years of experience as a Data Engineer or in a similar role
•    3+ years of experience with Big Data tools/technologies like Hadoop, Spark, Spark SQL, Kafka, Sqoop, Hive, S3, HDFS, or Cloud platforms e.g. AWS, GCP, etc.
•    3+ years of experience with high-velocity high-volume stream processing: Apache Kafka and Spark Streaming 
o    Experience with real-time data processing and streaming techniques using Spark structured streaming and Kafka
o    Deep knowledge of troubleshooting and tuning Spark applications
•    3+ years of experience with data ingestion from Message Queues (Tibco, IBM, etc.) and different file formats across different platforms like JSON, XML, CSV
•    3+ years of experience building, testing, and optimizing ‘Big Data’ data ingestion pipelines, architectures, and data sets
•    2+ years of experience with Kudu and Impala
•    2+ years of experience with Scala (and/or Python) and PySpark/Scala-Spark
•    2+ years of experience with NoSQL databases, including HBASE and/or Cassandra
•    Knowledge of Unix/Linux platform and shell scripting is a must
•    Strong analytical and problem solving skills


    Preferred (Not Required):   
•    Experience with Cloudera/Hortonworks HDP and HDF platforms
•    Strong SQL skills with ability to write intermediate complexity queries
•    Experience with GIT code versioning software
•    Experience with REST API and Web Services
•    Good business analyst and requirements gathering/writing skills


Bachelor’s Degree required.  Preferably in Information Systems, Computer Science, Computer Information Systems or related field

Company Overview

Since 1827, Norfolk Southern Corporation (NYSE: NSC) and its predecessor companies have safely moved the goods and materials that drive the U.S. economy. Today, it operates a customer-centric and operations-driven freight transportation network. Committed to furthering sustainability, Norfolk Southern helps its customers avoid 15 million tons of yearly carbon emissions by shipping via rail. Its dedicated team members deliver more than 7 million carloads annually, from agriculture to consumer goods, and is the largest rail shipper of auto products and metals in North America. Norfolk Southern also has the most extensive intermodal network in the eastern U.S., serving a majority of the country’s population and manufacturing base, with connections to every major container port on the Atlantic coast as well as the Gulf of Mexico and Great Lakes. Learn more by visiting

At Norfolk Southern, we believe in celebrating our individuality. By leveraging the unique backgrounds and viewpoints of our employees, we can create a culture of innovation, respect, and inclusion. We know that employees thrive in a workplace where differing viewpoints, ideas, and experiences are freely shared and valued. As such, we encourage all employees to contribute their distinctive skills and capabilities to our organization.

Equal employment opportunities are available to all applicants regardless of race, color, religion, age, sex, national origin, disability status, genetic information, veteran status, sexual orientation, and gender identity. Together, we power progress.

Effective December 8, 2021, NS has paused its implementation of the COVID-19 vaccine mandate given the dynamic legal proceedings.  The ultimate outcome of the federal contractor vaccine mandate is yet to be determined. It is still possible that the mandate, as it applies to NS, could be upheld in court in the near future. If the mandate is upheld, NS employees will be required to be vaccinated or have an approved medical or religious accommodation.

Nearest Major Market: Atlanta