Share this Job

Big Data Engineer

Date: Aug 12, 2022

Location: Atlanta, GA, US, 30318

Company: Norfolk Southern Corp.

#LI-EB23 #LI-Hybrid

Job Description:

Norfolk Southern Corporation is currently seeking an experienced Data Engineer – Big Data individual for their Midtown office in Atlanta, GA.  The successful candidate must have Big Data engineering experience and must demonstrate an affinity for working with others to create successful solutions.  Join a smart, highly skilled team with a passion for technology, where you will work on our state-of-the-art Big Data Platforms (Cloudera).  They must be a very good communicator, both written and verbal, and have some experience working with business areas to translate their business data needs and data questions into project requirements.  The candidate will participate in all phases of the Data Engineering life cycle and will independently and collaboratively write project requirements, architect solutions and perform data ingestion development and support duties. 

Required Skills and Experience:

  • 6+ years of overall IT experience
  • 3+ years of experience with high-velocity high-volume stream processing: Apache Kafka and Spark Streaming
    • Experience with real-time data processing and streaming techniques using Spark structured streaming and Kafka
    • Deep knowledge of troubleshooting and tuning Spark applications
  • 3+ years of experience with data ingestion from Message Queues (Tibco, IBM, etc.) and different file formats across different platforms like JSON, XML, CSV
  • 3+ years of experience with Big Data tools/technologies like Hadoop, Spark, Spark SQL, Kafka, Sqoop, Hive, S3, HDFS, or Cloud platforms e.g. AWS, GCP, etc.
  • 3+ years of experience building, testing, and optimizing ‘Big Data’ data ingestion pipelines, architectures and data sets
  • 2+ years of experience with Python (and/or Scala) and PySpark/Scala-Spark
  • 2+ years of experience with NoSQL databases, including HBASE and/or Cassandra
  • Knowledge of Unix/Linux platform and shell scripting is a must
  • Strong analytical and problem-solving skills

Preferred (Bonus) Skills and Experience:

  • Experience with Cloudera/Hortonworks CDP, HDP and HDF platforms
  • Experience with Kudu and Impala
  • Strong  SQL skills with ability to write intermediate complexity queries
  • Strong understanding of Relational & Dimensional modeling  
  • Experience with GIT code versioning software
  • Experience with REST API and Web Services
  • Good business analyst and requirements gathering/writing skills

Education:

Bachelor’s Degree required. 

Preferably in Information Systems, Computer Science, Computer Information Systems or related field

Norfolk Southern Corporation (NYSE: NSC) is a Fortune 300 organization and one of the nation’s premier transportation companies. Its Norfolk Southern Railway Company subsidiary operates approximately 19,500 route miles in 22 states and the District of Columbia, serves every major container port in the eastern United States, and provides efficient connections to other rail carriers. Norfolk Southern is a major transporter of industrial products, including chemicals, agriculture, and metals and construction materials. In addition, the railroad operates the most extensive intermodal network in the East and is a principal carrier of coal, automobiles, and automotive parts.

 

At Norfolk Southern, we believe in celebrating our individuality. By leveraging the unique backgrounds and viewpoints of our employees, we can create a culture of innovation, respect, and inclusion. We know that employees thrive in a workplace where differing viewpoints, ideas, and experiences are freely shared and valued. As such, we encourage all employees to contribute their distinctive skills and capabilities to our organization.

 

Equal employment opportunities are available to all applicants regardless of race, color, religion, age, sex, national origin, disability status, genetic information, veteran status, sexual orientation, and gender identity. Together, we power progress.


Nearest Major Market: Atlanta