Big Data Engineers job at Leidos

Job Description

Are you ready to join Leidos all-star team? Through training, teamwork, and exposure to challenging technical work, let Leidos show how to accelerate your career path.

The Leidos Innovations Center has an exciting opening for you, our next Senior Data Engineers to assist with the design, development and implementation of alternative data ingestion pipelines to augment the National Media Exploitation Center (NMEC) data services working in Reston, VA. The DOMEX Data Discovery Platform (D3P) program is a next generation machine learning pipeline platform providing cutting edge data enrichment, triage, and analytics capabilities to Defense and Intelligence Community members. This senor engineer will play a vital role collaborating as part of a cross-functional Agile team to create and enhance data ingestion pipelines and addressing Big data challenges.

You will work closely with the chief architect, systems engineers, software engineers, and data scientists on the following key tasks:

Provide Extraction, Transformation, and Load (ETL) experience coupled with enterprise search capabilities to solve Big Data challenges
Design and implement high-volume data ingestion and streaming pipelines using Open Source frameworks like Apache Spark, Flink, Nifi, and Kafka on AWS Cloud
Leverage strategic and analytical skills to understand and solve customer and business centric questions
Create prototypes and proofs of concept for iterative development
Learn new technologies and apply the knowledge in production systems
Monitor and troubleshoot performance issues on the enterprise data pipelines and the data lake
Partner with various teams to define and execute data acquisition, transformation, processing and make data actionable for operational and analytics initiatives

To be successful in this role you need these skills (required):

BS in Computer Science, Systems Engineering, or related technical field or equivalent experience with at least 8+ years in systems engineering or administration (6+ years with a MS/MIS Degree).
Must have an active Top Secret security clearance and able to obtain a TS/SCI with Polygraph.
3 years of experience with big data tools: Hadoop, Spark, Kafka, NiFi
3 years of experience with object-oriented/object function scripting languages: Python (preferred) and/or Java
3 years of experience with and managing data across relational SQL and NoSQL databases like MySQL, Postgres, Cassandra, HDFS, Redis, and Elasticsearch
3 years of experience working in a Linux environment
2 years of experience working with and designing REST APIs
Experience in designing/developing platform components like caching, messaging, event processing, automation, transformation and tooling frameworks
Experience developing data ingest workflows with stream-processing systems: Spark-Streaming, Kafka Streams and/or Flink
Experience transforming data in various formats, including JSON, XML, CSV, and zipped files
Experience with performance tuning of ETL jobs
Experience developing flexible ontologies to fit data from multiple sources and implementing the ontology in the form of database mappings / schemas
Strong interpersonal and communication skills necessary to work effectively with customers and other team members.

It would be great if you have specific experiences and skills with the following (preferred):

Data engineering experience in Intelligence Community or other government agencies
Experience with Microservices architecture components, including Docker and Kubernetes. Experience developing microservices to fit data cleansing, transformation and enrichment needs.
Experience with AWS cloud services: EC2, S3, EMR, RDS, Redshift, Athena and/or Glue
Experience with Jira, Confluence and extensive experience with Agile methodologies.
Knowledge about security and best practices.
Experience developing flexible data ingest and enrichment pipelines, to easily accommodate new and existing data sources
Experience with software configuration management tools such as Git/Gitlab, Salt, Confluence, etc.
Experience with continuous integration and deployment (CI/CD) pipelines and their enabling tools such as Jenkins, Nexus, etc.
Detailed oriented/self-motivated with the ability to learn and deploy new technology quickly

Additional Program Information
The DOMEX Data Discovery Platform (D3P) program will advance the state of the art in mission-focused big data analytics tools and micro-service development spanning the breadth of Agile sprints to multiyear research and development cycles. We are looking for you to have a demonstrated aptitude for problem solving complex technical issues, identifying, transforming, thinking outside the box, and a strong sense of accountability. Have a mix of technical excellence, intellectual curiosity, communications skills, customer-focus, and operational experience to improve the performance and user adoption of high-end data analytics platforms in partnership with a highly qualified, highly motivated team. Be motivated, self-driven team player who can multi-task and interact well with others and advise/consult with other team members on systems engineering and software development related issues.

APPLY: http://careers.leidos.com/d3p

Interested in this position?

Click here to learn how to get in touch