Principal Software Engineer, Big Data job at The Walt Disney Company

Job Description

Job Summary:

We have created a new Big Data Platforms group within Disney’s Direct-To-Consumer and International technology organization with the skills, drive, and passion to innovate, create, and succeed in enabling a Direct to Consumer Strategy for various digital products. We are here to disrupt and start a cultural revolution in the application of data and analytics across The Walt Disney Company, focused on Content Personalization/Recommendation, Deep User Understanding, and Audience Segmentation for Linear to Digital Ad Sales, and Analytics.

We need an experienced Data Platform Architect who can drive multiple data initiatives applying innovative architecture that can scale in the cloud. We’re looking for a creative and talented individual who loves designing scalable platforms, particularly at the peta-byte level and extract value from both structured and unstructured real-time data. More specifically, we need a technology leader to build a highly scalable and extensible Big Data platform which enables collection, storage, modeling, and analysis of massive data sets from numerous channels. You will also build a self-serve machine learning (and deep learning) pipeline for multiple data scientists to help develop, test, deploy, a/b test models on top of the data platform you are responsible for. You must be self-driven to continuously evaluate new technologies, innovate and deliver solutions for business-critical applications with little to no oversight from management team.

The platforms you design and build will be core assets in the delivering the highest quality content to over 150MM+ consumers on monthly basis. This is an opportunity to fundamentally evolve how DTCI delivers content and monetizes our audiences. Please note, this is a hands-on role with a great deal of time spent on building platforms and evangelizing the ideas through presentations and trainings.

Responsibilities:

Act as the proactive and technical architect point person for DTCI Consumer Data Platforms end to end (data collection through knowledge extraction via statistical machine learning and deep learning approaches, distribution via stream, APIs and files, ad-hoc analysis, reporting & visualization).
Present and educate management team about technical direction to achieve maximum profitability by using best data management technologies while reducing overall cost of operation.
Lead and coach other software engineers by developing re-usable frameworks. Review design and code produced by other engineers.
Provide expert level advice to data scientists, data engineers, and operations to deliver high quality analytics via machine learning and deep learning via data pipelines and APIs.
Lead the transformation of a peta-byte scale batch-based processing platform to a near real-time streaming platform using technologies such as Apache Kafka, Cassandra and Spark.
Design and build efficient ETL/ELT process to move data through the data processing pipeline to meet the demands of the business use cases using Java, Open Source, and AWS Products. Build easy to re-use workflow model and take the entire team to follow the pattern to implement all ETL process to improve efficiency and reduce cost.
Optimize and automate data ingestion, data processing and distribution data from variety of sources, including click stream data, ratings data, advertising data, 3rd party sources and sources not yet identified.
Manage complex data dependencies across datasets and incremental data loading workflows.
Design and build api/stream/batch based data export mechanism to be used by other DTCI products such as AdSales, Web, App platforms.
Be a fearless leader in championing smart, scalable and flexible design
Collaborating with product management and acting as the bridge between product management, engineering teams, and customers to understand requirements and technical solutions
Help us stay ahead of the curve by working closely with data management team, data engineers, our DevOps team, and analysts to design systems which can scale overnight in ways which make other groups envy

Basic Qualifications:

Roughly 10 years of professional experience building large scale data platforms from Architecture all the way to implementation and support. Platform is expected to handle Peta Bytes of data in cloud environment, on a real-time manner.
Code Ninja – Must be hands-on on latest technologies such as Java, Scala, Apache Spark, Apache Kafka, Hadoop, API design and development, No-SQL databases such as Cassandra, OLAP columnar storage systems, Bit Map indexes to handle millions of consumers and thousands of attributes while allowing real-time querying/ segmentation.
Visionary - Solid understanding of software development from design and architecture to build software for future.
Have a data toolbox – Familiar with technologies relevant to the data and integration space including Hadoop, Spark, Apache Druid, Cassandra, Java, Python, and ML frameworks.
Hunger to Learn & Teach – Genuine interest to learn new cutting edge technologies and share it with rest of the engineering team to keep them up-to-date on technology trends. We love to see your public GIT or similar profiles.
Problem solver – Enjoy new and meaningful technology or business challenges which require you to think and respond quickly
Passion and creativity – Are passionate about data, technology, & creative innovation
Open source – Prefer open source technologies and build it yourself mentality, open source contribution history is highly preferred.
Team player – Enjoy working collaboratively with a talented group of people to tackle challenging business problems so we all succeed (or fail fast) as a team

Preferred Qualifications:

Experience in building large data streaming platform will be a huge plus.
Experience in operationalizing Machine Learning workflows to scale will be a huge plus as well.
Experience with Content Personalization/Recommendation, Audience Segmentation for Linear to Digital Ad Sales, and/or Analytics
Presence in open source projects will be huge plus. We love to see our social profile if any.
Working experience in Machine Learning framework such as Apache Spark MLLib, TensorFlow or similar.
5+ years of hands-on experience in data and analytics technology, with focus on data architecture and large volume data processing. Experience with Java, Python, and/or SQL.
5+ years of experience building, coaching and leading software professionals
5+ years of experience working with relational databases, data services, big data, complex event processing and machine learning.
2+ years of experience with cloud deployments, AWS experience preferred. Proficiency with linux/unix based systems.

Required Education

Bachelor’s degree or better in Computer Science or a related technical field or equivalent job experience.

Preferred Education

Masters in Computer Science or similar is preferred.

About Direct-to-Consumer and International:

Comprised of Disney’s international media businesses and the Company’s various streaming services, the Direct-to-Consumer and International segment aligns technology, content and distribution platforms to expand the Company’s global footprint and deliver world-class, personalized entertainment experiences to consumers around the world. This segment is responsible for The Walt Disney Company’s direct-to-consumer businesses globally, including the ESPN+ sports streaming service, programmed in partnership with ESPN; the upcoming Disney-branded direct-to-consumer streaming service; and the Company’s ownership stake in Hulu. As part of the Direct-to-Consumer and International segment, Disney Streaming Services, developer of the ESPN+ and Disney-branded streaming platforms, oversees all consumer-facing digital technology and products across the Company.

About The Walt Disney Company:

The Walt Disney Company, together with its subsidiaries and affiliates, is a leading diversified international family entertainment and media enterprise with the following business segments: media networks, parks and resorts, studio entertainment, consumer products and interactive media. From humble beginnings as a cartoon studio in the 1920s to its preeminent name in the entertainment industry today, Disney proudly continues its legacy of creating world-class stories and experiences for every member of the family. Disney’s stories, characters and experiences reach consumers and guests from every corner of the globe. With operations in more than 40 countries, our employees and cast members work together to create entertainment experiences that are both universally and locally cherished.

This position is with ESPN Technology Services, Inc., which is part of a business segment we call Direct-to-Consumer and International.

ESPN Technology Services, Inc. is an equal opportunity employer. Applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status. Disney fosters a business culture where ideas and decisions from all people help us grow, innovate, create the best stories and be relevant in a rapidly changing world.

Interested in this position?

Click here to learn how to get in touch