Uber's vision is to make transportation as reliable as running water everywhere, for everyone. Data is key for Uber's 24x7 global business operations and making data available for different use cases across the company in a reliable, scalable and performant way is often challenging.

In this talk, we will discuss the overall data analytics eco-system at Uber and learn on how Uber shapes its data from a raw form to a modeled form by leveraging various in-house and open source technologies such as Hadoop, Hive on Tez/MR, Spark, Presto, Airflow and Enterprise technology such as HPE Vertica. Consumers of this data include Machine Learning & data science, city operations, Experimentation, Fraud, Marketplace and Growth Analytics.

We will also discuss on a whole different aspect of going back to basics on traditional data modeling and how it has helped us scale analytical and adhoc interactive queries while retaining the same standard SQL interface offered by SQL-on-Hadoop technologies like Hive, Presto and Spark. We will also discuss how we built and orchestrate ETL and Data processing pipelines leveraging Piper (forked from Airflow).

Finally, we will discuss couple of real time use cases of leveraging this framework and how this helped us power key business operations.