QCon is a practitioner-driven conference designed for technical team leads, architects, and project managers who influence software innovation in their teams.

Nathan Marz, Creator of the Storm and Cascalog open-source projects

Nathan Marz

Biography: Nathan Marz

Nathan Marz is the creator of many open source projects which are relied upon by over 50 companies around the world, including Cascalog and Storm. Nathan is also working on a book for Manning publications entitled "Big Data: principles and best practices of scalable realtime data systems".

Nathan was previously, he was the lead engineer at BackType before being acquired by Twitter in 2011. At Twitter, he started the streaming compute team which provides and develops shared infrastructure to support many critical realtime applications throughout the company.

Currently, Nathan is working on a new startup.

Twitter: @nathanmarz

Presentation: A Call for Sanity in NoSQL

Track: Big Data Architectures - handling too big, too fast or too versatile data / Time: Friday 14:30 - 15:20 / Location: Churchill Auditorium

The techniques, algorithms, and technologies for managing data have progressed greatly since the advent of the relational database 40 years ago, yet applications are getting HARDER to build, not easier. NoSQL provides scalability at the cost of sanity.

The complexities we face as software engineers runs deep inside the industry. We debate whether or not to "normalize" or "denormalize" our data, not recognizing the fact that you have to make that tradeoff at all is a sign of something seriously wrong. We have become so accustomed to painful schema implementations that people consider "schemaless" to be a feature. We use eventually consistent databases that require extremely intricate read-repair algorithms in order to work properly. Finally, we build systems that fall apart due to the slightest human mistake. That anyone – and seemingly everyone – runs systems with any of these complexities is insane. Sanity is within your reach, but you need to work for it. You need to stop placing the relational database on a holy pedestal and instead think of data systems from first principles. It turns out the technology is already here in order to build data systems that are scalable AND easy to reason about. In this talk, you'll learn how.