QCon is a practitioner-driven conference designed for technical team leads, architects, and project managers who influence software innovation in their teams.
Nathan Marz, TweetCreator of the Storm and Cascalog open-source projects
Biography: Nathan Marz
Nathan Marz is the creator of many open source projects which are relied upon by over 50 companies around the world, including Cascalog and Storm. Nathan is also working on a book for Manning publications entitled "Big Data: principles and best practices of scalable realtime data systems".
Nathan was previously, he was the lead engineer at BackType before being acquired by Twitter in 2011. At Twitter, he started the streaming compute team which provides and develops shared infrastructure to support many critical realtime applications throughout the company.
Currently, Nathan is working on a new startup.
Twitter: @nathanmarz
Presentation: TweetA Call for Sanity in NoSQL
The techniques, algorithms, and technologies for managing data have progressed greatly since the advent of the relational database 40 years ago, yet applications are getting HARDER to build, not easier. NoSQL provides scalability at the cost of sanity.
The complexities we face as software engineers runs deep inside the industry. We debate whether or not to "normalize" or "denormalize" our data, not recognizing the fact that you have to make that tradeoff at all is a sign of something seriously wrong. We have become so accustomed to painful schema implementations that people consider "schemaless" to be a feature. We use eventually consistent databases that require extremely intricate read-repair algorithms in order to work properly. Finally, we build systems that fall apart due to the slightest human mistake. That anyone – and seemingly everyone – runs systems with any of these complexities is insane. Sanity is within your reach, but you need to work for it. You need to stop placing the relational database on a holy pedestal and instead think of data systems from first principles. It turns out the technology is already here in order to build data systems that are scalable AND easy to reason about. In this talk, you'll learn how.