What are the characteristics of a good software engineer? It's a topic many people would argue endlessly about. This is not surprising given we are effectively living in the era of software alchemy. Some of the best programmers draw on strong scientific and engineering background and combine this with craft like coding skills in a virtuous feedback cycle. In this talk we explore the individual practices and techniques that can help bring out the engineer in you.

In this full-day workshop, you will learn applied big data solution patterns. most often, but not always using the public cloud.We'll cover Amazon Web Services and Google Cloud Platform, and work with in small groups to design solutions for common scenarios.

We'll discuss database categories, products and understand actual cost to use one or more database types in your solution(s). Using combinations of Big Relational (for hot, cold or cold data), and one or more types from the NoSQL category (Key-value, Wide-column, Document or Graph) and sometimes even along with NewSQL, we'll design implementable solutions. We'll also cover what types of problems are best suited to the Hadoop ecosystem (including use of Spark and the associated libraries).

We'll also look at product maturity (particularly surrounding cloud products such as AWS' Aurora, Athena and Google's Big Query and Big Table). Also we'll examine widely used partner products, such as ETL or visualization tools.

Finally, we'll look at data load-testing patterns and security practices, including considerations around connecting between different vendor clouds.