Presentation: Osiris: When Big Data Is Too Big for HBase
Share this on:
Abstract
As the #1 job site in the world, Indeed delivers hundreds of millions of searches per day to job seekers. To give our users the best experience possible, we analyze petabytes of data per day for machine learning, A/B testing, and reporting.
Learn how the Search Quality team at Indeed developed Osiris, a horizontally scalable key-value store built on Hadoop. Osiris is flexible enough to be used in everything from big data analysis to latency-sensitive, user-facing applications.
This talk will cover the requirements and scaling challenges we faced that led to the development of Osiris. We'll also discuss the details of how we built Osiris, including its unique key design and highly configurable storage engine -- both of which allow for use in a wide variety of applications. We'll end this talk with specific examples of how Osiris is used at Indeed.
Tracks
-
Microservices/ Serverless: Patterns and Practices
Stories of success and failure building modern service and function-based applications, including event sourcing, reactive, decomposition, & more.
-
Distributed Stateful Systems
Architecting and leveraging NoSQL revisitied
-
Evolving Java and the JVM: Mobile, Micro and Modular
Although the Java language is holding strong as a developer favourite, new languages and paradigms are being embraced on JVM.
-
The Practice & Frontiers of AI
Learn about machine learning in practice and on the horizon
-
Operating Systems: LinuxKit, Unikernels, & Beyond
Applied, practical, & real-world deep-dive into industry adoption of OS, containers and virtualisation, including Linux on Windows, LinuxKit, and Unikernels
-
Stream Processing in the Modern Age
Compelling applications of stream processing & recent advances in the field
-
Leading Edge Backend Languages
Code the future! How cutting-edge programming languages and their more-established forerunners can help solve today and tomorrow’s server-side technical problems.
-
Modern CS in the Real World
Applied trends in Computer Science that are likely to affect Software Engineers today.
-
DevEx: The Next Evolution of DevOps
Removing friction from the developer experience.
-
Bare Knuckle Performance
Killing latency and getting the most out of your hardware
-
Tech Ethics in Action
Learning from the experiences of real-world companies driving technology decisions from ethics as much as technology.
-
Security: Red XOR Blue Team
Security from the defender's AND the attacker's point of view
-
Architecting for Failure
If you're not architecting for failure you're heading for failure
-
Architectures You've Always Wondered About
Topics like next-gen architecture mixed with applied use cases found in today's large-scale systems, self-driving cars, network routing, scale, robotics, cloud deployments, and more.
-
Observability: Logging, Alerting and Tracing
Observability in modern large distributed computer systems
-
Speaker AMAs (Ask Me Anything)
-
Building Great Engineering Cultures & Organizations
Stories of cultural change in organizations
-
Speaker AMAs (Ask Me Anything)