Conference:March 6-8, 2017
Workshops:March 9-10, 2017
Presentation: SQL Server On Linux: Will It Perform Or Not?
Location:
- Windsor, 5th flr.
Duration
Day of week:
- Wednesday
Level:
- Advanced
Persona:
- Architect
Key Takeaways
- Understand the history and architecture of how SQL Server ported to Linux, and answer the question of “will it perform.”
- Dive into the core of the I/O Manager, Memory Manager, Scheduler, and how addressing between the Windows User Space and Linux was managed.
- Understand how Microsoft Research’s Drawbridge was used as a platform abstraction layer between Windows and the Linux kernel.
Abstract
Will SQL Server perform on Linux better than on Windows? Have you been wondering whether the multi-layer architecture the team revealed recently will hurt SQL Server’s performance? Are you still not convinced about the entire endeavor. Come, listen to the talk, learn about SQL Server’s platform agnostic architecture then derive the answers for yourself. The talk will briefly go over history of the project, high level architecture and dive fast into core of I/O Manager, Memory Manager, and Scheduler. To make the talk more fun, every topic will include lessons learned and experiences behind the scenes.
Interview
Two years ago there was a business decision to put SQL Server on Linux; so, as the engineering team, we have to figure out the “How” part. The task is technically challenging: tens of millions of lines of code that continues growing on a monthly basis, with over 500 developers working on it. The Drawbridge project, which I became familiar with while working at Microsoft Research on the Midori operating system, allows Windows applications to be compartmentalized and run in a wrapper environment.
Although, initially, Drawbridge was designed to allow high density applications to run in the cloud, and to provide security for applications running there, it turned out it could also be used as a platform abstraction layer, PAL, to bring Windows native applications to other platforms. Drawbridge was used as a key part to make SQL Server run on Linux.
It will start with a review of the technologies used in bringing SQL Server to Linux. Quickly, we will go over Drawbridge’s architecture. Then we will briefly describe how SQL Server leverages operating system services. The talk will follow with tying the two pieces together and will introduce a new platform abstraction layer, SQLPAL. The talk will dive deep into the implementation details of SQLPAL’s I/O subsystems, memory management, and scheduling with a focus on how we minimize the SQLPAL overhead.
LibOS is an operating system library that executes in user more. It is a key part of the Drawbridge system. In terms of implementation, LibOS is a modified Windows NT kernel without a few key OS services such as page-table mappings, interrupt handling, scheduling and others. LibOS is a native Windows binary. When executing, LibOS requires a small set of Application Binary Interfaces, ABIs, provided by a host either through a library or a kernel driver. ABIs provide to LibOS support for such services as memory management, threads, streams and synchronization primitives.
Our goal is to make SQL Server perform and scale its best for every customer scenario on any platform of customer choice. We use a number of popular macro benchmarks and customer workloads to track both performance and scalability on different types of hardware. In addition, we employ a number of micro benchmarks. The micro benchmarks help us to get better understanding of the system as well as catch potential regressions faster. Obviously, today, performance varies depending on the scenarios as well as hardware set up. However, we are seeing very promising results. Even though we haven’t finished yet we are getting closer and closer to achieving our goal.
We’d like to share the hardcore part of the technology that allows us to run SQL Server on Linux. We will spend quite a bit of time going over the details of the problems we have been solving. As a side effect, we hope to show that Microsoft Research enables us to provide very elegant solutions to very complicated problems.
Developers who have experience in systems programming or who are interested in user mode kernel runtimes will appreciate this talk. This talk is not about database programming. It’s about how to combine runtimes and what optimizations can be used to make systems go faster.
Similar Talks
Tracks
-
Architecting for Failure
Building fault tolerate systems that are truly resilient
-
Architectures You've Always Wondered about
QCon classic track. You know the names. Hear their lessons and challenges.
-
Modern Distributed Architectures
Migrating, deploying, and realizing modern cloud architecture.
-
Fast & Furious: Ad Serving, Finance, & Performance
Learn some of the tips and technicals of high speed, low latency systems in Ad Serving and Finance
-
Java - Performance, Patterns and Predictions
Skills embracing the evolution of Java (multi-core, cloud, modularity) and reenforcing core platform fundamentals (performance, concurrency, ubiquity).
-
Performance Mythbusting
Performance myths that need busting and the tools & techniques to get there
-
Dark Code: The Legacy/Tech Debt Dilemma
How do you evolve your code and modernize your architecture when you're stuck with part legacy code and technical debt? Lessons from the trenches.
-
Modern Learning Systems
Real world use of the latest machine learning technologies in production environments
-
Practical Cryptography & Blockchains: Beyond the Hype
Looking past the hype of blockchain technologies, alternate title: Weaselfree Cryptography & Blockchain
-
Applied JavaScript - Atomic Applications and APIs
Angular, React, Electron, Node: The hottest trends and techniques in the JavaScript space
-
Containers - State Of The Art
What is the state of the art, what's next, & other interesting questions on containers.
-
Observability Done Right: Automating Insight & Software Telemetry
Tools, practices, and methods to know what your system is doing
-
Data Engineering : Where the Rubber meets the Road in Data Science
Science does not imply engineering. Engineering tools and techniques for Data Scientists
-
Modern CS in the Real World
Applied, practical, & real-world dive into industry adoption of modern CS ideas
-
Workhorse Languages, Not Called Java
Workhorse languages not called Java.
-
Security: Lessons Learned From Being Pwned
How Attackers Think. Penetration testing techniques, exploits, toolsets, and skills of software hackers
-
Engineering Culture @{{cool_company}}
Culture, Organization Structure, Modern Agile War Stories
-
Softskills: Essential Skills for Developers
Skills for the developer in the workplace
Conference for Professional Software Developers
