Have you seen this video which went viral?
What fascinates me is that I remember upgrading from Win 3.1 to Win 95 and was blown away by how fast and smooth it was. The feeling of "this is it", this is fast enough to feel really productive. But sadly so often today's amazement dies to quickly becomes tomorrow's frustration. We become addicted to the buzz of doing more interesting things and doing them more quickly. When I started in DW in the 90's, we thought an analysts' attention spans were around 10 minutes: if complex TB scale queries came back in ten minutes they could keep their train of thought and go on asking interesting questions of the data. It wasn't long before 10 minutes was painful, we wanted one minute, then 10 seconds, then ...
At Hotsos 2016 in Dallas this week I was talking about flash performance. When Exadata X3 came out the performance was amazing. In some scenarios the flash cache could serve data faster than the Xeons could decompress ZLIB and for the first time LZO became a viable performance option for flash. With X5-EF, the NVMe flash subsystem can scan over 250 GB/sec dwarfing the QDR Infiniband links 40 GB/sec data return and consequently why having Smart Scan to cut the data down at the storage layer is now more essential than ever.
I look at 250+ Gb/sec and can't quite get my head around what that looks like as a processing rate - reading the entire Library of Congress in less then a minute? I wonder if a time will come when I will look at NVMe and think how on earth did with we live with the frustration? I don't know, but I do know that with X5 now and with future hardware options on the horizon, this is the most exciting time in my 30+ year career to be working at the heart of database performance architecture.
I'm going to be continuing this discussion with some of the performance findings from Hotsos: stay tuned.
Let me know if there are any Smart Scan performance topics you'd like to see posts on.