The Art of SQL Server FILESTREAM - Red Gate Software

Chapter 1: Storing and Managing Unstructured DataOne of the big advantages that the file system can offer, when handling LOB data, isstreaming. The streaming of data refers to transferring it in a manner that allows itto be processed as the data is transferred, rather than collecting all the data before theprocessing begins. Streaming significantly increases the performance of read and writeoperations on that data.SQL Server cannot stream LOB data. When a chunk of LOB data is written, SQL Servercreates a memory buffer large enough to hold the data, accepts all the data, and only thenstarts writing it into the database. Similarly, when LOB data is read, SQL Server createsa memory buffer large enough to hold the LOB value, reads all the data into the bufferand then sends it to the client connection that requested the data. So, to read or write a500 MB LOB value from or to a column, SQL Server needs to allocate 500 MB of memorybuffer to read or write the value.Exceptionally large objects, or multiple simultaneous requests to read or write LOBdata, can cause severe memory pressure and possibly result in the dreaded "insufficientmemory" error. Keep in mind that SQL Server is optimized for I/O operations involving8 KB pages, not massive objects. In any scenario, such queries can slow down otherqueries and affect overall server performance.A piece of Microsoft research which studied the performance differences between storingLOB data in the database and in the file system found that file system is the preferredoption for storing LOB data if the size is 1 MB or above.To BLOB or not to BLOB?You can find the Microsoft Research Paper, one of Jim Gray's last projects before he disappeared at sea, ToBLOB or Not To BLOB: Large Object Storage in a Database or a File System at http://brurl.com/fs2.36