A Practical Hardware Sizing Guide for Sybase IQ

More documents

Recommendations

Info

size of a page. As the page size increases, so does the block size. A data page of 128KB may compress to a single 8KB block. A 256KB page with the same data on it will consume 16KB on disk. Memory use won't increase or decrease as the page size is changed, per se, as the main and temporary caches are still the same size. If the memory that IQ can reference in its caches isn't increased, though, performance may suffer. A system with just a handful of users will, more than likely, not experience as many issues related to undersized memory as those environments with a lot of concurrent users. As concurrency increases, the number of pages that are in use will increase in both main and temporary cache. When moving to a larger page size (for instance, doubling from 128 KB page to 256 KB page) the number of pages that can be stored in the same amount of RAM is cut in half. If a user is referencing a fraction of the data on a page more pages will need to be brought in to handle the query. Should the amount of RAM dedicated to the caches not be increased, a burden will be added to the system that can force pages that are the least recently used to be flushed out to disk before the new data can be brought into RAM. To properly handle the same number of users with a doubled page size, it is possible that the IQ cache sizes need to be doubled in size. Should the page size be changed solely on the number of rows in the table, memory contention and a possible increase in disk activity can follow. Typically, systems with larger objects have more memory available and thus don‟t suffer from the impacts. If the RAM sizes don‟t change over time and the tables are expected to grow to the point where the rowcount sizing would force a larger page size, it is recommended that the page size increase be thought through as possibly disregarded. Versioning is one aspect that is often overlooked as it relates to page sizes. The number of pages needed to track versions won't change; it is the amount of cache or disk that would increase. If one were to double the page size, the amount of cache or storage needed for versions would also double. Final 36
Threads Startup Thread Allocation The default number of threads allocated in IQ during startup depends on two things only: the number of CPU cores and the number of user connections (-gm startup parameter). The total number of threads IQ allocates at startup can also be overridden by using the -iqmt startup option. By default, -iqmt is set to: 60*(min(numCores,4)) + 50*(numCores - 4) + 2*(numConnections + 2) + 1 On a 4 core system with -gm (numConnections) set to 20, that would be: iqmt = 60 * (4) + 50 * (4-4) + 2 * (20+2) + 1 iqmt = 285 On a 12 core system with -gm (numConnections) set to 50, that would be: iqmt = 60 * (4) + 50 * (12-4) + 2 * (50+2) + 1 iqmt = 745 There are two distinct types of threads: connection threads and server threads. They do not form one big pool for all uses. Connection threads come from the part of the default calculation based on number of connections the server is configured for: 2*(numConnections + 2). These are reserved for connections and will not be used as workers for query or load processing. This behavior can be observed by using the IQ Buffer Cache Monitor with the -threads or - debug option. When "Free Threads" equals "Reserved Threads" all that are left are those kept back for connection use with no threads left over for parallel operations. Server threads come from the part of the default calculation based on the number of CPU cores: 60*(min(numCores,4)) + 50*(numCores - 4). These threads are used in support of parallel loads or parallel query operations as well as those used for disk I/O. You must have free server threads to perform operations in parallel. When you specify a value for -iqmt, it is imperative that -iqmt be larger than the default number of threads. If a value lower than the default is specified, IQ will ignore it and start the server with “minimum threads” which provides ONLY the connection threads. Set -iqmt no lower than the default calculation as previously described. This allows IQ to keep a pool of threads for query and load work as well as for connections. There is an upper limit of 4096 threads for a 64-bit system. The total number of threads represented by -iqmt plus startup parameter -gn must not exceed this number. The value of –gn defaults to –gm + 5 where -gn is the number of threads used by the SQL Anywhere engine. Disk I/O Threads There are two teams of threads that perform disk I/O: sweeper and prefetchers. The sweepers write out dirty buffers and prefetchers read in buffers. The number of threads for each team is controlled by the following options: SWEEPER_THREADS_PERCENT Final 37
Page 1 and 2: A Practical Hardware Sizing Guide f
Page 3 and 4: Introduction Purpose This document
Page 5 and 6: that are undergoing constant data r
Page 7 and 8: CPUs CPUs for queries 0.1-1.5 CPUs
Page 9 and 10: � Temporary cache - This is set v
Page 11 and 12: A good starting point for setting M
Page 13 and 14: For fixed width data loads, load me
Page 15 and 16: 8,192 * 500 * N = 4,096,000 * N As
Page 17 and 18: memory sizes has been covered in de
Page 19 and 20: IQ 15 only allows 1 temp dbspace. T
Page 21 and 22: For each BLOB or CLOB being loaded
Page 23 and 24: Sizing Memory Data Loads and Change
Page 25 and 26: Sizing Storage IQ_SYSTEM_MAIN Sizin
Page 27 and 28: We could set this to 256K. IQ will
Page 29 and 30: As a rule of thumb, the total bandw
Page 31 and 32: Figure 3 - IQ Main Store Drive Mapp
Page 33 and 34: of backend servers. The number, con
Page 35: alance the page size, concurrent us
Page 39 and 40: those held back for connections are

A Practical Hardware Sizing Guide for Sybase IQ

Create successful ePaper yourself

Delete template?

Save as template?