Expert Oracle Exadata - Parent Directory

CHAPTER 13 MIGRATING TO EXADATAHowever, on the source system, which is about to send large amounts of data, the default TCP buffersizes may become a bottleneck. You should add these lines to /etc/sysctl.conf if they’re not therealready:net.core.rmem_default = 262144net.core.rmem_max = 1048576net.core.wmem_default = 262144net.core.wmem_max = 4194304Then issue a sysctl –p command to apply these values into the running kernel.The maximum read and write buffer sizes per TCP connection will be set to 1MB and 4MBrespectively, but the default starting value for buffers is 256kB for each. Linux kernels 2.4.27 (and higher)and 2.6.17 (and higher) can automatically tune the actual buffer sizes from default values up to allowedmaximums during runtime. If these parameters are not set, Linux kernel (2.6.18) defaults to 128kB buffersizes, which may become a bottleneck when transferring large amounts of data. The write buffer hasbeen configured bigger as the source data would be sending (writing) the large amounts of data, theamount of data received will be much smaller.The optimal buffer sizes are dependent on your network roundtrip time (RTT) and the network linkmaximum throughput (or desired throughput, whichever is lower). The optimal buffer size value can becalculated using the Bandwidth*Delay product (BDP) formula, also explained in the “Enabling HighPerformance Data Transfers” document mentioned earlier in this section. Note that changing the kernelparameters shown earlier means a global change within the server, and if your database server has a lotof processes running, the memory usage may rise thanks to the increased buffer sizes. So you might notwant to increase these parameters until the actual migration happens.Here’s an example from another O/S type. On Solaris 8 and newer versions, the default send andreceive buffer size is 48KB. This would result in even poorer throughput compared to Linux’s default128KB. On Solaris you can check the max buffer size and max TCP congestion window with the followingcommands:$ ndd /dev/tcp tcp_max_buf1048576$ ndd /dev/tcp tcp_cwnd_max1048576This output shows that the maximum (send or receive) buffer size in the Solaris O/S is 1MB. Thiscan be changed with the ndd –set command, but you’ll need root privileges for that:# ndd - set/dev/tcp tcp_max_buf 4194304# ndd - set/dev/tcp tcp_cwnd_max 4194304This, however just sets the maximum TCP buffer sizes (and the TCP congestion window) perconnection, but not the default buffer sizes, which a new connection would actually get. The defaultbuffer sizes can be read this way:$ ndd /dev/tcp tcp_xmit_hiwat49152$ ndd /dev/tcp tcp_recv_hiwat49152Both the default send buffer (xmit means transmit) and receive buffer sizes are 48KB. To changethese defaults to 4MB each, we would run these commands:# ndd -set /dev/tcp tcp_recv_hiwat 4194304# ndd -set /dev/tcp tcp_xmit_hiwat 4194304432