to /boot/kernel.pmap−crash. (It wasn't exactly a good kernel, but I wanted it on hand in case the new one was worse.) We then kicked PMAP_SHPGPERPROC up to 400, and increased the system's RAM to 192MB. (Yes, this cheap system was serving several hundred Web pages a second on 64MB of RAM, one IDE disk, and a Celeron 433!) After doing the config−make dance, the problem went away, and the server now has 30 days uptime.

Without the ability to tweak the kernel, we would have had no choice but to buy more hardware. Admittedly, this piece of hardware is pretty low−end. But if this hardware does the job with just a little software tweak, why not use it? If you're that desperate to spend money, send the checks to me.

Tweaking Kernel Performance

And how about improving performance?

The biggest kernel bottleneck is network mbufs. You'll see in Chapter 5 how mbufs are the chunks of memory that the kernel uses to handle network connections. They aren't the number of network connections the server can handle, but rather the memory used to hold network connections, and one connection might consume several mbufs. (You might want to read the discussion of mbufs in Chapter 5 before you start tuning them, but since we're discussing kernel configuration here, we'll discuss the mechanics now.)

The number of mbufs scales somewhat with the MAXUSERS kernel option discussed earlier in this chapter, but you will probably want to increase the setting on a high−production server. While the auto−scaling of MAXUSERS can help, this is still a very common tweak.

The NMBCLUSTERS option controls the number of mbufs created by the kernel. (This option won't appear in the GENERIC configuration file; you'll need to add it. NMBCLUSTERS does appear in the LINT file.)

...............................................................................................

options NMBCLUSTERS=1024

...............................................................................................

Network mbuf clusters are preallocated in kernel memory, so you can't just crank this value up to a million and forget about it, because that memory won't be available for other uses when the system gets busy. You do want your kernel to be able to open files and support your Web server, don't you?

One nmbcluster uses about 2KB of memory, so the preceding example reserves 2MB of memory for networking. (2 times 1024 is 2048, and 1MB is 1024KB.) This might not be much on a modern computer, but it is a considerable chunk on a 486s that can run FreeBSD. See why we want to customize this?

To calculate the number of mbuf clusters you need, first check how many network connections you have open at a fairly busy time. You can do this with the netstat(1) command. Netstat will show you how many network connections the system has, including TCP, UDP, loopback, and UNIX socket connections. All you need care about for mbufs clusters are TCP and UDP, so you can pull those

94

out with grep(1). Finally, you can use the wc(1) word−counting program to count the number of lines in the output, which gives you the number of TCP and UDP connections that the system is using right now.[7] Here are the commands:

...............................................................................................

#netstat −na | grep tcp | wc −l

427

#netstat −na | grep udp | wc −l

377

#

...............................................................................................

Note If you want to know how many network mbufs you're using at any given time, look at netstat −m. We'll discuss netstat in some detail in Chapter 5.

As you can see from the results, at this particular moment, the system has 427 running and available TCP network connections, and 377 active and available UDP network connections. This is roughly 800 total. To account for possible peaks, plan for twice the number of connections you see at a typical busy time.

Now that you know how many connections you have to handle, you need to know how much memory each connection requires. Each TCP connection requires a send buffer and a receive buffer. You can get their current size (in bits) from the sysctls net.inet.tcp.sendspace and net.inet.tcp.recvspace.

...............................................................................................

#sysctl net.inet.tcp.sendspace net.inet.tcp.sendspace: 16384

#sysctl net.inet.tcp.recvspace net.inet.tcp.recvspace: 16384

...............................................................................................

Bytes are difficult to work with, so we'll convert them to kilobytes; 16384 divided by 1024 is 16, so each buffer is 16KB on this system. (The default buffer size changed between FreeBSD 4.4 and 4.5, so you will want to check this on your system!) Since each network connection needs an incoming and an outgoing buffer, each TCP requires 32KB.

Similarly, each incoming UDP connection requires a buffer. You can't do much tuning with UDP, but assuming each UDP connection requires as much space as a TCP connection is reasonable for what we're doing here.

So, we know that each connection requires 32KB, and we know that our "average peak" usage is 800 connections. 800 x 32KB = 25600KB, or about 25MB. (1MB is actually 1024KB, but this is close enough for our purposes.) Then, to handle peaks and surges, double this to 50MB.

One mbuf cluster is 2KB, or 1024 mbuf clusters are 2MB, and we want 50MB of mbufs, so we multiply 50MB by 1024 and divide by 2 to get a total of 25600 mbuf clusters. So set the NMBCLUSTERS option to 25600 like so:

...............................................................................................

options NMBCLUSTERS=25600

95

...............................................................................................

Note If you're running a network server, it's a good idea to set NMBCLUSTERS to roughly a quarter of your physical RAM. 32MB of RAM set aside for mbufs, with 16KB send and receive buffers, gives you NMBCLUSTERS = 16384. This might not be adequate, or it might be too much, but it's a good place to start.

[7]For those of you who are newer UNIX administrators: Remember in the Introduction where we talked about UNIX commands being a language? Here's a good example. We have combined small commands to get a final answer without any tedious counting or searching through output ourselves. You might think that UNIX admins are extremely intelligent. Many of us are just creatively lazy.

Sharing Kernels

If you have several identical servers, you don't need to manually build a kernel on each; you can share your custom−built kernel across them. (The kernel file is just a binary, after all.)

To share a kernel, build and install one kernel and test it in every way you can think of. Then tar up /kernel and /modules and copy the tarball to each of the other servers. Back up the current kernel on each of the other servers, and decompress your tarball to install the new /kernel and /modules. Just reboot, and you're set.

96