hamming

126 CHAPTER 17Kaiser finally wrote for him a program which would, given the cutoff edge position wherever theexperimenter chose to put it, design the corresponding filter. The program: (1) designed the correspondingdifferentiating filter, (2) wrote the program to compute the smoothed output, and then (3) processed the datathrough this filter without any interference from the physicist.I later caught the physicist adjusting the cutoff edge for different parts of the energy data on the same run,and had to remind him there was such a thing as “degrees of freedom”, and what he was doing was nothonest data processing. I had much more trouble, once things were going well, to persuade him to get themost out of his expensive data, he should actually work in the square roots of the counts as they had equalvariances. But he finally saw the light and did so. He and Kaiser wrote a classic paper in the area, as itopened the door on a new range of things which could be done.My contribution? Mainly, first identifying the problem, next getting the right people together, thenmonitoring Kaiser to keep him straight on the fact filtering need not have exclusively to do with timesignals, and finally, reminding them of what they knew from statistics (or should have known and probablydid not).It seems to me from my experience this role is increasingly needed as people get to be more highlyspecialized and narrower and narrower in their knowledge. Someone has to keep the larger view and see toit things are done honestly. I think I came by this role from long a long education in the hands of JohnTukey, plus a good basic grounding in the form of the universal tool of Science, namely Mathematics. I willtalk in Chapter 23 about the nature of Mathematics.Most signal processing is indeed done on time signals. But most digital filters will probably be designedfor small, special purpose studies, not necessarily signals in time. This is where I ask for your futureattention. Suppose when you are in charge of things at the top, you are interested in some data which showspast records of relative expenses of manpower to equipment. It is bound to be noisy data, but you would liketo understand, in a basic sense, what is going on in the organization—what long term trends are happeningsoslowly people hardly sense them as they happen, but which never-the-less are fundamental to understandif you are to manage well. You will need a digital filter to smooth the data to get a glimpse of the trend, if itexists. You do not want to find a trend when it does not exist, but if it does you want to know pretty muchwhat it has been, so you can project what it is likely to be in the near future. Indeed, you might want toobserve, if the data will support it, any change in the slope of the trend. Some signals, such as the ratio offire power to tonnage of the corresponding ship, need not involve time at all, but will tell you somethingabout the current state of the Navy. You can, of course, also study the relationship as a function of time.I suggest strongly, at the top of your career you will be able to use a lot of low level digital filtering ofsignals, whether in time or not, so you will be better able to manage things. Hence, I claim, you willprobably design many more filters for such odd jobs than you will for radar data reduction and suchstandard things. It is usually in the new applications of knowledge where you can expect to find the greatestgains.Let me supply some warnings against the misuse of intellectual tools, and I will talk Chapter 27 on topicscloser to statistics than I have time for now. Fourier analysis implies linearity of the underlying model. Youcan use it on slightly nonlinear situations, but often elaborate Fourier analyses have failed because theunderlying phenomena was too nonlinear. I have seen millions of dollars go down that drain when it wasfairly obvious to the outsider the nonlinearities would vitiate all the linear analysis they could do using theFourier function approach. When this was pointed out to them, their reply seemed to be they did not knowwhat else to do, so they persisted in doing the wrong thing! I am not exaggerating here.How about nonlinear filters? The possibilities are endless, and must, of course, depend on the particularproblem you have on hand. I will take up only one, the running median filter. Given a set of data you