Asymptotically Optimal Low-Power Digital Filtering Using Adaptive Approximate Processing

: Techniques for reducing power consumption in digital circuits have become increasingly important because of the growing demand for portable multimedia devices. Digital ﬁlters, being ubiquitous in such devices, are a prime candidate for low-power design. We present a new algorithmic approach to low-power frequency-selective digital ﬁltering which is based on the concepts of adaptive approximate processing. This approach is formalized by introducing the class of approximate ﬁltering algorithms in which the order of a digital ﬁlter is dynamically varied to provide time-varying stopband attenuation in proportion to the time-varying signal-to-noise ratio (SNR) of the input signal, while maintaining a ﬁxed SNR at the ﬁlter output. Since power consumption in digital ﬁlter implementations is proportional to the order of the ﬁlter, dynamically varying the ﬁlter order is a strategy which may be used to conserve power. From this practical technique we abstract a theoretical problem which involves the determination of an optimal ﬁlter order based on observations of the input data and a set of concrete assumptions on the statistics of the input signal. Two solutions to this theoretical problem are presented, and the key results are used to interpret the solution to the practical low-power ﬁltering problem. We construct a framework to explore the statistical properties of approximate ﬁltering algorithms and show that under certain assumptions, the performance of approximate ﬁltering algorithms is asymptotically optimal. The theoretical and numerical simulation results demonstrate the efﬁcacy and versatility of asymptotically optimal low-power digital ﬁltering using adaptive approximate processing, and contribute to the evolution of low-power digital signal processing which continues to be an area of focused interest and innovation.


Introduction
Adaptive filtering and energy efficiency has been a topic of great interest in recent years. For example, the interest in reducing the power consumption of digital filters used in edge computing and sensor networks is growing rapidly [1]. A fundamental tradeoff between power consumption and accuracy has been utilized to determine energy-optimum design parameters for deep in-memory architectures for efficient hardware realizations of machine learning algorithms [2].
Remarkably, in a recent paper the authors note that "optimization for power is one of the most important design objectives in modern digital signal processing (DSP) applications," and then demonstrate the efficacy of a hybrid energy-efficient methodology for digital finite duration impulse response (FIR) filters [3].
Approximate signal processing may be used for applications in which it is desirable to dynamically adjust the quality of signal processing results to the availability of resources, such as time, bandwidth, memory, and power. Recently, excellent research has been accomplished in the area of incremental refinement structures for approximate signal processing in the context of power-efficient approximate multiplication for applications in which exact computation is not necessary [4][5][6][7][8]. The design of an energy efficient digital IIR filter using approximate multiplication, with a similar objective to the IIR filters presented in this paper, is presented in the research by Pilipović et al. [1]. Furthermore, approximate computing represents another recent innovation for minimizing power consumption via a variety of methods for reducing arithmetic activity [9][10][11][12].
The incessant demand for low-power signal processing algorithms has been persistent and consistent over the past three decades. The realm of energy efficiency has only grown in its intensity and breadth. In intensity, low-power speech recognition is now popular in mobile devices [13,14]. The evolution of energy efficient engineering has moved along stunningly creative paths to wearable devices [15], ingestible electronics [16,17] and cryptographic engines [18].
In this paper we consider the practical problem of dynamically reducing the order of a frequency-selective digital filter, omnipresent in all of the previously mentioned stateof-the-art signal processing electronic systems, to conserve power. We will demonstrate that it is possible to dynamically vary the stopband attenuation provided by a digital filter to obtain the minimum amount of attenuation needed to continuously maintain a given output signal-to-noise ratio (SNR), and show that such adaptive approximate filtering algorithms significantly reduce the required average power consumption relative to that of conventional fixed-order filtering algorithms. From this practical problem we abstract a theoretical problem which involves the determination of an optimal filter order based on observations of the input data and a set of concrete assumptions on the statistics of the input signal. Two solutions to this theoretical problem will be presented, and the key results will be used to interpret the solution to the practical low-power filtering problem.
An underlying theory for approximate filtering is developed. We construct a framework to explore the statistical properties of approximate filtering algorithms, and show that under certain assumptions the performance of approximate filtering algorithms is asymptotically optimal. The focus of the algorithm development is on applications involving frequencyselective digital filtering in which the goal is to reject one or more frequency bands while keeping the remaining portions of the input spectrum largely unaltered. Examples of such applications include lowpass filtering for signal upsampling and downsampling, bandpass filtering for subband coding, and lowpass filtering for frequency-division multiplexing and demultiplexing. In addition, approximate filtering algorithms appear to be useful in other domains in which digital filters are used such as prediction, smoothing, echo cancellation, or equalization.

Overview
A brief summary of approximate filtering is now given. The basic idea is to begin filtering a given input signal with a frequency-selective digital filter of some nominal order, as shown in Figure 1. This filter has well-defined passband and stopband regions in frequency. After L output samples have been produced, we use the most recent block of L input and output samples to form an easily computable low-power estimate of the current input SNR, defined as the ratio of the input signal power in the passband of the filter to the input signal power in the stopband of the filter. In Figure 1 the decision module D uses the signal power estimatesP x and P y to form an estimate of the temporally local input SNR. This estimate of the input SNR is then used to update the filter order to be the minimum value which guarantees that the output SNR, defined as the ratio of the output signal power in the passband of the filter to the output signal power in the stopband of the filter, will be greater than or equal to a pre-specified minimum tolerable output SNR. This filter order is then used to produce another block of L output samples, and the filter order update process is repeated. A key issue addressed in this paper is how well the low-power estimate of the filter order converges to the theoretical minimum order for situations satisfying certain statistical assumptions which are made in the derivation of the underlying theoretical framework for approximate filtering. Computer simulations are used to verify analytical results which we obtain in this paper that show that convergence to the correct filter order depends upon: 1) the number L of input and output samples used in estimating the input SNR, 2) the nominal order of the filter applied in generating the output samples that are used in estimating the input SNR, and 3) the proximity of the true input SNR to the boundaries in the input SNR space corresponding to changes in the optimal choice of filter order [19].
The adaptation mechanism used with an approximate filtering structure is designed to determine and use the filter with the smallest order while ensuring that the approximate filter output meets a pre-specified quality constraint. Minimization of the filter order used at any given time is desirable because of the resulting savings in power consumption by the underlying hardware [20]. The output quality criterion we use is designed to keep the output SNR (the ratio of the passband power to the stopband power in the filter output) above specified level. Other output quality constraints could easily be incorporated with minor modifications. One possible alternative output quality constraint is to keep the output signal power in the stopband of the filter below some pre-specified level [20].

Approximate Filter Structures
In approximate filtering algorithms the order of a frequencyselective digital filter is varied in a way defined by a control strategy and an approximate filter structure. A collection of frequency-selective digital filters, one for each filter order N in a given range N min < N < N max , constitutes an approximate filter structure H. Each filter structure H must possess the property that its progressively higher order filters have progressively increased average attenuation in the stopband region(s) while maintaining close to unity gain in the passband region(s). Approximate filter structures represent an important element in the characterization of approximate filtering algorithms.
Two classes of approximate filter structures, truncation and replacement filter structures, are important and used as a basis for classifying all approximate filter structures into one of four types [21]. A replacement filter structure is characterized by the relationship between the coefficients of filters of different orders being completely unconstrained; the coefficients of each individual filter may be selected or replaced independently. In a truncation filter structure this is not allowed. For a truncation filter structure with FIR constituent filter elements, the coefficients defining the lower order filters are constrained to be subsets of the coefficients defining the filter with maximum order N max . Similarly, in a truncation filter structure with IIR constituent elements, the pole/zero pairs defining the lower order filters are constrained to be subsets of the pole/zero pairs defining the filter with maximum order. Thus the lower order constituent elements in a truncation filter structure are truncated versions of the higher order constituent elements. It is clear that truncation filter structures may be described with fewer independent filter coefficients than replacement filter structures. Associated with this property is the fact that truncation filter structures require less memory, chip area, and bus accesses than replacement filter structures.
The frequency response magnitudes of the filters drawn from an exemplary approximate filter structure based on truncations of an IIR Butterworth filter are shown in Figure  2. The half-power frequency of the Butterworth filters is π/2. In this figure we show the magnitude-squared frequency responses for truncations of a 20th-order Butterworth filter with 3, 5, 7, 9, and 10 second-order sections. The key feature of an approximate filter structure is that the higherorder filters provide higher average stopband attenuation and thus have lower stopband power than the lower-order filters. This feature is clearly illustrated in Figure 2, and allows us incorporate a tradeoff between filter quality and filter cost into approximate filtering algorithms. The filter quality is measured by the average stopband attenuation, while the filter cost is measured by the required power consumption or equivalently the required filter order. The passband P B, stopband SB, and transition band T B regions for all filters in the approximate filter structure H are identical. The passband and stopband regions must be explicitly specified in the definition of an approximate filter structure, and by default the transition band is defined to span the remaining portions of the spectrum ω ∈ [−π, π] which are not included in the passband or stopband regions. Each of the individual filters which make up the constituent elements of the approximate filter structure H must be properly normalized. Possible normalizations include a unit energy normalization or a unity DC (zero frequency) gain normalization. We have stated our intention to study the problem of dynamically reducing the order of a frequency-selective digital filter to conserve power while maintaining a desired level of output quality. We have stressed that the key is use vary the filter order over time to provide time-varying stopband attenuation in proportion to the time-varying SNR of the input signal, while maintaining a fixed SNR at the filter output. From the practical problem of dynamically reducing the order of a frequency-selective digital filter to conserve power, in the next section we abstract and explore an intimately related theoretical problem. The solutions to this theoretical problem will provide a basis for understanding and analyzing approximate filtering algorithms.

Problem Statement
In this section we introduce a theoretical problem, termed the approximate filtering problem, which involves the determination of an optimal filter order based on observations of input data and a set of concrete assumptions on the statistics of the input signal. Two solutions to this theoretical problem are presented. One solution is guided by a low-power approach and achieves suboptimal performance with an extremely low computational cost. A second solution is guided by a maximum likelihood objective and provides superior performance while requiring much more computation. While computationally impractical, the maximum likelihood approach provides valuable insight as well as a performance benchmark for comparison with the lowpower solution. The key theoretical results are used to interpret the entire class of approximate filtering algorithms.
The fundamental theoretical problem we address in this paper is with power spectral density S x (ω) 2. a filter order set N = N min · · · N max containing M elements 3. a filter structure H = {h N min [n] · · · h N max [n]}, containing M frequency-selective filter elements, all with passband ω ∈ P B and stopband ω ∈ SB 4. a passband defined as ω ∈ P B for each filter in the filter structure H 5. a stopband defined as ω ∈ SB for each filter in the filter structure H 6. a minimum tolerable output signal-to-noise ratio OSNR tol Determine The optimal filter order N * , defined as the minimum order N ∈ N of the frequency-selective filter h N [n] ∈ H which provides sufficient stopband attenuation to assure where the output signal-to-noise ratio (SNR) is defined as The output power spectral density is the output power in the passband is and the output power in the stopband is We refer to the problem in (1)-(5), as the approximate filtering problem. Any particular solution to the problem involves defining a method for reliably estimating the optimal filter order N * based on observations of the input sequence We measure the performance of a particular solution in terms of how accurately, on average, the solution determines the correct value for N * . In this paper we will present two distinct solutions to the approximate filtering problem. The first is guided by a low-power consumption constraint and thus produces estimates of N * which require very low computational overhead. We call this the low-power (LP) solution to the approximate filtering problem. The second solution we present is motivated by a maximum likelihood formulation and is therefore termed the maximum likelihood (ML) solution to the approximate filtering problem. We will show that the ML solution requires much more computation and offers slightly better performance than the LP solution in certain circumstances. While computationally prohibitive for practical use, the ML solution is conceptually insightful as its performance may be used as a meaningful benchmark for comparison with the performance of the low-power solution.

Summary of Low-power Approach
The low-power (LP) approach to finding N * is computationally simple and will be shown to perform almost equally as well as the computationally prohibitive ML-based approach, even for signals statistically tailored to favor the ML-based approach.
A research group at Stanford University has designed and implemented an approximate filtering algorithm for the application of low-power interpolation and decimation. Excellent results have been obtained with a conventional interpolation/decimation system that was implemented and shown to consume an average of approximately 86.4 mW with a 5V power supply. Using an approximate filtering algorithm compiled onto a programmable processor, the power consumption was reduced by by 36% for the decimation system and 17% for the interpolation system [22].
The LP approach computes an estimate of the output SNR based on an estimate of the total power difference between the input and output of a digital frequency-selective filter. Conceptually, if the frequency-selective filter were an ideal piecewise-constant filter then the total power difference between the input and output signals would be equal to the exact input signal power in the stopband of the ideal filter. This is true since an ideal filter perfectly eliminates all the components of the input signal in its stopband, and passes with unity gain all the components of the input signal in its passband. Thus, ideally the output signal contains the exact passband components of the input signal, and nothing else. The input signal obviously contains both its passband and stopband signal components. Therefore the difference in input and output signal powers ideally gives the exact power in the stopband of the input signal. From this we may form a LP estimate O SNR LP [N, N 0 ] of the output SNR, and proceed to estimate Ideal filters are not practically realizable. However, if we use a non-ideal filter which approximates an ideal filter, our LP estimate of the output SNR based on the difference in input and output signal powers will approximate the true output SNR. To the extent that this is a good approximation, the LP approach to computing N * using (1) will be effective. The detailed derivation of the LP solution to the approximate filtering problem will be given in Section 3. A summary of the final result is presented now.
The LP estimateN * LP of the optimal filter order N * is determined by searching for the minimum value of N ∈ N satisfying where the LP estimate of the output SNR is defined as The expression in (8) may be rearranged with simple algebraic manipulations and substitutions to simplify the decision rule for selectingN * LP to be the minimum filter order N ∈ N satisfying where R is the ratio where the power window length L is the number of output samples which are produced by the filter h N0 [n] ∈ H beforeN * LP is computed. The nominal filter order N 0 may be chosen to be equal to any filter order N ∈ N ; however, as we shall see later, the choice N 0 = N max produces the best results.
The LP solution to the approximate filtering problem takes advantage of the fact that during filtering L samples of the output y[n] of the filter h N0 [n] are available, without additional computational cost. These samples are used to form the vector y and the ratio R. The function R tol [N ; N 0 , OSNR tol ] is a function of N assuming N 0 and OSNR tol have been fixed, and its values may be easily computed and stored in advance. The rule for computingN * LP is to search among the stored values of R tol [N ; N 0 , OSNR tol ] to find amongst all those which satisfy R tol [N ; N 0 , OSNR tol ] ≤ R the unique one which corresponds to the minimum value of N . This value of N is defined to beN * LP . We now consider a second solution to the approximate filtering problem, using an ML-based approach. This solution does not use the available output signal y[n] to compute its estimate of N * . In certain situations the ML solution will achieve better estimates of N * than the LP solution, but this performance advantage comes at the expense of requiring more computation.

Summary of Maximum Likelihood Approach
An alternative strategy for determining the optimal filter order N * defined in (1) involves computing an estimatê S x (ω) of the input power spectral density (PSD) S x (ω) , and using this PSD estimate to compute an estimate of the optimal filter order N * . Because the PSD estimateŜ x (ω) is based on ML estimates of the allpole parameters of the underlying random process which is assumed to be autoregressive (AR), we call this the ML approach. A summary of the final result of the ML approach to the approximate filtering problem is now presented, with the details of the derivation presented in Section 4.
Assuming that S x (ω) corresponds to a pth-order AR random process, the ML-based estimateŜx(ω) is defined aŝ where the ML-based estimatesâ = [â 1â2 · · ·â p ] T andσ 2 u for the parameters a = [ a 1 a 2 · · · a p ] T and σ 2 u are computed via the well-known Yule-Walker equations. This method for AR parameter estimation is well known as the autocorrelation method. The final result is that an ML estimateN * ML of N * may be computed via selecting the minimum value of N ∈ N satisfying where the ML estimate of the output SNR is defined as the ML estimate of the output power in the passband is defined as and the ML estimate of the output power in the stopband is defined as  (12). This approach will be shown to experimentally produce excellent estimates of N * , especially when the input signal is synthetically generated to be a true AR WSS random process. This is an intuitively natural result. Unfortunately the ML approach is not practically viable due to its excessive computational requirements. However it will serve as a meaningful benchmark for performance comparison with the LP approach which is the focal point of this paper.
Having now formally presented the statement of the approximate filtering problem as well as overviews of the two solutions which are developed in this paper, we move on to the detailed derivations of each solution. First we formulate the LP estimateN * LP in Section 3, and then we formulate the ML estimateN * ML in Section 4. The capabilities of this algorithm for reducing power consumption in digital filtering applications are truly remarkable.

Derivation of Low-power Solution
In this section we develop the low-power (LP) solution to the approximate filtering problem summarized in (1). The LP solution provides a method for computing the estimateN * LP of the optimal filter order N * based on low-power operations. The LP method is necessarily computationally simple, and thus it has the advantage of requiring significantly less average power than the ML-based solution which will be given in Section 4. After presenting some underlying assumptions, our approach to deriving an expression forN * LP begins with determining a LP estimate I SN R LP [N 0 ] of the input SNR based on the difference between the input power and the output power of a frequency-selective filter h N0 [n] ∈ H with nominal order N 0 ∈ N . We use our estimate I SN R LP [N 0 ] to produce an expression for a LP estimate OSNR LP [N, N 0 ] of the output SNR, which can be substituted into (1) for OSNR[N ] to determine the LP solutionN * LP to the approximate filtering problem.
To begin we suppose that a discrete-time WSS random process 1 x[n] with power spectrum S x (ω) is filtered using a digital frequency-selective filter with impulse response h N0 [n] ∈ H and order N 0 ∈ N to obtain an output signal y[n]. We assume that the filter h N0 [n] is taken from an approximate filter structure H and thus has a well-defined spectral passband P B, stopband SB, and transition band T B. We make the following key assumptions:

Assumption 2
For frequencies in the passband of H, |H N (ω)| 2 is approximately equal to unity This is assumed to be true for all N ∈ N , and thus for all Furthermore, this implies that for any finite, continuous, differentiable function f (ω). We note that no assumption is made about the shape of the function S x (ω) within the passband. S x (ω) in the passband is finite but otherwise arbitrary. Assumption 2 states that S x (ω) is negligible in the transition band. This is reasonable for situations in which the input stopband and passband components are separated by a guard band, as is the case in a whole host of communications applications [23].
As mentioned earlier, our first step is to determine a LP estimate of the input SNR under the stated assumptions. We define the signal-to-noise ratio (SNR) as the ratio of the signal power in the passband of H to the signal power in the stopband of H. The input SNR may be expressed as where and By invoking the Assumption 1 which states that S x (ω) is equal to an unknown constant σ 2 SB in the stopband, it follows where ∆ SB is the spectral width of the stopband. For example, if the stopband is defined as π/2 ≤ |ω| ≤ π, then ∆ SB = π. Assumption 2 states that |H N (ω) (23) becomes where P P B y [N 0 ] is the output power in the passband which was previously defined in (4). We note that the approximate expression in (24) for the input SNR is a function of N 0 due to its dependence on P P B y [N 0 ]. We now proceed to find an expression for P P B y [N 0 ]. First note that the total output power may be written as the sum of three spectrally-disjoint components where P P B y [N 0 ] is given in (4), P SB y [N 0 ] is given in (5), and If we now invoke the Assumption 3 which states that S x (ω) is negligible in the transition band, then P T B y [N 0 ] ≈ 0, and rearranging (26) produces We now examine the term P SB (5) with Assumption 1 which states that S x (ω) = σ 2 SB for ω ∈ SB, we obtain In order to obtain a more elementary expression for P SB y [N 0 ], it is apparent from (29) that an expression for the unknown parameter σ 2 SB is needed. For this purpose we consider the difference in input and output signal power where the total input signal power P x , is defined as As was shown for the total output power P y [N 0 ] in (26), the difference in input and output signal power may similarly be broken up into its spectrally disjoint passband, stopband, and transition band components We expand this and incorporate Assumption 1 to produce We now recall Assumption 2 which states that |H N0 (ω)| 2 ≈ 1 for ω ∈ P B. Under this assumption the first addend in the right-hand side of (33) is approximately zero. Furthermore, Assumption 3 which states that S x (ω) is negligible in the transition band, we may argue that the third addend in the right-hand side of (33) is approximately zero. We therefore obtain which may be rearranged to yield Substituting this approximate expression for σ 2 SB into (29) we obtain where is a particularly relevant measure of the spectral quality of the filter H N0 (ω). Note that P SB h [N 0 ] = 0 in the case of an ideal filter which was defined in (6). We now incorporate (28) into (24) and obtain Substituting in the approximate expression in (36) for Plugging in our approximate expression in (35) for σ 2 Armed with this expression for ISNR[N 0 ] which is valid under the stated assumptions, we now turn to the problem of computing low-power estimates of the signal-related quantities in (40): the total input power P x and the total output power P y [N 0 ].

Low-power Estimation
To obtain the LP estimate ISNR LP [N 0 ] of the input SNR based on (40), suppose that we have applied a filter of order N 0 to the input x[n] and have obtained L output samples prior to and including time n. We may then obtain the following estimatesP where the L × 1 signal vectors are defined as We note that the explicit dependence ofP y [N 0 ] andP x on n and L is omitted for notational simplicity. By incorporating the estimatesP y [N 0 ] andP x into (40) in place of P y [N 0 ] and P x , respectively, we obtain the LP estimate I SN R LP [N 0 ] for the input SNR This is our final expression for I SN R LP [N 0 ]. We observe that I SN R LP [N 0 ] is easily computable from the signaldependent quantities y T y and x T x and the integrals of the filter h N0 [n] that appear in (43).
In order to computeN * LP an estimate O SN R LP [N, N 0 ] of the output SNR is required. To proceed, we define SNRI[N ], the signal-to-noise ratio improvement factor, as the multiplicative factor by which the input SNR is multiplied by to obtain the output SNR. This will enable us to easily obtain our estimate O SN R LP [N, N 0 ] of the output SNR from our estimate I SN R LP [N 0 ] of the input SNR via a simple multiplication by SNRI[N ]. The SNR improvement factor is clearly a function of the filter order N ∈ N . The signal-tonoise ratio improvement factor satisfies the relationship which we may rearrange with substitution of the definitions for ISNR and OSNR[N ] from (20) and (2), respectively, to produce If we now invoke the Assumption 1 and Assumption 2 which were stated at the beginning of Section 3, our expression for SNRI[N ] reduces to Thus, the SNR improvement factor is inversely proportional to the power in the stopband of the filter H N (ω). We noted earlier in Section 2 that the frequency selective filters in H possess the property that as the filter order N increases, the average stopband attenuation also increases, and consequently the total stopband power decreases. Thus it is clear from (47) that the SNR improvement factor increases as the filter order N increases. The function SNRI[N ] is plotted vs. the filter order N for the Parks-McLellan FIR replacement filter structure in Figure 3 and for the Butterworth IIR truncation filter structure in Figure 4. In each case the stopband is defined as ω ∈ [5π/8, π]. These two filter structures are an essential component to low-power adaptive filter design [21]. As is clear from Figure 3 and Figure 4, the function SNRI[N ] monotonically increases with N . We refer to plots of the function SNRI[N ] as the performance profile for a given approximate filter structure H.  If the input SNR is relatively low we must select a relatively high filter order to obtain a sufficiently large SNR improvement factor to assure that the output SNR is maintained above the minimum tolerable level OSNR tol .
Conversely, when the input SNR is relatively high we will be able to select a relatively low filter order which will provide an SNR improvement factor that will assure OSNR[N ] ≥ OSNR tol .
To determine the LP solution to estimating N * , we replace the exact ISNR in (44) with the LP estimate ISNR LP [N 0 ] given in (43), to obtain which we may compare to OSNR tol to determine the low-power estimateN * LP for the optimal filter order N * as the minimum filter order N ∈ N satisfying The expression in (48) may be rearranged with simple algebraic manipulations and substitutions to produce By defining the ratio of quadratic forms in the above expression as and the function R tol [N ; N 0 , OSNR tol ] as the decision rule for selectingN * LP simplifies to choosing the minimum filter order N ∈ N satisfying The notation R tol [N ; N 0 , OSNR tol ] has been used to emphasize that R tol [N ; N 0 , OSNR tol ] is a function of the filter order N and is parameterized by the nominal filter order N 0 and the minimum tolerable output SNR OSNR tol . This enforces the fact that OSNR tol and N 0 are applicationspecific parameters which are to be fixed in advance, leaving R tol [N ; N 0 , OSNR tol ] a monotonic function of the single variable N . Note that the only signal-dependent quantity in (53) is the ratio of quadratic forms R, which was defined in (51). As a final note, if we desire to avoid the power hungry division involved in computing R, we may use an alternative form for the decision rule for selectingN * LP . The resulting simplified decision rule for selectingN * LP is to to choose the minimum filter order N ∈ H satisfying The low-power decision rule is now summarized. Summary of Method for DeterminingN * LP 1. Fix the values of the application-specific parameters N 0 , L, and OSNR tol 2. Compute R using (51) and the signal vectors x and y defined in (41) and (42), respectively 3. DetermineN * LP as the minimum value of N for which as described in (53) or (54) In summary, the LP solution to the approximate filtering problem invokes three explicit assumptions and relies on the signal-dependent estimatesP x andP y [N 0 ] given in (41) and (42), respectively. For situations in which the three assumptions are valid and in whichP x andP y [N 0 ] are good estimates of P x and P y [N 0 ], respectively, we expect excellent estimator performance using the LP estimateN * LP for N * . This issue is explored in the next section. The function SNRI[N ] and the nature of its dependence on N and H have been studied extensively [21].

Convergence Analysis
It is of interest to determine the degree to which the lowpower filter order estimateN * LP converges to the theoretically optimal filter order N * for input signals that satisfy the assumptions underlying the derivation ofN * LP . In this section, we illustrate empirically that better convergence is obtained as the duration L over whichP x andP y [N 0 ] are computed is made longer, and also as the nominal filter order N 0 is made larger. We also observe and discuss the fact that since optimal filter order selections partition the range of possible input SNR values, the relation of the actual input SNR to the boundaries in this partitioning is an important factor in determining whether or not the truly optimal filter order N * is exactly determined by the LP estimation method.
For our convergence analysis, we assume that the input signal satisfies the same conditions that were stipulated in the derivation of our expression forN * LP in Section 3. This means that we assume the input signal x[n] is a WSS random process. When L consecutive samples of the output y[n] are produced using a filter of order N 0 , it follows that these output samples also belong to a WSS random process. We conclude thatP x andP y [N 0 ] as defined in (41)  To verify the influence of the estimation interval L on the input SNR estimate, we applied the LP estimation method of (53) to a synthetically generated random signal x[n]. This signal was designed to have a flat spectrum in the passband |ω| ∈ [0, 3π/8], a flat spectrum in the stopband |ω| ∈ [5π/8, π], negligible energy in the transition band |ω| ∈ [3π/8, 5π/8], and a fixed SNR throughout its 10,000 point duration. The signal was filtered using an order-N 0 digital Butterworth filter. The L consecutive input and output samples (starting from the 1000th sample to avoid filter startup transient effects) were used to obtain the LP estimate ISNR LP [N 0 ] of the input SNR. For a case where the true SNR of the input signal x[n] was set to 0.07, in Figure 5 we show the LP estimates of the input SNR obtained for different values of L in the range 1 ≤ L ≤ 4000 and N 0 in the range 4 ≤ N 0 ≤ 10. It is clear from Figure 5 that as L and N 0 increase, the LP estimate of the input SNR visually converges to the true input SNR of 0.07. It should be noted that lower values of N 0 correspond to frequency response shapes which violate the underlying assumptions to a greater degree. We must keep in mind that unless the filter h N0 [n] is ideal as in (6), the estimate ISNR LP [N 0 ] will never truly converge to the true input SNR, no matter how large L is made. In Figure 5, we have also indicated the partitioning of the input SNR space in accordance with the corresponding optimal filter order N* which should be used to ensure a minimum tolerable output SNR of 1000. Except for very small values of L, it is seen that the LP estimate of the input SNR leads tô N * LP = N * . This result is dependent on the fact that an input SNR of .07 happens to fall near the middle of the input SNR space corresponding to N * = 4. For example, if the actual input SNR had been made equal to 0.04, the LP estimates of the input SNR may have crossed into the incorrect N * = 5 region for a larger set of values of L.
To close this section, we offer two remarks regarding the convergence properties of the LP optimal filter order estimatê N * LP . First, we note that This statement simply elucidates the fact that as N 0 → ∞, the filter h N0)[n] becomes ideal with no transition band. In the limiting case of (55) the power difference P x − P y [N 0 ] converges to the true value of P P B x , and consequently the ratio R = y T y/(x T x − y T y) converges to the true input SNR. While in practice we will never be able to realize the conditions of this limiting case, the result is nevertheless insightful. Secondly, we observe that under Assumption 1 which is the same expression we get for SNRI[N ] when we invoke the assumptions presented in the derivation of the LP estimation method. Remarkably, then, we may conclude that which implies that if we use sufficiently large values of L and N 0 to compute ISNR LP [N 0 ], then we can expect asymptotically optimal performance as the filter orders N ∈ N that we search over to computeN * LP increase without bound. A second note we make in closing is that by introducing the L × L convolution matrix we may express the vector y as and thus the expression for R in (51) simplifies to where the L × L matrices A and B satisfy In this formulation R has the form of a ratio of two quadratic forms in the random vector x. It is interesting to note that if the vector x is a multivariate Gaussian random vector, an extremely complicated nevertheless computable expression may be obtained for the variance of R as a function of the filter coefficients in H and the power window length L. Various forms of the variance of the random variable R may be found in the literature [24][25][26][27]. A future direction of this research is to analytically evaluate this variance and compare it to the sample variance computed in computer simulations. Furthermore, the problem of designing the filter h N [n] which appears in the matrix H[N ] to produce filter structures H which minimize the variance of R provides an exciting and challenging future avenue to pursue in the area of approximate filtering algorithms.
In addition, expressions for the mean and variance of x for a multivariate Gaussian random vector x with zero mean and covariance matrix Σ may be obtained in closed form [28]. Specifically, they are given by and While these expressions do not give the true mean and variance of the random variable R orN * LP , they do offer insight into the statistical properties of two signal-dependent quantities x involved in the simplified division-free decision rule for determininĝ N * LP given in (54), and thus are worth mentioning here.

Numerical Example
In this section we present a simple numerical example to demonstrate the efficacy of the LP solution to the approximate filtering problem. We first synthetically generated a random driving noise signal which consisted of independent and identically distributed samples distributed according to a unit variance, zero mean Gaussian probability density function (PDF). This driving noise sequence was then filtered with a 30th-order all-pole filter to create a WSS Gaussian random process. A plot of the PSD of this random process is shown in Figure 6. The all-pole filter parameters were selected to assure the power spectral density S x (ω) was negligible in the transition band, defined in this example to be 3π/8 ≤ |ω| ≤ 5π/8.
The exact all-pole parameters that were used in this example are given in a more elaborate research study [21]. The passband was defined to be 0 ≤ |ω| ≤ 3π/8 and the stopband was defined as 5π/8 ≤ |ω| ≤ π. As can be seen from the spectral shape of S x (ω), the input power spectral density is negligible in the transition band and relatively flat in the stopband in accordance with the assumptions underlying the development of the LP solution to the approximate filtering problem. Table 1. Summary of the results of the numerical example using the LP estimator in which the true value of N * = 10 and the true input SNR is 0.4831. The results are tabulated for power window lengths of L = 50, 100, 500, 1000, and 5000. In Table 1 we summarize the LP estimation results after 100 Monte Carlo trials were performed for each of the values of L = 50, 100, 500, 1000, and 5000. The sample mean and sample standard deviation (STD) are listed in Table 1. Clearly as L increases, the quality of our LP estimates N * LP and I SNR LP improve. This is evident from the fact that the sample standard deviations decrease as L increases. In addition, the low-power estimate I SNR LP of the input SNR converges towards the true value of 0.4831 as L increases. In this example the Parks-McLellan FIR replacement approximate filter structure was used with N 0 = N max = 64 and N min = 3.

LN
In Figure 7 we have plotted four histograms of the actual LP estimates of the input SNR for the same 100 Monte Carlo trials. Each histogram corresponds the estimates of the input SNR for different values of L. As L increases the estimates tighten up around their means. From the entries in Table 1 it is clear that while I SNR LP converges towards the true input SNR as L increases, the estimateN * LP does not coverage to N * in this example. This is a consequence of the fact that theoretically I SNR LP converges to ISNR as L and N 0 increase without bound, while the convergence ofN * LP to N * requires L, N 0 , and N max to increase without bound. We shall revisit this same numerical example in Section 4.2 and evaluate the performance of the ML solution for comparison to the results given here.

Derivation of Maximum Likelihood Solution
In this section we assume S x (ω) is the PSD of an autoregressive (AR) Gaussian random process, and we use a maximum likelihood (ML) objective to find estimates for the parameters defining this process. As we shall see the PSD S x (ω) is a function of these parameters, and this function is one-to-one (invertible). Therefore we may easily obtain the ML estimate [Ŝ x (ω)] ML by invoking the well-known invariance property of the ML estimator [29].
By inspecting (1) we observe that the optimal filter order N * is not in one-to-one correspondence with S x (ω). This is true since many different functionsŜ x (ω) could result in the same ratio of integrals which define OSNR[N ], and thus many different functionsŜ x (ω) could result in the same N * . Nevertheless, we shall still be able to find a maximum modified likelihood estimate, which we shall denoteN * ML , for the optimal filter order N * which is based on the maximum likelihood estimate [Ŝ x (ω)] ML . We will show that the estimatê N * ML , although not the estimate which truly maximizes the associated likelihood function, instead maximizes a modified likelihood function. Thus the performance ofN * ML will serve as a meaningful benchmark for comparison with the LP estimateN * LP of the optimal filter order which was presented in Section 3.

Maximum Likelihood Estimation
In this section we first present an expression for the asymptotic ML estimates of the AR parameters of a Gaussian random process. Directly following the insightful presentation in the textbook by Kay [29], we consider the random process generated as the output x[n] of a stable, causal all-pole filter excited at the input by a zero-mean white Gaussian noise sequence u[n]. The pth-order polynomial function A(z) is defined by the AR filter parameters [ a 1 a 2 · · · a p ] as If the all-pole filter H(z) is stable, the excitation noise sequence u[n] assures that the output x[n] is a WSS random process. The effect of the filter is to color the white noise sequence u[n]. The AR model is capable of producing a wide variety of PSD functions, depending on the choice of the AR filter parameters [ a 1 a 2 · · · a p ] and excitation noise variance σ 2 u . The problem is to estimate the parameters [ a 1 a 2 · · · a p ] and σ 2 u based on the observed data sequence [ x[0] · · · x[L − 1] ]. Once the parameter estimates [â 1â2 · · ·â p ] andσ 2 u are computed, the PSD is estimated aŝ We now given expressions for the asymptotic ML estimates for the parameters [ a 1 a 2 · · · a p ] and σ 2 u . First, the estimated autocorrelation function iŝ The set of equations to be solved for the asymptotic ML estimate of the AR filter parameters a is i=1 pr xx [k − l] = −r xx [k] k = 1, 2, · · · , p (71) which can be rewritten in matrix form as rxx [2] . . .
These are the well-known Yule-Walker equations, which may be recursively solved using the Levinson recursion algorithm [30]. What is left is to solve for the asymptotic ML estimate σ 2 u . The asymptotic ML estimate is given bŷ Thus, the asymptotic ML estimatesâ andσ 2 u for the parameters a and σ 2 u are given in (72) and (73), respectively. These estimates converge to the true ML estimates as L → ∞, and yield reasonable estimates for sufficiently large finite values L. We recall that our ML estimate of the PSD is This estimate may be used to determine an ML estimatê N * ML of the optimal filter order N * . This ML-based estimate is produced by choosingN * ML to be the minimum order N ∈ N of the frequency-selective filter h N [n] ∈ H which provides sufficient stopband attenuation to assure where the ML-based estimate of the output SNR is defined as the ML-based estimate of the output power in the passband is defined as and the ML-based estimate of the output power in the stopband is defined as Consequently, the estimateN * ML maximizes the modified likelihood function which is related to the true likelihood function as explained in the textbook by Kay [29].

Numerical Example
In this section we present the results of using the ML estimation method for determining N * on the same numerical example that was presented in Section 3.3. Numerical results are given to demonstrate the performance of the ML method. Recall from Section 3.3 that this example involves a random driving noise signal consisting of independent and identically distributed samples distributed according to a unit variance, zero mean Gaussian PDF. This driving noise signal was filtered with an 30th-order all-pole filter to create a WSS Gaussian random process with a PSD which was shown previously in Figure 6. In Table 2 we summarize the ML estimation results after 100 Monte Carlo trials were performed for each of the values of L = 50, 100, 500, 1000, and 5000. The sample mean and sample standard deviation (STD) are listed Table  2. Clearly as L increases, the quality of the ML optimal filter order estimateN * ML improves since its standard deviation decreases as L increases. In addition, the ML estimate of the optimal filter order converges towards the true value of N * = 10 as L becomes larger.
In the simulations we used the Yule-Walker equations and the Levinson recursion to solve for the ML AR coefficients which give the ML power spectrum estimate and thus the ML estimate of the optimal filter orderN * ML . While the Yule-Walker equations give the asymptotic ML estimates of the AR parameters which converge to the true ML estimates as L → ∞, it is well known that another method produces better estimates for finite data records [31]. This method of AR parameter estimation is known as the forward-backward least-squares method. Using the forward-backward leastsquares method to compute the AR parameter estimates would probably improve the performance of the estimatorN * ML . In our simulations here we used the Yule-Walker AR parameter estimates since we were guided by the mathematical optimality of the ML approach. Small errors in the AR parameter estimates should not have a significant effect on the estimator N * ML . This is true sinceN * ML is based on a ratio of integrals of the power spectrum. Small errors in the AR parameter estimates will be integrated out when computingN * ML . As a final note, we observe that in determining the ML estimates for the numerical example we assumed the order of the AR process was known to be equal to 30. This introduces an element of unfairness when comparing the performance of the ML and LP approaches. In practice the AR order would not be known exactly, and would thus have to be estimated. Furthermore, this example was specifically tailored for the ML approach, since the input signal was synthetically generated to represent a true AR random process. This provides a second reason why we would expect the ML estimates to outperform the LP estimates. Table 2. Summary of the results of the numerical example using the ML estimator in which the true value of N * = 10 and the true input SNR is 0.4831. The results are tabulated for power window lengths of L = 50, 100, 500, 1000, and 5000.

Experimental Design and Results of the Low-power Adaptive IIR Digital Filter
In this section we present computer simulations which show that significant power savings may be achieved when the order of an IIR digital filter is dynamically varied to provide timevarying stopband attenuation in proportion to the time-varying SNR of the input signal, while maintaining a fixed level of output quality. We highlight experiments involving speech signals to demonstrate the practical viability of the low-power adaptive IIR digital filter presented in this paper. As we shall see, significant reduction in power consumption over a fixedorder IIR digital filter is achieved in simulations involving the demultiplexing frequency-division multiplexed (FDM) speech signals.
We illustrate the potential of the low-power adaptive IIR digital filter to reduce power consumption in speech processing. We use a Butterworth truncation filter structure with 10 second-order sections. This approximate filter structure and the adaptation control strategy described in Section 3 was applied to two speech signals which had been frequency-division multiplexed. The power window length was chosen to be L = 100 and the minimum tolerable output SNR was set to 1,000. The IIR filters in the Butterworth truncation filter structure each had a half-power frequency of π/2. The stopband was defined to be between 5π/8 and π, while the passband was defined to be between 0 and 3π/8. One speech signal was spectrally centered in the passband region of the lowpass filter and the other was modulated into the stopband region of the lowpass filter. The sampling rate for each of the speech signal was 16,000 Hz. Figure 8 shows the speech signal in the passband, the speech signal in the stopband, and the evolution of the number of filter sections used by the approximate filtering technique. Since we are using cascades of second order subsections, the power consumption is directly proportional to the number of active subsections, which can be enabled or disabled with low overhead. Since we are calculating the model order over a window length L = 100, less than 1% overhead is needed for these calculations. Examination of the figure shows that as would be expected, the number of filter sections is large when the input SNR is small. Furthermore, the number of filter sections is small when the input SNR is high. If we compare the power consumption for in the low-power adaptive filter to the power consumption of a fixed IIR filter order need to handle the worst-case signal statistics, we see that a power reduction of approximately 63% is achieved in this simulation. This can be best understand with a visual inspection of Figure  8, where we can see the time-varying number of filter sections as a function of sample number ranges between 2 and 10, while the number of sections in a fixed-order IIR filter that would ensure we are able to maintain the minimum tolerable output SNR of 1,000 at all times is 8. For comparison of the of the power savings achieved by the adaptive digital filter presented in this paper to other results recently reported in the literature, we present the findings of three other experiments. First, a recent state-of-the-art digital FIR filter using data-driven clock gating and multibit flip-flops combined achieved 22% to 25% power reduction compared to that using a conventional design [3]. Secondly, in another very recent study, a novel design for an energyefficient IIR digital filter achieved nearly 63% reduction in energy with a negligible deviation of the frequency response from the standard implementation [1]. Finally, a low power reconfigurable FIR digital filter based on dual mode operation achieved power savings up to 37.97% in simulations using speech signal processing, similar to the simulations in this section [32]. These results demonstrate how low-power digital signal processing continues to be an area of focused interest and innovation.
The low-power (LP) approach to finding N * is computationally simple and viable for direct implementation in hardware for practical applications. A research group at Stanford University has designed and implemented an approximate filtering algorithm for the application of lowpower interpolation and decimation. Excellent results have been obtained with a conventional interpolation/decimation system that was implemented and shown to consume an average of approximately 86.4 mW with a 5V power supply. Using an approximate filtering algorithm compiled onto a programmable processor, the power consumption was reduced by 36% for the decimation system and 17% for the interpolation system [22].

Conclusion
We have considered the practical problem of dynamically reducing the order of a frequency-selective digital filter to reduce average power consumption, and presented the class of approximate filtering algorithms for which this is accomplished. Approximate filtering algorithms were developed by abstracting a theory from the practical lowpower filtering problem. The theory centered on the problem of determining an optimal filter order based on observations of the input data and a set of concrete assumptions on the statistics of the input signal. We explored the statistical properties of this theory, and showed that under certain assumptions the class of approximate filtering algorithms is asymptotically optimal. The theory served the purpose of aiding us in understanding interpreting the performance of approximate filtering algorithms.
The low-power approach to adaptively finding the optimal filter order is computationally simple and viable for direct implementation in hardware for practical applications in which power savings in the range of 17% to 35% have been achieved. The theoretical and numerical simulation results demonstrate the efficacy and versatility of asymptotically optimal lowpower digital filtering using adaptive approximate processing, and contribute to the evolution of low-power digital signal processing which continues to be an area of focused interest and innovation.