Danilo P. (Imperial College, London) Mandic,
Danilo P. Mandic,
Vanessa Su Lee (Shell EP Europe) Goh,
Vanessa Su Lee Goh
e.a.
John Wiley & Sons
e druk, 2009
9780470066355
Complex Valued Nonlinear Adaptive Filters
Noncircularity, Widely Linear and Neural Models
Specificaties
Gebonden, 344 blz.
|
Engels
John Wiley & Sons |
e druk, 2009
ISBN13: 9780470066355
Rubricering
Onderdeel van serie
Adaptive and Cognitive Dynamic Systems: Signal Processing, Learning, Communications and Control
Verwachte levertijd ongeveer 16 werkdagen
Samenvatting
This book was written in response to the growing demand for a text that provides a unified treatment of linear and nonlinear complex valued adaptive filters, and methods for the processing of general complex signals (circular and noncircular). It brings together adaptive filtering algorithms for feedforward (transversal) and feedback architectures and the recent developments in the statistics of complex variable, under the powerful frameworks of CR (Wirtinger) calculus and augmented complex statistics. This offers a number of theoretical performance gains, which is illustrated on both stochastic gradient algorithms, such as the augmented complex least mean square (ACLMS), and those based on Kalman filters. This work is supported by a number of simulations using synthetic and real world data, including the noncircular and intermittent radar and wind signals.
Specificaties
Inhoudsopgave
<p>Preface xiii</p>
<p>Acknowledgements xvii</p>
<p>1 The Magic of Complex Numbers 1</p>
<p>1.1 History of Complex Numbers 2</p>
<p>1.2 History of Mathematical Notation 8</p>
<p>1.3 Development of Complex Valued Adaptive Signal Processing 9</p>
<p>2 Why Signal Processing in the Complex Domain? 13</p>
<p>2.1 Some Examples of Complex Valued Signal Processing 13</p>
<p>2.2 Modelling in C is Not Only Convenient But Also Natural 19</p>
<p>2.3 Why Complex Modelling of Real Valued Processes? 20</p>
<p>2.4 Exploiting the Phase Information 23</p>
<p>2.5 Other Applications of Complex Domain Processing of Real Valued Signals 26</p>
<p>2.6 Additional Benefits of Complex Domain Processing 29</p>
<p>3 Adaptive Filtering Architectures 33</p>
<p>3.1 Linear and Nonlinear Stochastic Models 34</p>
<p>3.2 Linear and Nonlinear Adaptive Filtering Architectures 35</p>
<p>3.3 State Space Representation and Canonical Forms 39</p>
<p>4 Complex Nonlinear Activation Functions 43</p>
<p>4.1 Properties of Complex Functions 43</p>
<p>4.2 Universal Function Approximation 46</p>
<p>4.3 Nonlinear Activation Functions for Complex Neural Networks 48</p>
<p>4.4 Generalised Splitting Activation Functions (GSAF) 53</p>
<p>4.5 Summary: Choice of the Complex Activation Function 54</p>
<p>5 Elements of CR Calculus 55</p>
<p>5.1 Continuous Complex Functions 56</p>
<p>5.2 The Cauchy Riemann Equations 56</p>
<p>5.3 Generalised Derivatives of Functions of Complex Variable 57</p>
<p>5.4 CR–derivatives of Cost Functions 62</p>
<p>6 Complex Valued Adaptive Filters 69</p>
<p>6.1 Adaptive Filtering Configurations 70</p>
<p>6.2 The Complex Least Mean Square Algorithm 73</p>
<p>6.3 Nonlinear Feedforward Complex Adaptive Filters 80</p>
<p>6.4 Normalisation of Learning Algorithms 85</p>
<p>6.5 Performance of Feedforward Nonlinear Adaptive Filters 87</p>
<p>6.6 Summary: Choice of a Nonlinear Adaptive Filter 89</p>
<p>7 Adaptive Filters with Feedback 91</p>
<p>7.1 Training of IIR Adaptive Filters 92</p>
<p>7.2 Nonlinear Adaptive IIR Filters: Recurrent Perceptron 97</p>
<p>7.3 Training of Recurrent Neural Networks 99</p>
<p>7.4 Simulation Examples 102</p>
<p>8 Filters with an Adaptive Stepsize 107</p>
<p>8.1 Benveniste Type Variable Stepsize Algorithms 108</p>
<p>8.2 Complex Valued GNGD Algorithms 110</p>
<p>8.3 Simulation Examples 113</p>
<p>9 Filters with an Adaptive Amplitude of Nonlinearity 119</p>
<p>9.1 Dynamical Range Reduction 119</p>
<p>9.2 FIR Adaptive Filters with an Adaptive Nonlinearity 121</p>
<p>9.3 Recurrent Neural Networks with Trainable Amplitude of Activation Functions 122</p>
<p>9.4 Simulation Results 124</p>
<p>10 Data–reusing Algorithms for Complex Valued Adaptive Filters 129</p>
<p>10.1 The Data–reusing Complex Valued Least Mean Square (DRCLMS) Algorithm 129</p>
<p>10.2 Data–reusing Complex Nonlinear Adaptive Filters 131</p>
<p>10.3 Data–reusing Algorithms for Complex RNNs 134</p>
<p>11 Complex Mappings and M¨obius Transformations 137</p>
<p>11.1 Matrix Representation of a Complex Number 137</p>
<p>11.2 The M¨obius Transformation 140</p>
<p>11.3 Activation Functions and M¨obius Transformations 142</p>
<p>11.4 All–pass Systems as M¨obius Transformations 146</p>
<p>11.5 Fractional Delay Filters 147</p>
<p>12 Augmented Complex Statistics 151</p>
<p>12.1 Complex Random Variables (CRV) 152</p>
<p>12.2 Complex Circular Random Variables 158</p>
<p>12.3 Complex Signals 159</p>
<p>12.4 Second–order Characterisation of Complex Signals 161</p>
<p>13 Widely Linear Estimation and Augmented CLMS (ACLMS) 169</p>
<p>13.1 Minimum Mean Square Error (MMSE) Estimation in C 169</p>
<p>13.2 Complex White Noise 172</p>
<p>13.3 Autoregressive Modelling in C 173</p>
<p>13.4 The Augmented Complex LMS (ACLMS) Algorithm 175</p>
<p>13.5 Adaptive Prediction Based on ACLMS 178</p>
<p>14 Duality Between Complex Valued and Real Valued Filters 183</p>
<p>14.1 A Dual Channel Real Valued Adaptive Filter 184</p>
<p>14.2 Duality Between Real and Complex Valued Filters 186</p>
<p>14.3 Simulations 188</p>
<p>15 Widely Linear Filters with Feedback 191</p>
<p>15.1 The Widely Linear ARMA (WL–ARMA) Model 192</p>
<p>15.2 Widely Linear Adaptive Filters with Feedback 192</p>
<p>15.4 The Augmented Kalman Filter Algorithm for RNNs 198</p>
<p>15.5 Augmented Complex Unscented Kalman Filter (ACUKF) 200</p>
<p>15.6 Simulation Examples 203</p>
<p>16 Collaborative Adaptive Filtering 207</p>
<p>16.1 Parametric Signal Modality Characterisation 207</p>
<p>16.2 Standard Hybrid Filtering in R 209</p>
<p>16.3 Tracking the Linear/Nonlinear Nature of Complex Valued Signals 210</p>
<p>16.4 Split vs Fully Complex Signal Natures 214</p>
<p>16.5 Online Assessment of the Nature of Wind Signal 216</p>
<p>16.6 Collaborative Filters for General Complex Signals 217</p>
<p>17 Adaptive Filtering Based on EMD 221</p>
<p>17.1 The Empirical Mode Decomposition Algorithm 222</p>
<p>17.2 Complex Extensions of Empirical Mode Decomposition 226</p>
<p>17.3 Addressing the Problem of Uniqueness 230</p>
<p>17.4 Applications of Complex Extensions of EMD 230</p>
<p>18 Validation of Complex Representations Is This Worthwhile? 233</p>
<p>18.1 Signal Modality Characterisation in R 234</p>
<p>18.2 Testing for the Validity of Complex Representation 239</p>
<p>18.3 Quantifying Benefits of Complex Valued Representation 243</p>
<p>Appendix A: Some Distinctive Properties of Calculus in C 245</p>
<p>Appendix B: Liouville′s Theorem 251</p>
<p>Appendix C: Hypercomplex and Clifford Algebras 253</p>
<p>Appendix D: Real Valued Activation Functions 257</p>
<p>Appendix E: Elementary Transcendental Functions (ETF) 259</p>
<p>Appendix F: The O Notation and Standard Vector and Matrix Differentiation 263</p>
<p>Appendix G: Notions From Learning Theory 265</p>
<p>Appendix H: Notions from Approximation Theory 269</p>
<p>Appendix I: Terminology Used in the Field of Neural Networks 273</p>
<p>Appendix J: Complex Valued Pipelined Recurrent Neural Network (CPRNN) 275</p>
<p>Appendix K: Gradient Adaptive Step Size (GASS) Algorithms in R 279</p>
<p>Appendix L: Derivation of Partial Derivatives from Chapter 8 283</p>
<p>Appendix M: A Posteriori Learning 287</p>
<p>Appendix N: Notions from Stability Theory 291</p>
<p>Appendix O: Linear Relaxation 293</p>
<p>Appendix P: Contraction Mappings, Fixed Point Iteration and Fractals 299</p>
<p>References 309</p>
<p>Index 321</p>
<p>Acknowledgements xvii</p>
<p>1 The Magic of Complex Numbers 1</p>
<p>1.1 History of Complex Numbers 2</p>
<p>1.2 History of Mathematical Notation 8</p>
<p>1.3 Development of Complex Valued Adaptive Signal Processing 9</p>
<p>2 Why Signal Processing in the Complex Domain? 13</p>
<p>2.1 Some Examples of Complex Valued Signal Processing 13</p>
<p>2.2 Modelling in C is Not Only Convenient But Also Natural 19</p>
<p>2.3 Why Complex Modelling of Real Valued Processes? 20</p>
<p>2.4 Exploiting the Phase Information 23</p>
<p>2.5 Other Applications of Complex Domain Processing of Real Valued Signals 26</p>
<p>2.6 Additional Benefits of Complex Domain Processing 29</p>
<p>3 Adaptive Filtering Architectures 33</p>
<p>3.1 Linear and Nonlinear Stochastic Models 34</p>
<p>3.2 Linear and Nonlinear Adaptive Filtering Architectures 35</p>
<p>3.3 State Space Representation and Canonical Forms 39</p>
<p>4 Complex Nonlinear Activation Functions 43</p>
<p>4.1 Properties of Complex Functions 43</p>
<p>4.2 Universal Function Approximation 46</p>
<p>4.3 Nonlinear Activation Functions for Complex Neural Networks 48</p>
<p>4.4 Generalised Splitting Activation Functions (GSAF) 53</p>
<p>4.5 Summary: Choice of the Complex Activation Function 54</p>
<p>5 Elements of CR Calculus 55</p>
<p>5.1 Continuous Complex Functions 56</p>
<p>5.2 The Cauchy Riemann Equations 56</p>
<p>5.3 Generalised Derivatives of Functions of Complex Variable 57</p>
<p>5.4 CR–derivatives of Cost Functions 62</p>
<p>6 Complex Valued Adaptive Filters 69</p>
<p>6.1 Adaptive Filtering Configurations 70</p>
<p>6.2 The Complex Least Mean Square Algorithm 73</p>
<p>6.3 Nonlinear Feedforward Complex Adaptive Filters 80</p>
<p>6.4 Normalisation of Learning Algorithms 85</p>
<p>6.5 Performance of Feedforward Nonlinear Adaptive Filters 87</p>
<p>6.6 Summary: Choice of a Nonlinear Adaptive Filter 89</p>
<p>7 Adaptive Filters with Feedback 91</p>
<p>7.1 Training of IIR Adaptive Filters 92</p>
<p>7.2 Nonlinear Adaptive IIR Filters: Recurrent Perceptron 97</p>
<p>7.3 Training of Recurrent Neural Networks 99</p>
<p>7.4 Simulation Examples 102</p>
<p>8 Filters with an Adaptive Stepsize 107</p>
<p>8.1 Benveniste Type Variable Stepsize Algorithms 108</p>
<p>8.2 Complex Valued GNGD Algorithms 110</p>
<p>8.3 Simulation Examples 113</p>
<p>9 Filters with an Adaptive Amplitude of Nonlinearity 119</p>
<p>9.1 Dynamical Range Reduction 119</p>
<p>9.2 FIR Adaptive Filters with an Adaptive Nonlinearity 121</p>
<p>9.3 Recurrent Neural Networks with Trainable Amplitude of Activation Functions 122</p>
<p>9.4 Simulation Results 124</p>
<p>10 Data–reusing Algorithms for Complex Valued Adaptive Filters 129</p>
<p>10.1 The Data–reusing Complex Valued Least Mean Square (DRCLMS) Algorithm 129</p>
<p>10.2 Data–reusing Complex Nonlinear Adaptive Filters 131</p>
<p>10.3 Data–reusing Algorithms for Complex RNNs 134</p>
<p>11 Complex Mappings and M¨obius Transformations 137</p>
<p>11.1 Matrix Representation of a Complex Number 137</p>
<p>11.2 The M¨obius Transformation 140</p>
<p>11.3 Activation Functions and M¨obius Transformations 142</p>
<p>11.4 All–pass Systems as M¨obius Transformations 146</p>
<p>11.5 Fractional Delay Filters 147</p>
<p>12 Augmented Complex Statistics 151</p>
<p>12.1 Complex Random Variables (CRV) 152</p>
<p>12.2 Complex Circular Random Variables 158</p>
<p>12.3 Complex Signals 159</p>
<p>12.4 Second–order Characterisation of Complex Signals 161</p>
<p>13 Widely Linear Estimation and Augmented CLMS (ACLMS) 169</p>
<p>13.1 Minimum Mean Square Error (MMSE) Estimation in C 169</p>
<p>13.2 Complex White Noise 172</p>
<p>13.3 Autoregressive Modelling in C 173</p>
<p>13.4 The Augmented Complex LMS (ACLMS) Algorithm 175</p>
<p>13.5 Adaptive Prediction Based on ACLMS 178</p>
<p>14 Duality Between Complex Valued and Real Valued Filters 183</p>
<p>14.1 A Dual Channel Real Valued Adaptive Filter 184</p>
<p>14.2 Duality Between Real and Complex Valued Filters 186</p>
<p>14.3 Simulations 188</p>
<p>15 Widely Linear Filters with Feedback 191</p>
<p>15.1 The Widely Linear ARMA (WL–ARMA) Model 192</p>
<p>15.2 Widely Linear Adaptive Filters with Feedback 192</p>
<p>15.4 The Augmented Kalman Filter Algorithm for RNNs 198</p>
<p>15.5 Augmented Complex Unscented Kalman Filter (ACUKF) 200</p>
<p>15.6 Simulation Examples 203</p>
<p>16 Collaborative Adaptive Filtering 207</p>
<p>16.1 Parametric Signal Modality Characterisation 207</p>
<p>16.2 Standard Hybrid Filtering in R 209</p>
<p>16.3 Tracking the Linear/Nonlinear Nature of Complex Valued Signals 210</p>
<p>16.4 Split vs Fully Complex Signal Natures 214</p>
<p>16.5 Online Assessment of the Nature of Wind Signal 216</p>
<p>16.6 Collaborative Filters for General Complex Signals 217</p>
<p>17 Adaptive Filtering Based on EMD 221</p>
<p>17.1 The Empirical Mode Decomposition Algorithm 222</p>
<p>17.2 Complex Extensions of Empirical Mode Decomposition 226</p>
<p>17.3 Addressing the Problem of Uniqueness 230</p>
<p>17.4 Applications of Complex Extensions of EMD 230</p>
<p>18 Validation of Complex Representations Is This Worthwhile? 233</p>
<p>18.1 Signal Modality Characterisation in R 234</p>
<p>18.2 Testing for the Validity of Complex Representation 239</p>
<p>18.3 Quantifying Benefits of Complex Valued Representation 243</p>
<p>Appendix A: Some Distinctive Properties of Calculus in C 245</p>
<p>Appendix B: Liouville′s Theorem 251</p>
<p>Appendix C: Hypercomplex and Clifford Algebras 253</p>
<p>Appendix D: Real Valued Activation Functions 257</p>
<p>Appendix E: Elementary Transcendental Functions (ETF) 259</p>
<p>Appendix F: The O Notation and Standard Vector and Matrix Differentiation 263</p>
<p>Appendix G: Notions From Learning Theory 265</p>
<p>Appendix H: Notions from Approximation Theory 269</p>
<p>Appendix I: Terminology Used in the Field of Neural Networks 273</p>
<p>Appendix J: Complex Valued Pipelined Recurrent Neural Network (CPRNN) 275</p>
<p>Appendix K: Gradient Adaptive Step Size (GASS) Algorithms in R 279</p>
<p>Appendix L: Derivation of Partial Derivatives from Chapter 8 283</p>
<p>Appendix M: A Posteriori Learning 287</p>
<p>Appendix N: Notions from Stability Theory 291</p>
<p>Appendix O: Linear Relaxation 293</p>
<p>Appendix P: Contraction Mappings, Fixed Point Iteration and Fractals 299</p>
<p>References 309</p>
<p>Index 321</p>