Statistical mechanics of neocortical interactions - Lester Ingber's ...

More documents

Recommendations

Info

Statistical Mechanics of Neocortical ... - 30 - Lester Ingberable to bounce over any mountain to access any valley, giv en enough bounces. As the temperature ismade relatively colder, the ball cannot bounce so high, and it also can settle to become trapped inrelatively smaller ranges of valleys.Imagine that a mountain range is aptly described by a “cost function.” Define probability distributions ofthe two directional parameters, called generating distributions since they generate possible valleys orstates to explore. Define another distribution, called the acceptance distribution, which depends on thedifference of cost functions of the present generated valley to be explored and the last saved lowest valley.The acceptance distribution decides probabilistically whether to stay in a new lower valley or to bounceout of it. All the generating and acceptance distributions depend on temperatures.In 1984 [93], it was established that SA possessed a proof that, by carefully controlling the rates ofcooling of temperatures, it could statistically find the best minimum, e.g., the lowest valley of ourexample above. This was good news for people trying to solve hard problems which could not be solvedby other algorithms. The bad news was that the guarantee was only good if they were willing to run SAforever. In 1987, a method of fast annealing (FA) was developed [94], which permitted lowering thetemperature exponentially faster, thereby statistically guaranteeing that the minimum could be found insome finite time. However, that time still could be quite long. Shortly thereafter, Very Fast SimulatedReannealing (VFSR) was developed [25], now called Adaptive Simulated Annealing (ASA), which isexponentially faster than FA.ASA has been applied to many problems by many people in many disciplines [27,28,95]. The feedbackof many users regularly scrutinizing the source code ensures its soundness as it becomes more flexibleand powerful. The code is available via the world-wide web (WWW) as http://www.ingber.com/ whichalso can be accessed anonymous FTP from ftp.ingber.com.2. Mathematical OutlineASA considers a parameter α i k in dimension i generated at annealing-time k with the rangeα i k ∈[A i , B i ] ,(A.1)calculated with the random variable y i ,α i k+1 = α i k + y i (B i − A i ),
Statistical Mechanics of Neocortical ... - 31 - Lester Ingbery i ∈[−1, 1] .(A.2)The generating function g T (y) is defined,Πg T (y) = Di=112(|y i | + T i ) ln(1 + 1/T i ) ≡ Π DgT i (y i ) ,i=1(A.3)where the subscript i on T i specifies the parameter index, and the k-dependence in T i (k) for the annealingschedule has been dropped for brevity. Its cumulative probability distribution isy 1 y DG T (y) =∫ ... ∫ dy′1 ...dy′ D g T (y′) ≡Π DGT i (y i ),−1−1i=1G i T (y i ) = 1 2 + sgn (yi )2ln(1 + |y i |/T i )ln(1 + 1/T i ). (A.4)y i is generated from a u i from the uniform distributionu i ∈U[0, 1] ,y i = sgn (u i − 1 2 )T i[(1 + 1/T i ) |2ui −1| − 1] .(A.5)It is straightforward to calculate that for an annealing schedule for T iT i (k) = T 0i exp(−c i k 1/D ) ,(A.6)a global minima statistically can be obtained. I.e.,∞Σ g k ≈Σ ∞ [ Π D 1k 0 k 0 i=1 2|y i ] 1 |c i k = ∞ . (A.7)Control can be taken over c i , such thatT fi = T 0i exp(−m i ) when k f = exp n i ,c i = m i exp(−n i /D) ,(A.8)where m i and n i can be considered “free” parameters to help tune ASA for specific problems.
Page 1 and 2: Statistical mechanics of neocortica
Page 3 and 4: Statistical Mechanics of Neocortica
Page 55: Statistical Mechanics of Neocortica

Statistical mechanics of neocortical interactions - Lester Ingber's ...

Create successful ePaper yourself

Delete template?

Save as template?