Hopfield网络
Hopfield网络由美国加州理工学院物理学教授J. J. Hopfield于1982年提出[1]
网络从输出到输入有反馈连接，在输入的激励下，会产生不断的状态变化，是一种单层反馈神经网络，也可以被视为一种循环神经网络
Hopfield神经网络是反馈网络中最简单且应用广泛的模型，它具有联想记忆的功能，是神经网络发展历史上的一个重要的里程碑
根据网络的输出是离散量或是连续量，Hopfield神经网络大致分为两种：离散Hopfield网络（Discrete Hopfield Neural Network, DHNN）和连续Hopfield网络（Continuous Hopfield Neural Network, CHNN）
网络大致结构如下图所示：
离散Hopfield网络可以视为一个类脑模型，主要是因为其可用于联想记忆，即联想存储器，这是类人智能的特点之一
人类的所谓“触景生情”就是见到一些类同于过去接触的景物，容易产生对过去情景的回味和思忆
对于Hopfield 网络，用它作联想记忆时，首先通过一个学习训练的过程确定网络中的权系数，使所记忆的信息在网络的n维超立方体的某一个顶角达到能量最小
当网络的权重矩阵确定之后，只要向网络给出输入向量，这个向量可能是局部数据．即不完全或部分不正确的数据，但是网络仍然能够产生所记忆信息的完整输出
1984年，J. Hopfield设计并研制了网络模型的电路，并成功地解决了旅行商计算难题（TSP快速寻优问题）
现今，Hopfield网络主要在联想和识别方面有一些应用，如带噪声点阵符号识别和模糊或缺失数字图像的复原等
一些更加具体的介绍、更多的公式推导及应用示例可参见Ref.[2][3][4]等
Python代码实现
本文所介绍的是无自反馈的且算法改进后离散Hopfield网络
该改进算法根据Hebb归一化学习原则，并采用了Kronecker积的方法完成了实现（不幸的是解释文档已丢失）
但从算法的实测性能来看，远胜于NeuPy、NeuroLab等模块包中的Hopfield网络方法
以下为所编写的算法代码和示例DEMO：
【Hopfield.py】
#!/usr/bin/env python
#-*- coding: utf-8 -*-
'''
Hopfield Improved Algorithm
@Author: Alex Pan
@From: CASIA
@Date: 2017.03
'''
import numpy as np
################################### Global Parameters ###################################
# Data Type
uintType = np.uint8
floatType = np.float32
################################### Global Parameters ###################################
# Hopfield Class
class HOP(object):
def __init__(self, N):
# Bit Dimension
self.N = N
# Weight Matrix
self.W = np.zeros((N, N), dtype = floatType)
# Calculate Kronecker Square Product of [factor] itself OR use np.kron()
def kroneckerSquareProduct(self, factor):
ksProduct = np.zeros((self.N, self.N), dtype = floatType)
# Calculate
for i in xrange(0, self.N):
ksProduct[i] = factor[i] * factor
return ksProduct
# Training a single stableState once a time, mainly to train [W]
def trainOnce(self, inputArray):
# Learn with normalization
mean = float(inputArray.sum()) / inputArray.shape[0]
self.W = self.W + self.kroneckerSquareProduct(inputArray - mean) / (self.N * self.N) / mean / (1 - mean)
# Erase diagonal self-weight
index = range(0, self.N)
self.W[index, index] = 0.
# Overall training function
def hopTrain(self, stableStateList):
# Preprocess List to Array type
stableState = np.asarray(stableStateList, dtype = uintType)
# Exception
if np.amin(stableState) < 0 or np.amax(stableState) > 1:
print 'Vector Range ERROR!'
return
# Train
if len(stableState.shape) == 1 and stableState.shape[0] == self.N:
print 'stableState count: 1'
self.trainOnce(stableState)
elif len(stableState.shape) == 2 and stableState.shape[1] == self.N:
print 'stableState count: ' + str(stableState.shape[0])
for i in xrange(0, stableState.shape[0]):
self.trainOnce(stableState[i])
else:
print 'SS Dimension ERROR! Training Aborted.'
return
print 'Hopfield Training Complete.'
# Run HOP to output
def hopRun(self, inputList):
# Preprocess List to Array type
inputArray = np.asarray(inputList, dtype = floatType)
# Exception
if len(inputArray.shape) != 1 or inputArray.shape[0] != self.N:
print 'Input Dimension ERROR! Runing Aborted.'
return
# Run
matrix = np.tile(inputArray, (self.N, 1))
matrix = self.W * matrix
ouputArray = matrix.sum(1)
# Normalize
m = float(np.amin(ouputArray))
M = float(np.amax(ouputArray))
ouputArray = (ouputArray - m) / (M - m)
# Binary
''' \SWITCH/ : 1/-1 OR 1/0
ouputArray[ouputArray < 0.5] = -1.
''' # \Division/
ouputArray[ouputArray < 0.5] = 0.
# ''' # \END/
ouputArray[ouputArray > 0] = 1.
return np.asarray(ouputArray, dtype = uintType)
# Reset HOP to initialized state
def hopReset(self):
# Weight Matrix RESET
self.W = np.zeros((self.N, self.N), dtype = floatType)
# Utility routine for printing the input vector: [NperGroup] numbers each piece
def printFormat(vector, NperGroup):
string = ''
for index in xrange(len(vector)):
if index % NperGroup == 0:
''' \SWITCH/ : Single-Row OR Multi-Row
string += ' '
''' # \Division/
string += '\n'
# ''' # \END/
# ''' \SWITCH/ : Image-Matrix OR Raw-String
if str(vector[index]) == '0':
string += ' '
elif str(vector[index]) == '1':
string += '*'
else:
string += str(vector[index])
''' # \Division/
string += str(vector[index])
# ''' # \END/
string += '\n'
print string
# DEMO of Hopfield Net
def HOP_demo():
zero = [0, 1, 1, 1, 0,
1, 0, 0, 0, 1,
1, 0, 0, 0, 1,
1, 0, 0, 0, 1,
1, 0, 0, 0, 1,
0, 1, 1, 1, 0]
one = [0, 1, 1, 0, 0,
0, 0, 1, 0, 0,
0, 0, 1, 0, 0,
0, 0, 1, 0, 0,
0, 0, 1, 0, 0,
0, 0, 1, 0, 0]
two = [1, 1, 1, 0, 0,
0, 0, 0, 1, 0,
0, 0, 0, 1, 0,
0, 1, 1, 0, 0,
1, 0, 0, 0, 0,
1, 1, 1, 1, 1]
hop = HOP(5 * 6)
hop.hopTrain([zero, one, two])
half_zero = [0, 1, 1, 1, 0,
1, 0, 0, 0, 1,
1, 0, 0, 0, 1,
0, 0, 0, 0, 0,
0, 0, 0, 0, 0,
0, 0, 0, 0, 0]
print 'Half-Zero:'
printFormat(half_zero, 5)
result = hop.hopRun(half_zero)
print 'Recovered:'
printFormat(result, 5)
half_two = [0, 0, 0, 0, 0,
0, 0, 0, 0, 0,
0, 0, 0, 0, 0,
0, 1, 1, 0, 0,
1, 0, 0, 0, 0,
1, 1, 1, 1, 1]
print 'Half-Two:'
printFormat(half_two, 5)
result = hop.hopRun(half_two)
print 'Recovered:'
printFormat(result, 5)
half_two = [1, 1, 1, 0, 0,
0, 0, 0, 1, 0,
0, 0, 0, 1, 0,
0, 0, 0, 0, 0,
0, 0, 0, 0, 0,
0, 0, 0, 0, 0]
print 'Another Half-Two:'
printFormat(half_two, 5)
result = hop.hopRun(half_two)
print 'Recovered:'
printFormat(result, 5)
##########################
if __name__ == '__main__':
HOP_demo()Reference
[1] J. J. Hopfield, “Neural networks and physical systems with emergent collective computational abilities”, Proceedings of the National Academy of Sciences of the USA, vol. 79 no. 8 pp. 2554–2558, April 1982
