728x90
09-1 순차 데이터와 순환 신경망¶
순환 신경망¶
☆ 순차 데이터(sequential data)는 텍스트나 시계열 데이터와 같이 순서에 의미가 있는 데이터
☆ 완전 연결 신경망이나 합성곱 신경망은 하나의 샘플(또는 하나의 배치)을 사용하여 정방향 계산을 수행하고 나면 그 샘플은 버려지고 다음 샘플을 처리할 때 재사용하지 않습니다.
☆ 입렵 데이터의 흐름이 앞으로만 전달되는 신경망을 피드포워드 신경망(feedforward neural network)이라 한다.
☆ 순환 신경망(RNN)은 완전 연결 신경망 + 이전 데이터의 흐름을 순환하는 고리, 이전 타임스텝의 샘플을 기억하지만 오래될수록 순환되는 정보는 희미해 진다.
☆ 순환 신경망(RNN)은 순환층을 1개 이상 사용한 신경망
☆ 타임스텝(timestep) : 샘플을 처리하는 한 단계
☆ 순환 신명망에서는 특별히 층을 셀(cell)이라고 하며 셀의 출력을 은닉 상태라고 부른다.
☆ 은닉층의 활성화 함수로는 하이퍼볼릭 탄젠트 함수인 tanh가 많이 사용된다.
☆ 순환 신경망은 기존 신경망과 동일하며 추가적으로 입력과 이전 타임스텝의 은닉 상태를 사용하여 현재 타임스템의 은닉 상태를 만든다.
셀의 가중치와 입출력¶
☆ 순환층은 일반적으로 샘플마다 2개의 차원을 가진다.
☆ 하나의 샘플은 시퀀스 길이와 단어 표현의 2차원 배열이다.
☆ 순환층을 통과하면 차원이 사라지고 순환층의 뉴런 개수만큼 출력된다.
☆ 순환 신경망의 출력층 : 밀집층을 두어 클래스를 분류(다중 분류 : 소프트맥스 함수, 이진 분류 : 시그모이드 함수)
☆ 순환 신경망의 마지막 셀의 출력은 1차원이므로 Flatten으로 펼칠 필요 없다.
☆ 마지막 타임스템의 출력만 다음 층으로 전달
09-2 순환 신경망으로 IMDB 리뷰 분류하기¶
IMDB 리뷰 데이터셋¶
☆ IMDB 리뷰 데이터셋은 유명한 인터넷 영화 데이터베이스인 imdb.com에서 수집한 리뷰를 감상평에 따라 긍정과 부정으로 분류해 놓은 데이터 셋입니다.
☆ 자연어 처리(NLP) : 컴퓨터를 사용해 인간의 언어를 처리하는 분야
☆ 분리된 단어를 토큰(token)이라고 부르며 하나의 샘플은 여러 개의 토큰으로 이루어져 있다.
☆ 토큰에 할당한느 정수 중에 몇 개는 특정한 용도로 예약되어 있는 경우가 많다. 예를 들어 0은 패딩, 1은 문장의 시작, 2는 어휘 사전에 없는 토큰 등등
In [1]:
# IMDB에서 자주 등장하는 단어 500개만 추출
from tensorflow.keras.datasets import imdb
(train_input, train_target), (test_input, test_target) = imdb.load_data(
num_words=500)
In [2]:
# 25,000개의 긍정과 부정으로 이루어진 데이터 셋
print(train_input.shape, test_input.shape)
(25000,) (25000,)
In [3]:
# 첫 번째 리뷰의 길이는 218개의 토큰으로 이루어져 있다.
print(len(train_input[0]))
218
In [4]:
# num_word를 500으로 지정했기 때문에 어휘 사전에 없는 단어는 모두 2로 표시
print(train_input[0])
[1, 14, 22, 16, 43, 2, 2, 2, 2, 65, 458, 2, 66, 2, 4, 173, 36, 256, 5, 25, 100, 43, 2, 112, 50, 2, 2, 9, 35, 480, 284, 5, 150, 4, 172, 112, 167, 2, 336, 385, 39, 4, 172, 2, 2, 17, 2, 38, 13, 447, 4, 192, 50, 16, 6, 147, 2, 19, 14, 22, 4, 2, 2, 469, 4, 22, 71, 87, 12, 16, 43, 2, 38, 76, 15, 13, 2, 4, 22, 17, 2, 17, 12, 16, 2, 18, 2, 5, 62, 386, 12, 8, 316, 8, 106, 5, 4, 2, 2, 16, 480, 66, 2, 33, 4, 130, 12, 16, 38, 2, 5, 25, 124, 51, 36, 135, 48, 25, 2, 33, 6, 22, 12, 215, 28, 77, 52, 5, 14, 407, 16, 82, 2, 8, 4, 107, 117, 2, 15, 256, 4, 2, 7, 2, 5, 2, 36, 71, 43, 2, 476, 26, 400, 317, 46, 7, 4, 2, 2, 13, 104, 88, 4, 381, 15, 297, 98, 32, 2, 56, 26, 141, 6, 194, 2, 18, 4, 226, 22, 21, 134, 476, 26, 480, 5, 144, 30, 2, 18, 51, 36, 28, 224, 92, 25, 104, 4, 226, 65, 16, 38, 2, 88, 12, 16, 283, 5, 16, 2, 113, 103, 32, 15, 16, 2, 19, 178, 32]
In [5]:
# 0(부정), 1(긍정)으로 이진 분류되어 있다.
print(train_target[:20])
[1 0 0 1 0 0 1 0 1 0 1 0 0 0 0 0 1 1 0 1]
In [6]:
# 데이터 분류
from sklearn.model_selection import train_test_split
train_input, val_input, train_target, val_target = train_test_split(
train_input, train_target, test_size=0.2, random_state=42)
In [7]:
# 훈련 데이터 길이 파악 위한 코드
import numpy as np
lengths = np.array([len(x) for x in train_input])
In [9]:
# 길이에 대한 히스토그램
import matplotlib.pyplot as plt
plt.hist(lengths)
plt.xlabel('length')
plt.ylabel('frequency')
plt.show()
☆ pad_sequences() : 특정 길이가 되도록 패딩 시켜주는 함수
☆ maxlen에 원하는 길이를 지정하면 이보다 긴 경우는 잘라내고 짧은 경우는 0으로 패딩한다.
☆ maxlen보다 긴 시퀀스의 앞부분을 자른다. 이렇게 하는 이유는 시퀀스의 뒷부분의 정보가 더 유용하리라 기대하기 때문
☆ pad_sequences() 함수의 padding 매개변수의 기본값인 'pre'를 'post'로 바꾸면 샘플의 뒷부분에 패딩 추가할 수 있다.
In [10]:
# 길이가 100내외로 분포가 많으므로 100으로 패딩한다.
from tensorflow.keras.preprocessing.sequence import pad_sequences
train_seq = pad_sequences(train_input, maxlen=100)
# 100으로 패딩된 함수
print(train_seq.shape)
(20000, 100)
In [11]:
# 100의 길이가 되지 않아 0으로 채워졌음 확인할 수 있다.
print(train_seq[5])
[ 0 0 0 0 1 2 195 19 49 2 2 190 4 2 352 2 183 10
10 13 82 79 4 2 36 71 269 8 2 25 19 49 7 4 2 2
2 2 2 10 10 48 25 40 2 11 2 2 40 2 2 5 4 2
2 95 14 238 56 129 2 10 10 21 2 94 364 352 2 2 11 190
24 484 2 7 94 205 405 10 10 87 2 34 49 2 7 2 2 2
2 2 290 2 46 48 64 18 4 2]
In [12]:
# 검증 데이터도 동일하게 적용
val_seq = pad_sequences(val_input, maxlen=100)
순환 신경망 만들기¶
☆ SimpleRNN : 가장 기본적인 순환층 클래스
☆ SimpleRNN 클래스의 activation 매개변수의 기본값은 'tanh'로 하이퍼볼릭 탄젠트 함수 사용
In [13]:
from tensorflow import keras
model = keras.Sequential()
model.add(keras.layers.SimpleRNN(8, input_shape=(100, 500))) # 100인 이유는 앞에서 100으로 패딩했기 때문
# 500개 중에 하나만 1이고 나머지는 0으로 만들어 정수 사이에 있던 크기 속성을 없애는 원-핫 인코딩을 사용
# 500은 처음 임포트할 때 단어의 개수를 500으로 지정했기에 500 사용
model.add(keras.layers.Dense(1, activation='sigmoid'))
In [14]:
# 원-핫 인코딩 배열 반환 함수
train_oh = keras.utils.to_categorical(train_seq)
In [15]:
# 정수마다 500차원의 배열로 변경되었다.
print(train_oh.shape)
(20000, 100, 500)
In [16]:
# 특정 값 하나만 빼고 0이므로 모두 더하면 1이다.
print(np.sum(train_oh[0][0]))
1.0
In [17]:
# 검증 데이터도 동일하게 적용
val_oh = keras.utils.to_categorical(val_seq)
In [18]:
# 모델 구조 출력
model.summary()
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
simple_rnn (SimpleRNN) (None, 8) 4072
_________________________________________________________________
dense (Dense) (None, 1) 9
=================================================================
Total params: 4,081
Trainable params: 4,081
Non-trainable params: 0
_________________________________________________________________
순환 신경망 훈련하기¶
In [19]:
# 학습
rmsprop = keras.optimizers.RMSprop(learning_rate=1e-4) # 학습률 변경을 위한 코드
model.compile(optimizer=rmsprop, loss='binary_crossentropy',
metrics=['accuracy'])
checkpoint_cb = keras.callbacks.ModelCheckpoint('best-simplernn-model.h5',
save_best_only=True)
early_stopping_cb = keras.callbacks.EarlyStopping(patience=3,
restore_best_weights=True)
history = model.fit(train_oh, train_target, epochs=100, batch_size=64,
validation_data=(val_oh, val_target),
callbacks=[checkpoint_cb, early_stopping_cb])
# 약 0.80의 정확도
Epoch 1/100
313/313 [==============================] - 23s 68ms/step - loss: 0.6973 - accuracy: 0.4994 - val_loss: 0.6958 - val_accuracy: 0.5022
Epoch 2/100
313/313 [==============================] - 15s 49ms/step - loss: 0.6925 - accuracy: 0.5176 - val_loss: 0.6922 - val_accuracy: 0.5174
Epoch 3/100
313/313 [==============================] - 13s 41ms/step - loss: 0.6861 - accuracy: 0.5494 - val_loss: 0.6827 - val_accuracy: 0.5684
Epoch 4/100
313/313 [==============================] - 10s 32ms/step - loss: 0.6752 - accuracy: 0.5889 - val_loss: 0.6692 - val_accuracy: 0.6114
Epoch 5/100
313/313 [==============================] - 9s 30ms/step - loss: 0.6607 - accuracy: 0.6245 - val_loss: 0.6527 - val_accuracy: 0.6516
Epoch 6/100
313/313 [==============================] - 9s 30ms/step - loss: 0.6448 - accuracy: 0.6557 - val_loss: 0.6360 - val_accuracy: 0.6774
Epoch 7/100
313/313 [==============================] - 9s 30ms/step - loss: 0.6281 - accuracy: 0.6795 - val_loss: 0.6192 - val_accuracy: 0.6998
Epoch 8/100
313/313 [==============================] - 9s 30ms/step - loss: 0.6104 - accuracy: 0.7008 - val_loss: 0.6009 - val_accuracy: 0.7146
Epoch 9/100
313/313 [==============================] - 9s 30ms/step - loss: 0.5882 - accuracy: 0.7269 - val_loss: 0.5722 - val_accuracy: 0.7446
Epoch 10/100
313/313 [==============================] - 9s 30ms/step - loss: 0.5624 - accuracy: 0.7466 - val_loss: 0.5591 - val_accuracy: 0.7436
Epoch 11/100
313/313 [==============================] - 9s 30ms/step - loss: 0.5460 - accuracy: 0.7545 - val_loss: 0.5408 - val_accuracy: 0.7572
Epoch 12/100
313/313 [==============================] - 10s 33ms/step - loss: 0.5303 - accuracy: 0.7624 - val_loss: 0.5302 - val_accuracy: 0.7634
Epoch 13/100
313/313 [==============================] - 11s 37ms/step - loss: 0.5183 - accuracy: 0.7664 - val_loss: 0.5177 - val_accuracy: 0.7702
Epoch 14/100
313/313 [==============================] - 11s 36ms/step - loss: 0.5071 - accuracy: 0.7725 - val_loss: 0.5099 - val_accuracy: 0.7694
Epoch 15/100
313/313 [==============================] - 11s 34ms/step - loss: 0.4970 - accuracy: 0.7753 - val_loss: 0.4994 - val_accuracy: 0.7750
Epoch 16/100
313/313 [==============================] - 11s 35ms/step - loss: 0.4887 - accuracy: 0.7799 - val_loss: 0.4930 - val_accuracy: 0.7768
Epoch 17/100
313/313 [==============================] - 10s 33ms/step - loss: 0.4807 - accuracy: 0.7846 - val_loss: 0.4879 - val_accuracy: 0.7800
Epoch 18/100
313/313 [==============================] - 10s 32ms/step - loss: 0.4739 - accuracy: 0.7873 - val_loss: 0.4827 - val_accuracy: 0.7822
Epoch 19/100
313/313 [==============================] - 10s 32ms/step - loss: 0.4671 - accuracy: 0.7913 - val_loss: 0.4804 - val_accuracy: 0.7826
Epoch 20/100
313/313 [==============================] - 10s 33ms/step - loss: 0.4610 - accuracy: 0.7944 - val_loss: 0.4790 - val_accuracy: 0.7850
Epoch 21/100
313/313 [==============================] - 10s 30ms/step - loss: 0.4567 - accuracy: 0.7976 - val_loss: 0.4736 - val_accuracy: 0.7836
Epoch 22/100
313/313 [==============================] - 13s 41ms/step - loss: 0.4528 - accuracy: 0.7980 - val_loss: 0.4758 - val_accuracy: 0.7850
Epoch 23/100
313/313 [==============================] - 11s 34ms/step - loss: 0.4490 - accuracy: 0.7992 - val_loss: 0.4700 - val_accuracy: 0.7864
Epoch 24/100
313/313 [==============================] - 9s 30ms/step - loss: 0.4457 - accuracy: 0.8010 - val_loss: 0.4692 - val_accuracy: 0.7804
Epoch 25/100
313/313 [==============================] - 12s 38ms/step - loss: 0.4418 - accuracy: 0.8026 - val_loss: 0.4664 - val_accuracy: 0.7878
Epoch 26/100
313/313 [==============================] - 10s 32ms/step - loss: 0.4383 - accuracy: 0.8035 - val_loss: 0.4650 - val_accuracy: 0.7868
Epoch 27/100
313/313 [==============================] - 10s 31ms/step - loss: 0.4348 - accuracy: 0.8068 - val_loss: 0.4596 - val_accuracy: 0.7936
Epoch 28/100
313/313 [==============================] - 10s 31ms/step - loss: 0.4333 - accuracy: 0.8059 - val_loss: 0.4656 - val_accuracy: 0.7924
Epoch 29/100
313/313 [==============================] - 10s 31ms/step - loss: 0.4302 - accuracy: 0.8090 - val_loss: 0.4625 - val_accuracy: 0.7816
Epoch 30/100
313/313 [==============================] - 10s 30ms/step - loss: 0.4284 - accuracy: 0.8094 - val_loss: 0.4556 - val_accuracy: 0.7956
Epoch 31/100
313/313 [==============================] - 9s 30ms/step - loss: 0.4269 - accuracy: 0.8105 - val_loss: 0.4567 - val_accuracy: 0.7916
Epoch 32/100
313/313 [==============================] - 9s 30ms/step - loss: 0.4240 - accuracy: 0.8119 - val_loss: 0.4540 - val_accuracy: 0.7942
Epoch 33/100
313/313 [==============================] - 9s 29ms/step - loss: 0.4230 - accuracy: 0.8122 - val_loss: 0.4564 - val_accuracy: 0.7870
Epoch 34/100
313/313 [==============================] - 9s 29ms/step - loss: 0.4209 - accuracy: 0.8144 - val_loss: 0.4526 - val_accuracy: 0.7954
Epoch 35/100
313/313 [==============================] - 9s 30ms/step - loss: 0.4196 - accuracy: 0.8155 - val_loss: 0.4547 - val_accuracy: 0.7970
Epoch 36/100
313/313 [==============================] - 10s 30ms/step - loss: 0.4181 - accuracy: 0.8166 - val_loss: 0.4493 - val_accuracy: 0.7918
Epoch 37/100
313/313 [==============================] - 9s 30ms/step - loss: 0.4170 - accuracy: 0.8160 - val_loss: 0.4554 - val_accuracy: 0.7952
Epoch 38/100
313/313 [==============================] - 9s 30ms/step - loss: 0.4148 - accuracy: 0.8184 - val_loss: 0.4507 - val_accuracy: 0.7964
Epoch 39/100
313/313 [==============================] - 9s 30ms/step - loss: 0.4135 - accuracy: 0.8184 - val_loss: 0.4491 - val_accuracy: 0.7924
Epoch 40/100
313/313 [==============================] - 9s 30ms/step - loss: 0.4132 - accuracy: 0.8187 - val_loss: 0.4517 - val_accuracy: 0.7982
Epoch 41/100
313/313 [==============================] - 10s 31ms/step - loss: 0.4115 - accuracy: 0.8196 - val_loss: 0.4491 - val_accuracy: 0.7966
Epoch 42/100
313/313 [==============================] - 10s 32ms/step - loss: 0.4105 - accuracy: 0.8189 - val_loss: 0.4503 - val_accuracy: 0.7972
Epoch 43/100
313/313 [==============================] - 10s 32ms/step - loss: 0.4095 - accuracy: 0.8181 - val_loss: 0.4479 - val_accuracy: 0.7982
Epoch 44/100
313/313 [==============================] - 10s 32ms/step - loss: 0.4087 - accuracy: 0.8213 - val_loss: 0.4485 - val_accuracy: 0.7900
Epoch 45/100
313/313 [==============================] - 11s 35ms/step - loss: 0.4078 - accuracy: 0.8205 - val_loss: 0.4532 - val_accuracy: 0.7966
Epoch 46/100
313/313 [==============================] - 10s 31ms/step - loss: 0.4066 - accuracy: 0.8223 - val_loss: 0.4461 - val_accuracy: 0.7886
Epoch 47/100
313/313 [==============================] - 9s 30ms/step - loss: 0.4056 - accuracy: 0.8207 - val_loss: 0.4454 - val_accuracy: 0.7960
Epoch 48/100
313/313 [==============================] - 9s 30ms/step - loss: 0.4050 - accuracy: 0.8198 - val_loss: 0.4456 - val_accuracy: 0.7928
Epoch 49/100
313/313 [==============================] - 10s 30ms/step - loss: 0.4042 - accuracy: 0.8235 - val_loss: 0.4477 - val_accuracy: 0.7996
Epoch 50/100
313/313 [==============================] - 9s 30ms/step - loss: 0.4035 - accuracy: 0.8218 - val_loss: 0.4459 - val_accuracy: 0.7978
In [20]:
# 훈련 손실과 검증 손실 그래프
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.xlabel('epoch')
plt.ylabel('loss')
plt.legend(['train', 'val'])
plt.show()
☆ 순환 신경망은 원-핫 인코딩으로 변환을 해야했습니다. 하지만 원-핫 인코딩의 단점은 입력 데이터가 엄청 커진다는 것입니다.
In [21]:
# 500배가 커진 것을 확인할 수 있다
print(train_seq.nbytes, train_oh.nbytes)
8000000 4000000000
단어 임베딩을 사용하기¶
☆ 순환 신경망에서 텍스트를 처리할 때 즐겨 사용하는 방법은 단어 임베딩(word embedding)
☆ 단어 임베딩 : 각 단어를 고정된 크기의 실수 벡터로 바꾸어 주는 것이다.
☆ 원-핫 인코딩된 벡터보다 훨씬 의미 있는 값으로 채워져 있기 때문에 자연어 처리에서 더 좋은 성능을 내는 경우가 많다.
In [22]:
# 모델 구현
model2 = keras.Sequential()
model2.add(keras.layers.Embedding(500, 16, input_length=100)) # 임베딩 벡터 크기 16을 사용
model2.add(keras.layers.SimpleRNN(8))
model2.add(keras.layers.Dense(1, activation='sigmoid'))
model2.summary()
Model: "sequential_1"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
embedding (Embedding) (None, 100, 16) 8000
_________________________________________________________________
simple_rnn_1 (SimpleRNN) (None, 8) 200
_________________________________________________________________
dense_1 (Dense) (None, 1) 9
=================================================================
Total params: 8,209
Trainable params: 8,209
Non-trainable params: 0
_________________________________________________________________
In [23]:
# 학습
rmsprop = keras.optimizers.RMSprop(learning_rate=1e-4)
model2.compile(optimizer=rmsprop, loss='binary_crossentropy',
metrics=['accuracy'])
checkpoint_cb = keras.callbacks.ModelCheckpoint('best-embedding-model.h5',
save_best_only=True)
early_stopping_cb = keras.callbacks.EarlyStopping(patience=3,
restore_best_weights=True)
history = model2.fit(train_seq, train_target, epochs=100, batch_size=64,
validation_data=(val_seq, val_target),
callbacks=[checkpoint_cb, early_stopping_cb])
# 비슷한 성능을 가지지만 순환층의 가중치 개수는 훨씬 작고 훈련 세트 크기도 훨씬 줄어들었습니다.
Epoch 1/100
313/313 [==============================] - 10s 27ms/step - loss: 0.6618 - accuracy: 0.6025 - val_loss: 0.6349 - val_accuracy: 0.6620
Epoch 2/100
313/313 [==============================] - 9s 28ms/step - loss: 0.6089 - accuracy: 0.6977 - val_loss: 0.5955 - val_accuracy: 0.7168
Epoch 3/100
313/313 [==============================] - 9s 28ms/step - loss: 0.5768 - accuracy: 0.7361 - val_loss: 0.5715 - val_accuracy: 0.7304
Epoch 4/100
313/313 [==============================] - 9s 28ms/step - loss: 0.5543 - accuracy: 0.7561 - val_loss: 0.5536 - val_accuracy: 0.7556
Epoch 5/100
313/313 [==============================] - 9s 29ms/step - loss: 0.5358 - accuracy: 0.7693 - val_loss: 0.5370 - val_accuracy: 0.7608
Epoch 6/100
313/313 [==============================] - 8s 25ms/step - loss: 0.5204 - accuracy: 0.7763 - val_loss: 0.5235 - val_accuracy: 0.7692
Epoch 7/100
313/313 [==============================] - 8s 25ms/step - loss: 0.5062 - accuracy: 0.7819 - val_loss: 0.5108 - val_accuracy: 0.7740
Epoch 8/100
313/313 [==============================] - 8s 25ms/step - loss: 0.4932 - accuracy: 0.7907 - val_loss: 0.5011 - val_accuracy: 0.7752
Epoch 9/100
313/313 [==============================] - 8s 26ms/step - loss: 0.4824 - accuracy: 0.7950 - val_loss: 0.4913 - val_accuracy: 0.7856
Epoch 10/100
313/313 [==============================] - 8s 26ms/step - loss: 0.4720 - accuracy: 0.7991 - val_loss: 0.4841 - val_accuracy: 0.7884
Epoch 11/100
313/313 [==============================] - 8s 26ms/step - loss: 0.4625 - accuracy: 0.8033 - val_loss: 0.4788 - val_accuracy: 0.7852
Epoch 12/100
313/313 [==============================] - 8s 26ms/step - loss: 0.4542 - accuracy: 0.8056 - val_loss: 0.4730 - val_accuracy: 0.7876
Epoch 13/100
313/313 [==============================] - 8s 26ms/step - loss: 0.4467 - accuracy: 0.8064 - val_loss: 0.4697 - val_accuracy: 0.7918
Epoch 14/100
313/313 [==============================] - 8s 25ms/step - loss: 0.4403 - accuracy: 0.8112 - val_loss: 0.4682 - val_accuracy: 0.7876
Epoch 15/100
313/313 [==============================] - 8s 25ms/step - loss: 0.4342 - accuracy: 0.8134 - val_loss: 0.4615 - val_accuracy: 0.7926
Epoch 16/100
313/313 [==============================] - 8s 26ms/step - loss: 0.4297 - accuracy: 0.8145 - val_loss: 0.4614 - val_accuracy: 0.7936
Epoch 17/100
313/313 [==============================] - 8s 26ms/step - loss: 0.4252 - accuracy: 0.8148 - val_loss: 0.4623 - val_accuracy: 0.7896
Epoch 18/100
313/313 [==============================] - 8s 26ms/step - loss: 0.4216 - accuracy: 0.8186 - val_loss: 0.4568 - val_accuracy: 0.7906
Epoch 19/100
313/313 [==============================] - 8s 26ms/step - loss: 0.4180 - accuracy: 0.8185 - val_loss: 0.4569 - val_accuracy: 0.7912
Epoch 20/100
313/313 [==============================] - 8s 25ms/step - loss: 0.4150 - accuracy: 0.8193 - val_loss: 0.4586 - val_accuracy: 0.7914
Epoch 21/100
313/313 [==============================] - 8s 25ms/step - loss: 0.4126 - accuracy: 0.8216 - val_loss: 0.4582 - val_accuracy: 0.7914
In [24]:
# 훈련 손실과 검증 손실 그래프
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.xlabel('epoch')
plt.ylabel('loss')
plt.legend(['train', 'val'])
plt.show()
09-3 LSTM과 GRU 셀¶
☆ 기본 순환층은 시퀀스가 길수록 순환되는 은닉 상태에 담긴 정보가 희석되기 때문에 학습하기 어려워 진다.
☆ 이런 문제점을 해결하기 위해 LSTM과 GRU 셀이 발명
LSTM 구조 (Long Short-Term Memory)¶
In [25]:
# 데이터 셋 준비
from tensorflow.keras.datasets import imdb
from sklearn.model_selection import train_test_split
(train_input, train_target), (test_input, test_target) = imdb.load_data(
num_words=500)
train_input, val_input, train_target, val_target = train_test_split(
train_input, train_target, test_size=0.2, random_state=42)
In [26]:
from tensorflow.keras.preprocessing.sequence import pad_sequences
train_seq = pad_sequences(train_input, maxlen=100) # 샘플 길이 100으로 맞춘다
val_seq = pad_sequences(val_input, maxlen=100)
In [27]:
# LSTM 모델 구현
from tensorflow import keras
model = keras.Sequential()
model.add(keras.layers.Embedding(500, 16, input_length=100))
model.add(keras.layers.LSTM(8))
model.add(keras.layers.Dense(1, activation='sigmoid'))
model.summary()
Model: "sequential_2"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
embedding_1 (Embedding) (None, 100, 16) 8000
_________________________________________________________________
lstm (LSTM) (None, 8) 800
_________________________________________________________________
dense_2 (Dense) (None, 1) 9
=================================================================
Total params: 8,809
Trainable params: 8,809
Non-trainable params: 0
_________________________________________________________________
In [28]:
# 학습
rmsprop = keras.optimizers.RMSprop(learning_rate=1e-4)
model.compile(optimizer=rmsprop, loss='binary_crossentropy',
metrics=['accuracy'])
checkpoint_cb = keras.callbacks.ModelCheckpoint('best-lstm-model.h5',
save_best_only=True)
early_stopping_cb = keras.callbacks.EarlyStopping(patience=3,
restore_best_weights=True)
history = model.fit(train_seq, train_target, epochs=100, batch_size=64,
validation_data=(val_seq, val_target),
callbacks=[checkpoint_cb, early_stopping_cb])
Epoch 1/100
313/313 [==============================] - 18s 49ms/step - loss: 0.6910 - accuracy: 0.5780 - val_loss: 0.6877 - val_accuracy: 0.6346
Epoch 2/100
313/313 [==============================] - 14s 46ms/step - loss: 0.6763 - accuracy: 0.6623 - val_loss: 0.6491 - val_accuracy: 0.7058
Epoch 3/100
313/313 [==============================] - 13s 42ms/step - loss: 0.6083 - accuracy: 0.7072 - val_loss: 0.5843 - val_accuracy: 0.7248
Epoch 4/100
313/313 [==============================] - 13s 41ms/step - loss: 0.5712 - accuracy: 0.7322 - val_loss: 0.5630 - val_accuracy: 0.7356
Epoch 5/100
313/313 [==============================] - 13s 43ms/step - loss: 0.5494 - accuracy: 0.7502 - val_loss: 0.5440 - val_accuracy: 0.7534
Epoch 6/100
313/313 [==============================] - 14s 43ms/step - loss: 0.5323 - accuracy: 0.7591 - val_loss: 0.5303 - val_accuracy: 0.7580
Epoch 7/100
313/313 [==============================] - 13s 42ms/step - loss: 0.5171 - accuracy: 0.7675 - val_loss: 0.5171 - val_accuracy: 0.7664
Epoch 8/100
313/313 [==============================] - 13s 42ms/step - loss: 0.5037 - accuracy: 0.7760 - val_loss: 0.5079 - val_accuracy: 0.7684
Epoch 9/100
313/313 [==============================] - 13s 42ms/step - loss: 0.4910 - accuracy: 0.7831 - val_loss: 0.4963 - val_accuracy: 0.7792
Epoch 10/100
313/313 [==============================] - 13s 42ms/step - loss: 0.4796 - accuracy: 0.7900 - val_loss: 0.4872 - val_accuracy: 0.7746
Epoch 11/100
313/313 [==============================] - 13s 42ms/step - loss: 0.4694 - accuracy: 0.7927 - val_loss: 0.4769 - val_accuracy: 0.7814
Epoch 12/100
313/313 [==============================] - 15s 47ms/step - loss: 0.4602 - accuracy: 0.7972 - val_loss: 0.4695 - val_accuracy: 0.7918
Epoch 13/100
313/313 [==============================] - 13s 43ms/step - loss: 0.4519 - accuracy: 0.8012 - val_loss: 0.4620 - val_accuracy: 0.7870
Epoch 14/100
313/313 [==============================] - 14s 43ms/step - loss: 0.4453 - accuracy: 0.8039 - val_loss: 0.4562 - val_accuracy: 0.7916
Epoch 15/100
313/313 [==============================] - 14s 44ms/step - loss: 0.4389 - accuracy: 0.8047 - val_loss: 0.4512 - val_accuracy: 0.7954
Epoch 16/100
313/313 [==============================] - 13s 42ms/step - loss: 0.4341 - accuracy: 0.8077 - val_loss: 0.4472 - val_accuracy: 0.7964
Epoch 17/100
313/313 [==============================] - 14s 44ms/step - loss: 0.4299 - accuracy: 0.8083 - val_loss: 0.4447 - val_accuracy: 0.7968
Epoch 18/100
313/313 [==============================] - 14s 44ms/step - loss: 0.4261 - accuracy: 0.8106 - val_loss: 0.4410 - val_accuracy: 0.7988
Epoch 19/100
313/313 [==============================] - 15s 47ms/step - loss: 0.4231 - accuracy: 0.8116 - val_loss: 0.4418 - val_accuracy: 0.7950
Epoch 20/100
313/313 [==============================] - 13s 42ms/step - loss: 0.4206 - accuracy: 0.8120 - val_loss: 0.4371 - val_accuracy: 0.7986
Epoch 21/100
313/313 [==============================] - 13s 42ms/step - loss: 0.4180 - accuracy: 0.8135 - val_loss: 0.4363 - val_accuracy: 0.8004
Epoch 22/100
313/313 [==============================] - 13s 43ms/step - loss: 0.4161 - accuracy: 0.8130 - val_loss: 0.4363 - val_accuracy: 0.7954
Epoch 23/100
313/313 [==============================] - 13s 42ms/step - loss: 0.4144 - accuracy: 0.8133 - val_loss: 0.4322 - val_accuracy: 0.8024
Epoch 24/100
313/313 [==============================] - 13s 42ms/step - loss: 0.4132 - accuracy: 0.8127 - val_loss: 0.4315 - val_accuracy: 0.8030
Epoch 25/100
313/313 [==============================] - 13s 42ms/step - loss: 0.4120 - accuracy: 0.8128 - val_loss: 0.4307 - val_accuracy: 0.8008
Epoch 26/100
313/313 [==============================] - 13s 41ms/step - loss: 0.4109 - accuracy: 0.8138 - val_loss: 0.4314 - val_accuracy: 0.8030
Epoch 27/100
313/313 [==============================] - 13s 41ms/step - loss: 0.4096 - accuracy: 0.8138 - val_loss: 0.4311 - val_accuracy: 0.8044
Epoch 28/100
313/313 [==============================] - 13s 42ms/step - loss: 0.4085 - accuracy: 0.8146 - val_loss: 0.4301 - val_accuracy: 0.8058
Epoch 29/100
313/313 [==============================] - 13s 42ms/step - loss: 0.4078 - accuracy: 0.8150 - val_loss: 0.4293 - val_accuracy: 0.8020
Epoch 30/100
313/313 [==============================] - 13s 42ms/step - loss: 0.4072 - accuracy: 0.8146 - val_loss: 0.4325 - val_accuracy: 0.7976
Epoch 31/100
313/313 [==============================] - 15s 47ms/step - loss: 0.4069 - accuracy: 0.8137 - val_loss: 0.4298 - val_accuracy: 0.8064
Epoch 32/100
313/313 [==============================] - 14s 44ms/step - loss: 0.4060 - accuracy: 0.8149 - val_loss: 0.4301 - val_accuracy: 0.7988
In [29]:
# 훈련 손실과 검증 손실 그래프
import matplotlib.pyplot as plt
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.xlabel('epoch')
plt.ylabel('loss')
plt.legend(['train', 'val'])
plt.show()
2개의 층을 연결하기¶
In [30]:
# 모델 구현
model3 = keras.Sequential()
model3.add(keras.layers.Embedding(500, 16, input_length=100))
model3.add(keras.layers.LSTM(8, dropout=0.3, return_sequences=True)) # 모든 타임스텝의 은닉 상태를 출력하기 위한 코드
model3.add(keras.layers.LSTM(8, dropout=0.3))
model3.add(keras.layers.Dense(1, activation='sigmoid'))
model3.summary()
Model: "sequential_3"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
embedding_2 (Embedding) (None, 100, 16) 8000
_________________________________________________________________
lstm_1 (LSTM) (None, 100, 8) 800
_________________________________________________________________
lstm_2 (LSTM) (None, 8) 544
_________________________________________________________________
dense_3 (Dense) (None, 1) 9
=================================================================
Total params: 9,353
Trainable params: 9,353
Non-trainable params: 0
_________________________________________________________________
In [31]:
# 학습
rmsprop = keras.optimizers.RMSprop(learning_rate=1e-4)
model3.compile(optimizer=rmsprop, loss='binary_crossentropy',
metrics=['accuracy'])
checkpoint_cb = keras.callbacks.ModelCheckpoint('best-2rnn-model.h5',
save_best_only=True)
early_stopping_cb = keras.callbacks.EarlyStopping(patience=3,
restore_best_weights=True)
history = model3.fit(train_seq, train_target, epochs=100, batch_size=64,
validation_data=(val_seq, val_target),
callbacks=[checkpoint_cb, early_stopping_cb])
Epoch 1/100
313/313 [==============================] - 37s 99ms/step - loss: 0.6928 - accuracy: 0.5337 - val_loss: 0.6921 - val_accuracy: 0.5722
Epoch 2/100
313/313 [==============================] - 27s 85ms/step - loss: 0.6892 - accuracy: 0.6129 - val_loss: 0.6832 - val_accuracy: 0.6634
Epoch 3/100
313/313 [==============================] - 26s 82ms/step - loss: 0.6618 - accuracy: 0.6692 - val_loss: 0.6246 - val_accuracy: 0.7012
Epoch 4/100
313/313 [==============================] - 27s 85ms/step - loss: 0.5916 - accuracy: 0.7160 - val_loss: 0.5640 - val_accuracy: 0.7292
Epoch 5/100
313/313 [==============================] - 26s 84ms/step - loss: 0.5515 - accuracy: 0.7406 - val_loss: 0.5365 - val_accuracy: 0.7454
Epoch 6/100
313/313 [==============================] - 29s 94ms/step - loss: 0.5305 - accuracy: 0.7494 - val_loss: 0.5180 - val_accuracy: 0.7608
Epoch 7/100
313/313 [==============================] - 29s 92ms/step - loss: 0.5102 - accuracy: 0.7632 - val_loss: 0.5012 - val_accuracy: 0.7704
Epoch 8/100
313/313 [==============================] - 32s 103ms/step - loss: 0.4971 - accuracy: 0.7706 - val_loss: 0.4922 - val_accuracy: 0.7720
Epoch 9/100
313/313 [==============================] - 29s 91ms/step - loss: 0.4874 - accuracy: 0.7763 - val_loss: 0.4824 - val_accuracy: 0.7776
Epoch 10/100
313/313 [==============================] - 25s 81ms/step - loss: 0.4795 - accuracy: 0.7796 - val_loss: 0.4734 - val_accuracy: 0.7830
Epoch 11/100
313/313 [==============================] - 26s 82ms/step - loss: 0.4693 - accuracy: 0.7842 - val_loss: 0.4676 - val_accuracy: 0.7822
Epoch 12/100
313/313 [==============================] - 25s 81ms/step - loss: 0.4633 - accuracy: 0.7876 - val_loss: 0.4647 - val_accuracy: 0.7830
Epoch 13/100
313/313 [==============================] - 26s 84ms/step - loss: 0.4588 - accuracy: 0.7928 - val_loss: 0.4648 - val_accuracy: 0.7860
Epoch 14/100
313/313 [==============================] - 26s 81ms/step - loss: 0.4553 - accuracy: 0.7944 - val_loss: 0.4553 - val_accuracy: 0.7876
Epoch 15/100
313/313 [==============================] - 26s 82ms/step - loss: 0.4530 - accuracy: 0.7930 - val_loss: 0.4526 - val_accuracy: 0.7906
Epoch 16/100
313/313 [==============================] - 26s 82ms/step - loss: 0.4484 - accuracy: 0.7958 - val_loss: 0.4534 - val_accuracy: 0.7918
Epoch 17/100
313/313 [==============================] - 29s 93ms/step - loss: 0.4452 - accuracy: 0.7988 - val_loss: 0.4475 - val_accuracy: 0.7922
Epoch 18/100
313/313 [==============================] - 27s 86ms/step - loss: 0.4422 - accuracy: 0.7984 - val_loss: 0.4480 - val_accuracy: 0.7942
Epoch 19/100
313/313 [==============================] - 25s 78ms/step - loss: 0.4438 - accuracy: 0.7985 - val_loss: 0.4463 - val_accuracy: 0.7924
Epoch 20/100
313/313 [==============================] - 25s 80ms/step - loss: 0.4404 - accuracy: 0.7985 - val_loss: 0.4430 - val_accuracy: 0.7946
Epoch 21/100
313/313 [==============================] - 25s 79ms/step - loss: 0.4366 - accuracy: 0.8001 - val_loss: 0.4452 - val_accuracy: 0.7902
Epoch 22/100
313/313 [==============================] - 25s 79ms/step - loss: 0.4343 - accuracy: 0.8009 - val_loss: 0.4424 - val_accuracy: 0.7980
Epoch 23/100
313/313 [==============================] - 25s 79ms/step - loss: 0.4322 - accuracy: 0.8044 - val_loss: 0.4404 - val_accuracy: 0.7932
Epoch 24/100
313/313 [==============================] - 25s 79ms/step - loss: 0.4335 - accuracy: 0.8016 - val_loss: 0.4382 - val_accuracy: 0.7984
Epoch 25/100
313/313 [==============================] - 25s 80ms/step - loss: 0.4313 - accuracy: 0.8025 - val_loss: 0.4380 - val_accuracy: 0.7962
Epoch 26/100
313/313 [==============================] - 25s 78ms/step - loss: 0.4288 - accuracy: 0.8010 - val_loss: 0.4377 - val_accuracy: 0.8010
Epoch 27/100
313/313 [==============================] - 25s 79ms/step - loss: 0.4295 - accuracy: 0.8021 - val_loss: 0.4354 - val_accuracy: 0.7962
Epoch 28/100
313/313 [==============================] - 25s 80ms/step - loss: 0.4276 - accuracy: 0.8019 - val_loss: 0.4340 - val_accuracy: 0.7996
Epoch 29/100
313/313 [==============================] - 26s 84ms/step - loss: 0.4276 - accuracy: 0.8049 - val_loss: 0.4351 - val_accuracy: 0.7966
Epoch 30/100
313/313 [==============================] - 27s 86ms/step - loss: 0.4246 - accuracy: 0.8059 - val_loss: 0.4358 - val_accuracy: 0.8020
Epoch 31/100
313/313 [==============================] - 33s 106ms/step - loss: 0.4248 - accuracy: 0.8058 - val_loss: 0.4343 - val_accuracy: 0.8002
In [32]:
# 훈련 손실과 검증 손실 그래프
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.xlabel('epoch')
plt.ylabel('loss')
plt.legend(['train', 'val'])
plt.show()
GRU 구조¶
☆ LSTM처럼 셀 상태를 계산하지 않고 은닉 상태 하나만 포함합니다.
☆ LSTM보다 가중치가 적어 계산량이 적지만 LSTM 못지않은 좋은 성능을 낸다.
In [33]:
# 모델 구현
model4 = keras.Sequential()
model4.add(keras.layers.Embedding(500, 16, input_length=100))
model4.add(keras.layers.GRU(8))
model4.add(keras.layers.Dense(1, activation='sigmoid'))
model4.summary()
Model: "sequential_4"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
embedding_3 (Embedding) (None, 100, 16) 8000
_________________________________________________________________
gru (GRU) (None, 8) 624
_________________________________________________________________
dense_4 (Dense) (None, 1) 9
=================================================================
Total params: 8,633
Trainable params: 8,633
Non-trainable params: 0
_________________________________________________________________
In [34]:
# 학습
rmsprop = keras.optimizers.RMSprop(learning_rate=1e-4)
model4.compile(optimizer=rmsprop, loss='binary_crossentropy',
metrics=['accuracy'])
checkpoint_cb = keras.callbacks.ModelCheckpoint('best-gru-model.h5',
save_best_only=True)
early_stopping_cb = keras.callbacks.EarlyStopping(patience=3,
restore_best_weights=True)
history = model4.fit(train_seq, train_target, epochs=100, batch_size=64,
validation_data=(val_seq, val_target),
callbacks=[checkpoint_cb, early_stopping_cb])
Epoch 1/100
313/313 [==============================] - 19s 52ms/step - loss: 0.6925 - accuracy: 0.5281 - val_loss: 0.6921 - val_accuracy: 0.5502
Epoch 2/100
313/313 [==============================] - 16s 50ms/step - loss: 0.6909 - accuracy: 0.5799 - val_loss: 0.6902 - val_accuracy: 0.5866
Epoch 3/100
313/313 [==============================] - 15s 49ms/step - loss: 0.6882 - accuracy: 0.6016 - val_loss: 0.6869 - val_accuracy: 0.6020
Epoch 4/100
313/313 [==============================] - 14s 45ms/step - loss: 0.6835 - accuracy: 0.6159 - val_loss: 0.6815 - val_accuracy: 0.6148
Epoch 5/100
313/313 [==============================] - 13s 43ms/step - loss: 0.6758 - accuracy: 0.6299 - val_loss: 0.6724 - val_accuracy: 0.6256
Epoch 6/100
313/313 [==============================] - 13s 42ms/step - loss: 0.6626 - accuracy: 0.6477 - val_loss: 0.6561 - val_accuracy: 0.6436
Epoch 7/100
313/313 [==============================] - 13s 42ms/step - loss: 0.6388 - accuracy: 0.6709 - val_loss: 0.6262 - val_accuracy: 0.6740
Epoch 8/100
313/313 [==============================] - 13s 43ms/step - loss: 0.5912 - accuracy: 0.7077 - val_loss: 0.5634 - val_accuracy: 0.7212
Epoch 9/100
313/313 [==============================] - 13s 42ms/step - loss: 0.5264 - accuracy: 0.7448 - val_loss: 0.5227 - val_accuracy: 0.7472
Epoch 10/100
313/313 [==============================] - 13s 42ms/step - loss: 0.5029 - accuracy: 0.7596 - val_loss: 0.5086 - val_accuracy: 0.7544
Epoch 11/100
313/313 [==============================] - 13s 42ms/step - loss: 0.4873 - accuracy: 0.7719 - val_loss: 0.4922 - val_accuracy: 0.7694
Epoch 12/100
313/313 [==============================] - 13s 42ms/step - loss: 0.4749 - accuracy: 0.7793 - val_loss: 0.4820 - val_accuracy: 0.7742
Epoch 13/100
313/313 [==============================] - 13s 42ms/step - loss: 0.4651 - accuracy: 0.7859 - val_loss: 0.4748 - val_accuracy: 0.7804
Epoch 14/100
313/313 [==============================] - 13s 43ms/step - loss: 0.4571 - accuracy: 0.7901 - val_loss: 0.4687 - val_accuracy: 0.7806
Epoch 15/100
313/313 [==============================] - 13s 42ms/step - loss: 0.4508 - accuracy: 0.7946 - val_loss: 0.4631 - val_accuracy: 0.7848
Epoch 16/100
313/313 [==============================] - 13s 42ms/step - loss: 0.4450 - accuracy: 0.7988 - val_loss: 0.4613 - val_accuracy: 0.7900
Epoch 17/100
313/313 [==============================] - 14s 44ms/step - loss: 0.4409 - accuracy: 0.8019 - val_loss: 0.4573 - val_accuracy: 0.7900
Epoch 18/100
313/313 [==============================] - 13s 42ms/step - loss: 0.4372 - accuracy: 0.8050 - val_loss: 0.4545 - val_accuracy: 0.7916
Epoch 19/100
313/313 [==============================] - 13s 42ms/step - loss: 0.4343 - accuracy: 0.8065 - val_loss: 0.4523 - val_accuracy: 0.7930
Epoch 20/100
313/313 [==============================] - 13s 42ms/step - loss: 0.4312 - accuracy: 0.8083 - val_loss: 0.4534 - val_accuracy: 0.7898
Epoch 21/100
313/313 [==============================] - 13s 42ms/step - loss: 0.4287 - accuracy: 0.8083 - val_loss: 0.4498 - val_accuracy: 0.7920
Epoch 22/100
313/313 [==============================] - 13s 42ms/step - loss: 0.4267 - accuracy: 0.8102 - val_loss: 0.4483 - val_accuracy: 0.7960
Epoch 23/100
313/313 [==============================] - 13s 42ms/step - loss: 0.4254 - accuracy: 0.8113 - val_loss: 0.4482 - val_accuracy: 0.7948
Epoch 24/100
313/313 [==============================] - 13s 42ms/step - loss: 0.4236 - accuracy: 0.8134 - val_loss: 0.4487 - val_accuracy: 0.7892
Epoch 25/100
313/313 [==============================] - 14s 43ms/step - loss: 0.4223 - accuracy: 0.8130 - val_loss: 0.4460 - val_accuracy: 0.7948
Epoch 26/100
313/313 [==============================] - 17s 53ms/step - loss: 0.4208 - accuracy: 0.8150 - val_loss: 0.4477 - val_accuracy: 0.7928
Epoch 27/100
313/313 [==============================] - 15s 47ms/step - loss: 0.4201 - accuracy: 0.8152 - val_loss: 0.4451 - val_accuracy: 0.7954
Epoch 28/100
313/313 [==============================] - 15s 49ms/step - loss: 0.4185 - accuracy: 0.8153 - val_loss: 0.4480 - val_accuracy: 0.7906
Epoch 29/100
313/313 [==============================] - 15s 48ms/step - loss: 0.4181 - accuracy: 0.8153 - val_loss: 0.4495 - val_accuracy: 0.7930
Epoch 30/100
313/313 [==============================] - 15s 47ms/step - loss: 0.4171 - accuracy: 0.8162 - val_loss: 0.4436 - val_accuracy: 0.7948
Epoch 31/100
313/313 [==============================] - 15s 48ms/step - loss: 0.4165 - accuracy: 0.8153 - val_loss: 0.4438 - val_accuracy: 0.7992
Epoch 32/100
313/313 [==============================] - 15s 47ms/step - loss: 0.4156 - accuracy: 0.8159 - val_loss: 0.4431 - val_accuracy: 0.7942
Epoch 33/100
313/313 [==============================] - 15s 47ms/step - loss: 0.4153 - accuracy: 0.8177 - val_loss: 0.4420 - val_accuracy: 0.7974
Epoch 34/100
313/313 [==============================] - 15s 50ms/step - loss: 0.4149 - accuracy: 0.8170 - val_loss: 0.4409 - val_accuracy: 0.7968
Epoch 35/100
313/313 [==============================] - 19s 62ms/step - loss: 0.4138 - accuracy: 0.8166 - val_loss: 0.4414 - val_accuracy: 0.7950
Epoch 36/100
313/313 [==============================] - 20s 63ms/step - loss: 0.4134 - accuracy: 0.8163 - val_loss: 0.4408 - val_accuracy: 0.7978
Epoch 37/100
313/313 [==============================] - 20s 63ms/step - loss: 0.4130 - accuracy: 0.8164 - val_loss: 0.4397 - val_accuracy: 0.7956
Epoch 38/100
313/313 [==============================] - 20s 63ms/step - loss: 0.4127 - accuracy: 0.8166 - val_loss: 0.4406 - val_accuracy: 0.7952
Epoch 39/100
313/313 [==============================] - 20s 63ms/step - loss: 0.4122 - accuracy: 0.8172 - val_loss: 0.4467 - val_accuracy: 0.7948
Epoch 40/100
313/313 [==============================] - 21s 68ms/step - loss: 0.4118 - accuracy: 0.8152 - val_loss: 0.4398 - val_accuracy: 0.7982
In [35]:
# 훈련 손실과 검증 손실 그래프
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.xlabel('epoch')
plt.ylabel('loss')
plt.legend(['train', 'val'])
plt.show()
In [36]:
# 베스트 모델로 테스트 세트 성능 계산
test_seq = pad_sequences(test_input, maxlen=100)
rnn_model = keras.models.load_model('best-2rnn-model.h5')
rnn_model.evaluate(test_seq, test_target)
782/782 [==============================] - 15s 18ms/step - loss: 0.4308 - accuracy: 0.7982
Out[36]:
[0.43081778287887573, 0.7981600165367126]In [38]:
from IPython.core.display import display, HTML
display(HTML("<style>.container {width:80% !important;}</style>"))
728x90
'Book report > 혼자 공부하는 머신러닝 + 딥러닝' 카테고리의 다른 글
| [혼자 공부하는 머신러닝 + 딥러닝] Chapter 8 이미지를 위한 인공 신경망 (0) | 2021.09.27 |
|---|---|
| [혼자 공부하는 머신러닝 + 딥러닝] Chapter 7 딥러닝을 시작합니다 (0) | 2021.09.04 |
| [혼자 공부하는 머신러닝 + 딥러닝] Chapter 6 비지도 학습 (0) | 2021.09.03 |
| [혼자 공부하는 머신러닝 + 딥러닝] Chapter 5 트리 알고리즘 (0) | 2021.09.01 |
| [혼자 공부하는 머신러닝 + 딥러닝] Chapter 4 다양한 분류 알고리즘 (0) | 2021.08.31 |