6

I am just trying to use pre-trained vgg16 to make prediction in Keras like this.

from scipy import ndimage
from keras.preprocessing import image
from keras.applications.vgg16 import preprocess_input

im = scipy.misc.imread("cat_dog/validation/cats/cat.1362.jpg").astype(np.float32)
im = scipy.misc.imresize(im, (224, 224)).astype(np.float32)

#im /= 255.0
#im = im - np.mean(im, axis=2, keepdims=True)

im = np.expand_dims(im, axis=0)
im = preprocess_input(im)

out = vgg16_model.predict(im)
np.argmax(out)

It seemed that im /= 255.0 give very bad prediction. I commented it out and it started making good prediction. I also added preprocess_input(...) but that doesn't seem to affect prediction for the few random trials I did.

The question is that according to this great blog:

https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html

under "Using the bottleneck features of a pre-trained network: 90% accuracy in a minute", pre-trained VGG16 is in a transfer learning context. And if you look at this gist, you see this line of code:

def save_bottlebeck_features():
    datagen = ImageDataGenerator(rescale=1. / 255)
    etc.

The preprocessing of input seemed to be 1/255.0 during caching of features from the last conv layer. This is sort of puzzling. I further also looked up how preprocess_input(...) is defined in the code I have, and found for 'tf':

x /= 127.5
x -= 1.

Which you can check here.

kawingkelvin
  • 221
  • 1
  • 2
  • 7

2 Answers2

2

The pre-trained weights that are available on Keras are trained with the preprocessing steps defined in preprocess_input() function that is made available for each network architecture (VGG16, InceptionV3, etc).

For example

from keras.applications.vgg16 import preprocess_input

If you are using the weights that comes with keras for fine tuning, then you should use the corresponding preprocess_input() function for the network. It can be different from the original preprocessing steps mentioned in the paper. 

from keras.applications.vgg16 import VGG16
model = VGG16(weights='imagenet', include_top=False)

If you want to stick to the original preprocessing steps, you can find pre-trained weights that is trained with the original preprocessing steps instead of using the weights that comes with Keras.

Maybe at the time of writing the blog post the weights were trained with different preprocessing steps.

Checkout this github issue to learn more.

Arun Ponnusamy
  • 21
  • 4
1

I have a new thinking on this. I think it maybe ok to use a different but reasonable preprocessing (such as 1/255.) in the context of transfer learning (pre-training), than whats originally used to train the VGG16. As long as the top most conv-conv-...pool representation is useful for your new task, and it empirically checked out fine for what F. Chollet did in his blog.

kawingkelvin
  • 221
  • 1
  • 2
  • 7