Keras custom loss function: Accessing current input pattern

In Keras (with Tensorflow backend), is the current input pattern available to my custom loss function?

The current input pattern is defined as the input vector used to produce the prediction. For example, consider the following: X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42, shuffle=False). Then the current input pattern is the current X_train vector associated with the y_train (which is termed y_true in the loss function).

When designing a custom loss function, I intend to optimize/minimize a value that requires access to the current input pattern, not just the current prediction.

I’ve taken a look through https://github.com/fchollet/keras/blob/master/keras/losses.py

I’ve also looked through “Cost function that isn’t just y_pred, y_true?“

I am also familiar with previous examples to produce a customized loss function:

import keras.backend as K

def customLoss(y_true,y_pred):
    return K.sum(K.log(y_true) - K.log(y_pred))

Presumably (y_true,y_pred) are defined elsewhere. I’ve taken a look through the source code without success and I’m wondering whether I need to define the current input pattern myself or whether this is already accessible to my loss function.

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

You can wrap the loss function as a inner function and pass your input tensor to it (as commonly done when passing additional arguments to the loss function).

def custom_loss_wrapper(input_tensor):
    def custom_loss(y_true, y_pred):
        return K.binary_crossentropy(y_true, y_pred) + K.mean(input_tensor)
    return custom_loss

input_tensor = Input(shape=(10,))
hidden = Dense(100, activation='relu')(input_tensor)
out = Dense(1, activation='sigmoid')(hidden)
model = Model(input_tensor, out)
model.compile(loss=custom_loss_wrapper(input_tensor), optimizer='adam')

You can verify that input_tensor and the loss value (mostly, the K.mean(input_tensor) part) will change as different X is passed to the model.

X = np.random.rand(1000, 10)
y = np.random.randint(2, size=1000)
model.test_on_batch(X, y)  # => 1.1974642

X *= 1000
model.test_on_batch(X, y)  # => 511.15466

Method 2

You can use add_loss to pass external layers to your loss, in your case the input tensor.

Here an example:

def CustomLoss(y_true, y_pred, input_tensor):
    return K.binary_crossentropy(y_true, y_pred) + K.mean(input_tensor)

X = np.random.uniform(0,1, (1000,10))
y = np.random.randint(0,2, 1000)

inp = Input(shape=(10,))
hidden = Dense(100, activation='relu')(inp)
out = Dense(1, activation='sigmoid')(hidden)
target = Input((1,))
model = Model([inp,target], out)

model.add_loss( CustomLoss( target, out, inp ) )
model.compile(loss=None, optimizer='adam')
model.fit(x=[X,y], y=None, epochs=3)

To use the model in inference mode (removing the target from inputs)

final_model = Model(model.input[0], model.output)
final_model.predict(X)

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0