Reshaping tensor in custom loss function in Keras

I'm getting None type when reshaping the tensor. This happens when compiling the model with loss function and the optimizer (before starting the training). What do I do?

Error:

TypeError: Failed to convert object of type <class 'tuple'> to Tensor. Contents: (None, -1). Consider casting elements to a supported type.  

Custom loss function:

def custom_loss(y_true, y_pred):

        y_pred = K.reshape(y_pred, (K.get_variable_shape(y_pred)[0], -1))
        y_true = K.reshape(y_true, (K.get_variable_shape(y_true)[0], -1))
        y_pred = K.std(y_pred, axis=0)
        y_true = K.std(y_true, axis=0)
        loss = (1/2) * (y_pred - y_true) ** 2
        loss = K.mean(loss)

        return loss

2 answers

  • answered 2018-11-08 08:05 Gabe

    This happens because your y_true or y_pred tensors shape is not defined properly. None means here that the tensors shape is not strictly set, but it can vary based on the previous operations what we can't see. Or you just initialized your tensor like this.

    How to fix it:

    • First you should investigate how the y_true or y_pred gets its shape and avoid getting a None shape, so the tensors will have a deterministic number of rows and cols

    Example:

    Your loss function works for proper inputs:

    import tensorflow as tf
    from keras import backend as K
    
    
    def custom_loss(y_true, y_pred):
        y_pred = K.reshape(y_pred, (K.get_variable_shape(y_pred)[0], -1))
        y_true = K.reshape(y_true, (K.get_variable_shape(y_true)[0], -1))
        y_pred = K.std(y_pred, axis=0)
        y_true = K.std(y_true, axis=0)
        loss = (1 / 2) * (y_pred - y_true) ** 2
    
        return loss
    
    
    a = tf.constant([[1.0, 2., 3.]])
    b = tf.constant([[1., 2., 3.]])
    loss = custom_loss(a, b)
    loss = tf.Print(loss, [loss], "loss")
    
    with tf.Session() as sess:
        _ = sess.run([loss])
    

    But when using a placeholder where I have not defined the shape, will throw the same exception

    a = tf.placeholder(tf.float32, (None, 32))
    

  • answered 2018-11-08 08:09 jeevaa_v

    I fixed it:

    def custom_loss(y_true, y_pred):
    
            y_pred = K.reshape(y_pred, (K.shape(y_pred)[0], -1))
            y_true = K.reshape(y_true, (K.shape(y_true)[0], -1))
            y_pred = K.std(y_pred, axis=0)
            y_true = K.std(y_true, axis=0)
            loss = (1/2) * (y_pred - y_true) ** 2
            loss = K.mean(loss)
    
            return loss
    

    The problem was with finding the first dimension of the y_true and y_predict. During compilation, it will not get a true shape and hence it returns a None value. So, instead of getting the integer value of the shape, I get the tf.Tensor shape. I changed K.get_variable_shape(y_true) to just K.shape(y_true) and it solves.