Getting nan in tensorflow
I wanted to build a simple machine learning model with tensorflow so that I could understand the process of machine learning and be able do stuff myself.
The dataset that I decided to use are from kaggle.
https://www.kaggle.com/andonians/randomlinearregression/version/2
Since I'm a beginner I didn't want to divide the dataset into 2 parts, namely training and validation.
My code is as follows
train=train.reindex(np.random.permutation(train.index))
train['x']=train['x']/(train['x'].max())
train['y']=train['y']/(train['y'].max())
train_features=np.array(train[['x']])
train_label=np.array(train[['y']])
train_features=tf.convert_to_tensor(train_features)
train_label=tf.convert_to_tensor(train_label)
w=tf.convert_to_tensor(tf.Variable(tf.truncated_normal([1,700],mean=0.0,stddev=1.0,dtype=tf.float64)))
b=tf.convert_to_tensor(tf.Variable(tf.zeros(1,dtype=tf.float64)))
def cal(x,y):
prediction=tf.add(tf.matmul(w,x),b)
error=tf.reduce_mean(tf.square(yprediction))
return [prediction,error]
y,cost=cal(train_features,train_label)
learning_rate=0.05
epochs=3000
init=tf.global_variables_initializer()
optimize=tf.train.GradientDescentOptimizer(learning_rate).minimize(cost)
with tf.Session() as sess:
sess.run(init)
for i in list(range(epochs)):
sess.run(optimize)
print(sess.run(cost))
But the output that I get is nan. nan all the way upto 3000 lines( epochs is 3000).
I can't seem to figure out the reason. I even tried to run it without normalization i.e. without converting the values between 0 and 1. I have also decreased the learning rate to 0.0005. But it seems to have no effect.
Thanks in advance.
P.S. I have not included the test set as I first want to train and see if it works. I will add it later.
See also questions close to this topic

Bugs in PongGame
I recently created a pong game using tkinter. Except for a few sudden bugs, everything seems to be fine.
I tried debugging and changing the code, but it only makes the bugs even worse.
from tkinter import Tk from random import choice import os def clear(): os.system('cls' if os.name == 'nt' else 'clear') class Pong: def __init__(self): self.tk = Tk() self.p1 = [7, 4] self.p2 = [7, 58] self.move = [choice([1, 1]), choice([1, 1])] self.l = 5 self.ball = [9, 31] self.score1 = self.score2 = 0 self.terminal = [ list(f'SCORE 0 : 0'), list(), list('  '), list('x x'), list('x x'), list('x x'), list('x x'), list('x   x'), list('x   x'), list('x  O  x'), list('x   x'), list('x   x'), list('x x'), list('x x'), list('x x'), list('x x'), list('  ') ] def moveUp(self, p): i, j = p l = self.l if i == 3: return self.terminal[i + l  1][j] = ' ' for x in range(i  1, i + l  1): self.terminal[x][j] = '' p[0] = 1 def moveDown(self, p): i, j = p l = self.l if i + l == len(self.terminal)  1: return self.terminal[i][j] = ' ' for x in range(i + 1, i + l + 1): self.terminal[x][j] = '' p[0] += 1 def moveComputer(self): if self.move[0] == 1: self.moveUp(self.p2) if self.move[0] == 1: self.moveDown(self.p2) def moveBall(self): try: if self.terminal[self.ball[0] + self.move[0]][self.ball[1]] == '': self.move[0] *= 1 if self.terminal[self.ball[0]][self.ball[1] + self.move[1]] == 'x': self.terminal[0][24] = str(int(self.terminal[0][24]) + 1) self.terminal[self.ball[0]][self.ball[1]] = ' ' self.ball = [9, 31] self.move = [choice([1, 1]), choice([1, 1])] if self.terminal[self.ball[0]][self.ball[1] + self.move[1]] == '': self.move[1] *= 1 elif self.terminal[self.ball[0] + self.move[0]][self.ball[1] + self.move[1]] == '': self.move[0] *= 1 self.move[1] *= 1 self.terminal[self.ball[0]][self.ball[1]] = ' ' self.ball[0] += self.move[0] self.ball[1] += self.move[1] self.terminal[self.ball[0]][self.ball[1]] = 'O' if ' ' in self.terminal[2][1:1]: i = self.terminal[0][1:1].index(' ') self.terminal[i] = '' if ' ' in self.terminal[1][1:1]: i = self.terminal[1][1:1].index(' ') self.terminal[i] = '' self.print() if self.move[1] == 1: self.moveComputer() except: if (self.terminal[self.ball[0]  1] == '') and (self.terminal[self.ball[0] + 1] == '') or \ (self.terminal[self.ball[0] + 1] == '') and (self.terminal[self.ball[0]  1] == ''): self.move[0] *= 1 else: self.move[0] *= 1 self.move[1] *= 1 self.tk.after(25, self.moveBall) def play(self): self.tk.bind('<Up>', lambda x: self.moveUp(self.p1)) self.tk.bind('<Down>', lambda x: self.moveDown(self.p1)) self.moveBall() self.tk.mainloop() def print(self): clear() print('\n'.join(''.join(i) for i in self.terminal)) pong = Pong() pong.play()
How can I get the bugs removed?
Thanks for you help!
PS: I'm sorry for pasting the entire code, but the bugs could be anywhere.
EDIT: This program should be run on a CMD console

Avoiding 'character argument not in range' python3 decode
I am trying to decode the content of a requests.get() call to a specific url. The url which causes the problem is not always the same across multiple runs of the code, however the part of the requested content which produces the problem has a triple backslash, which gives an error when decoding using 'unicodeescape'.
As a simplified version of the code running in Python 3.6.1
r=b'\xf0\\\xebI' r.decode('unicodeescape').strip().replace('{','\n')
produces the following error:
OverflowError: character argument not in range(0x110000) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "<stdin>", line 1, in <module> OverflowError: decoding with 'unicodeescape' codec failed (OverflowError: character argument not in range(0x110000))
I would like to just skip over the part which produces the error. I am a novice python programmer so any help is greatly appreciated.

Keras Model Network 3D Input to 2D Output
I have a 3D Input (Samples, Steps, Features). So each sample has a chain of steps that have different features. Now, I want a 2D Output (Samples, Steps) where I have samples and at each step of the sample a 0 or a 1 calculated from the model.
So I think it is a sequential binary classification problem. I have some difficult to define the model, especially the output layer.
Here are the shapes of numpy arrays:
x_train.shape (200, 1657, 669)
x_test.shape(41, 1657, 669)
y_train.shape(200, 1657)
y_test.shape(41, 1657)
I tried this model but the output was not the one I was expecting
n_residues, n_features, n_outputs = trainX.shape[1], trainX.shape[2], trainy.shape[1] model = Sequential() model.add(Conv1D(filters=64, kernel_size=3, activation='relu', input_shape=(n_residues,n_features))) model.add(Conv1D(filters=64, kernel_size=3, activation='relu')) model.add(Dropout(0.1)) model.add(MaxPooling1D(pool_size=2)) model.add(Flatten()) model.add(Dense(100, activation='relu')) model.add(Dense(n_outputs, activation='sigmoid')) model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy']) # fit network model.fit(trainX, trainy, epochs=epochs, batch_size=batch_size, verbose=verbose)
Thanks!

How Can I Multiply Two Dynamic Dimension Tensors
I want to get the distance between every input vectors. But for the reason of tensorflow do not have a function to get the distance directly, I have use the formula as (a+b)^2 = a^2 + 2ab + b^2. However, as the dimension of input vector is dynamic(e.g. [None, 64]), I cannot get multiply result correctly. So how can I get the multiply between two dynamic tensor?
Here is the code I used in keras with tensorflow background.
import numpy as np from keras.layers import Input, Conv2D, LeakyReLU, MaxPooling2D, Flatten, Dense, BatchNormalization, Lambda from keras.models import Model from keras import backend as K from keras.layers import Layer from keras import backend as K from keras.engine.topology import Layer from keras.datasets import mnist from keras import optimizers from keras.utils import to_categorical class MyLayer(Layer): def __init__(self, output_dim, distant_parameter = 0.05, **kwargs): self.output_dim = output_dim self.distant_parameter = distant_parameter super(MyLayer, self).__init__(**kwargs) def build(self, input_shape): self.kernel = self.add_weight(name = 'MyKernel', shape = (self.output_dim, input_shape[1]), initializer = 'uniform', trainable = True) super(MyLayer, self).build(input_shape) def call(self, inputs): import tensorflow as tf test = True outputs = () with tf.variable_scope('pairwise_dist'): na = tf.reduce_sum(tf.square(self.kernel), 1) nb = tf.reduce_sum(tf.square(inputs), 1) # nb cannot be calculated accuracy and the output a [none, none, none, ..., none] na = tf.reshape(na, [1, 1]) nb = tf.reshape(nb, [1, 1]) Tul = tf.exp( self.distant_parameter * tf.sqrt(tf.maximum(nb  2*tf.matmul(inputs, self.kernel, False, True) + na, 0.0))) SumTul = tf.reduce_sum(Tul, 1) SumTul = tf.reshape(SumTul, [1, 1]) outputs = tf.divide(Tul, SumTul) return outputs def compute_output_shape(self, input_shape): output_shape = list(input_shape) output_shape[1] = self.output_dim #output_shape[1] = input_shape[0] return tuple(output_shape) def TestModel(): InpLay = Input(shape=(3, 28, 28)) Block1 = Conv2D(7, kernel_size=(3, 3),activation='linear', input_shape=(3, 28, 28), padding='same')(InpLay) Block1 = LeakyReLU(alpha=0.1)(Block1) Block1 = MaxPooling2D((2, 2),padding='same')(Block1) Block2 = Conv2D(14, (3, 3), activation='linear',padding='same')(Block1) Block2 = LeakyReLU(alpha=0.1)(Block2) Block2 = MaxPooling2D(pool_size=(2, 2),padding='same')(Block2) Block3 = Conv2D(28, (3, 3), activation='linear',padding='same')(Block2) Block3 = LeakyReLU(alpha=0.1)(Block3) Block3 = MaxPooling2D(pool_size=(2, 2),padding='same')(Block3) Finals = Flatten()(Block3) Finals = Dense(64, activation='linear')(Finals) Finals = LeakyReLU(alpha=0.1)(Finals) Finals = BatchNormalization(axis = 1)(Finals) Finals = MyLayer(10)(Finals) model = Model(inputs = InpLay, outputs = Finals) model.summary() return model def RWTrain(): #Import MNIST dataset (x_train, y_train),(x_test, y_test) = mnist.load_data() x_new_train = [] for i in range(0, len(x_train)): x_new_train.append([x_train[i], x_train[i], x_train[i]]) y_new_train = to_categorical(y_train) x_new_train = np.array(x_new_train) y_new_train = np.array(y_new_train) #Import Model model = TestModel() sgd = optimizers.SGD(lr=0.02, decay=1e6, momentum=0.9, nesterov=True) #Using Stochastic gradient descent(SGD) for optimizer model.compile(loss='categorical_crossentropy', metrics=['accuracy'], optimizer = sgd) #Training model model.fit(x = x_new_train, y = y_new_train, validation_split=0.1, epochs = 1) model.save_weights("./Output/Model.h5") if __name__ == '__main__': import sys RWTrain()

Get current tensorflow variable value from class
I have defined the following class for linear regression, and in order to see my parameters I tried to get the values, but it does not work, since the weights are a variable and cannot be evaluated without initialize them.
import functools import tensorflow as tf import numpy as np from sklearn import datasets from dataclasses import dataclass @dataclass(frozen=True) class Parameters: weights: tf.Tensor bias: tf.Tensor def lazy_property(function): attribute = '_' + function.__name__ @property @functools.wraps(function) def wrapper(self): if not hasattr(self, attribute): setattr(self, attribute, function(self)) return getattr(self, attribute) return wrapper
class LinearRegression:
def __init__(self, data, target, epoche=100, learning_rate=0.1): self.data = data self.epoche = epoche self.learning_rate = learning_rate self.target=target self.prediction self.cost self.parameters self.optimize @lazy_property def prediction(self): parameter =self.parameters() prediction = tf.add(tf.multiply(self.data, parameter.weights), parameter.bias) return prediction @lazy_property def cost(self): loss=tf.reduce_mean(tf.square(self.prediction  self.target))/2 return loss @lazy_property def optimize(self): optimizer = tf.train.GradientDescentOptimizer(self.learning_rate) return optimizer.minimize(self.cost) @staticmethod def parameters(): weights = tf.Variable(np.random.randn(), name="W", dtype=tf.float32) bias = tf.Variable(np.random.randn(), name="b", dtype=tf.float32) return Parameters(weights, bias) if __name__ == '__main__': diabetes = datasets.load_diabetes() diabetes_X = diabetes.data[:, np.newaxis, 2] diabetes_X_train = diabetes_X[:20] diabetes_X_test = diabetes_X[20:] diabetes_y_train = diabetes.target[:20] diabetes_y_test = diabetes.target[20:] data = tf.placeholder(tf.float32, [None, 1]) target = tf.placeholder(tf.float32, [None, ]) model = LinearRegression(data, target) sess = tf.Session() sess.run(tf.global_variables_initializer()) for epoch in range(30): batch = diabetes_X_train sess.run(model.optimize, {data: batch.data, target: diabetes_y_train}) error = sess.run(model.cost, {data: diabetes_X_train, target: diabetes_y_train}) datas= np.array([[12]]) prediction = sess.run(model.prediction, {data:datas}) print('Epoch {:2d} error {:f}'.format(epoch + 1, error)) print(prediction) print(sess.run(model.parameters().weights))
The last commant end up in the following error: tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value W_1 [[{{node _retval_W_1_0_0}}]]
Could there be defined any other method to obtain the variable from inside the class?

check tensorflow version in CMake
I'm trying to check tensorflow(build from source) version in CMake.
I'm a newbie of CMake. So, I can't find good method. If tensorflow build from source, there are "include"("eager", c_api.h, c_api_experimental.h, LICENSE) folder, and "lib"(libtensorflow.so, libtensorflow_framwork.so) folder.
I tried find_package because of PACKAGE_FIND_VERSION variable. Although TensorFlow_FOUND variable was set, PACKAGE variable was not set. Maybe there need something like .version file:(
The reason I'm trying to do is for version check. My program needs tensorflow 1.10. If there is prebuilt tensorflow already in user(/usr/include, /usr/lib), it should check version if it is 1.10.
Is there any good method for this?
Thank you for reading.

How to implement this simple NN?
I'm watching a lecture on deep learning and it shows (schematically) a model with 2D input ( the input is random and sampled from Gaussian) followed by 12 conv layers to eventually produce an RGB image.
That's all about it, no training at all! he relied on initialization only. The point is to illustrate the concept of NN power to produce complex distributions from a simple distribution.
How can I dot that in Keras? I know it is simple but I can't do it. Is it true that I need DEconvolutional layers?

Multiple Training Jobs in One Instance
Can we train multiple models using one Sagemaker Instance? We are currently having 7 Models for different problems. Can we use one instance for all the models?

Negative eigenvalues in PCA
I've a matrix
x (1000*25)
that contains random floats in the interval(5,5)
.nFeatures=25
andnPoints=1000
. I'm using this code to find the eigenvalues of the covariance matrix, but I'm getting negative eigenvalues.#centering the data for i in range(0,nFeatures): sum=0 for j in range(0,nPoints): sum+=x[j][i] for j in range(0,nPoints): x[j][i]=sum/nPoints #covariance matrix and its eigenvalues & eigenvectors c=np.dot(x.T,x) eValue,eVector=np.linalg.eig(c) print(eValue)
I get the output to be:
[ 1.47374029e+02 8.84275505e13 8.01150077e13 6.77987718e13 5.19228948e13 4.01775609e13 3.55055652e13 3.55433578e13 2.54817200e13 2.51137659e13 2.23836773e13 1.77611044e13 1.57643867e13 1.34409360e13 1.04358065e13 9.31186264e14 8.05736392e14 5.69664362e14 4.39721071e14 3.59268864e14 2.84466680e14 2.42670536e14 2.30979465e15 7.18313504e15 9.35335475e15]
Now since my covariance matrix is a semipositive definite matrix, I the eigenvalues shouldn't be negative. Please help me in identifying what I'm doing wrong.
Also, I read this post, and no, there are no missing values in my dataset.

Cost Function increasing in Multiclassification Neural Network
I am trying to build a Multiclassification neural network from scratch to learn the FashionMNIST dataset, which I got from Kaggle. Here is a link to the dataset.
Here is a link to my code.
I have checked the code many times, but I am unable to find the reason of why the cost function is increasing with number of iterations. Could someone help me debug it.

Downloading an entire git repo to google colab
There is a repository on github which contains image dataset in it. I want to download the entire repository on my google colab. But it does not seem to be working. Here is what I have tried
from google.colab import drive drive.mount('/content/drive') # kaggle is the folder that I created on my drive !cd "drive/My Drive/kaggle" # Downloading the repo on my drive !git clone https://github.com/ardamavi/SignLanguageDigitsDataset.git
But I am getting an error
Cloning into 'SignLanguageDigitsDataset'... fatal: I don't handle protocol '/content/https'
What was expected :
How can I download the entire repo on my drive or in the run time storage of google colab. Either of them will do.
Note : I tried all possible answers that I was getting as suggestion for this question but none of them worked for me.

Is there any alternative way to download kaggle competition data in Colab?
I am trying to use google colab for Kaggle Competitions. However, everything went fine until I tried to download the data. I got 403  Forbidden error. I am able to download other competition's data such as usconsumerfinancecomplaints but not the LANLEarthquakePrediction data.
Initial Step: To move kaggle.json into the folder where the API expects to find it,
!mkdir p ~/.kaggle/ && mv kaggle.json ~/.kaggle/ && chmod 600 ~/.kaggle/kaggle.json
When executing the following code in colab,
!kaggle competitions download c LANLEarthquakePrediction
I got the following error message,
403  Forbidden
I don't know what's wrong, but unable to download the data from Kaggle. Is there anyway we can resolve the problem?