Efficient least squares fit of 2D polynomials to data?
I have some data on a regular 2D grid that I would like to decompose into an orthogonal basis function set. What is the most efficient way to do the fit?
Here's some sample code:
import numpy as np
sze = (128,128)
x, y = np.linspace(1, 1, sze[1]), np.linspace(1, 1, sze[0])
xv, yv = np.meshgrid(x, y)
rho = np.sqrt(xv**2 + yv**2)
phi = np.arctan2(yv, xv)
data = (rho * np.cos(phi))
data[rho > 1] = 0
# now have some synthetic data  need to fit to a basis function set that has singleindex members:
# 0: 1
# 1: rho * cos(phi)
# 2: rho * sin(phi)
# ...
# 24: 70 rho^8  140 rho^6 + 90 rho^4  20 rho^2 + 1
See also questions close to this topic

Tensorflow performance: Numpy matrix o TF matrix?
I have the following code:
with tf.Session() as sess: sess.run(init_vars) cols = sess.run(tf.shape(descriptors)[1]) descriptor_matrix = np.zeros((n_batches*batch_size, cols)) while True: batch_descriptor = sess.run(descriptors, feed_dict={dropout_prob: 1}) descriptor_matrix[i:i+elements_in_batch] = np.array(batch_descriptor)
I am mixing tensors and numpy vectors. Does this have an important impact on performance? Why is it? Should I just use tensors instead?

Return dataframe rows where the values in a column are not of type date
I have a dataframe
df
that looks like:Name Date of birth Bob Steve 22/07/1963 Jo pencil Karen 03/02/1953 Frank 29/09/1994
Is there a way to return rows where
Date of birth
is not a date?In the above example I would have returned:
Name Date of birth Bob Jo pencil
Where
Date of birth
is not a date.I can identify where there is a blank value for Date of birth using:
missingDoBError = df.loc[df['Date of birth'].isnull()]
I have tried to find Date of birth values where the value is not a date format at set to NaT by using:
if pd.to_datetime(df['Date of birth'], format='%d%b%Y', errors='coerce').notnull().all():
But I can't get this to work.

How to implement tSNE in a model?
I split my data to train/test. When i use PCA It is straight forward.
from sklearn.decomposition import PCA pca = PCA() X_train_pca = pca.fit_transform(X_train) X_test_pca = pca.transform(X_test)
From here i can use X_train_pca and X_test_pca in the next step and so on..
But when i use tSNE
from sklearn.manifold import TSNE X_train_tsne = TSNE(n_components=2, random_state=0).fit_transform(X_train)
I can't seem to transform the test set so that i can use the tSNE data for the next step e.g. SVM.
Any help?

Calculating the centroids of two superposed gaussian functions
I am trying to find a solution to the following problem. I have a set of points which should model a sum of 2 Gaussian functions centered at different points. I need to find these two points. Up to now my approach has been to find the centroid of the whole set and cut the set of date below and above it. Then I calculate the centroid of each piece and those are my centers. This approach however cuts the information of, say, the left Gaussian which leaks into the right half of the data. This makes the procedure fail when the Gaussians are close together. Is there way to do this more intelligently? Due to the computational difficulty I would prefer if the solution didn't involve curve fitting.

Gaussian fit MATLAB
Hello every on I want to fit the following data excel data in to Guassian model. does any body know how I can do that. I do not know MATLAB but I need this to make how my data fit in a lab report.
X I 0 0.012 2 0.034 4 0.089 6 0.184 8 0.253 10 0.273 12 0.268 14 0.251 16 0.24 18 0.233 20 0.225 22 0.226 24 0.228 26 0.23 28 0.234 30 0.248 32 0.262 34 0.254 36 0.224 38 0.214 40 0.093
I wan to use the following formula

Fitting an ellipse to a set of 2D points
I'm trying to fit an ellipse to a set of points described by x and y coordinates.
I found a detailed explanation of how to do it here http://nicky.vanforeest.com/misc/fitEllipse/fitEllipse.html and tried out the code, but it doesn't seem to work.
It finds the center correctly, but the angle and axes are completely wrong, as you can see in this image: https://i.imgur.com/VLEeNKQ.png
The red points are my data points and the blue aptch is an ellipse drawn from the obtained parameters. Now, the data isn't a perfect ellipse, but the fit is nowhere near it. I'd like to get the fit closer to the actual data.
Here's the code in question.
import numpy as np from numpy.linalg import eig, inv def fitEllipse(x,y): x = x[:,np.newaxis] y = y[:,np.newaxis] D = np.hstack((x*x, x*y, y*y, x, y, np.ones_like(x))) S = np.dot(D.T,D) C = np.zeros([6,6]) C[0,2] = C[2,0] = 2; C[1,1] = 1 E, V = eig(np.dot(inv(S), C)) n = np.argmax(np.abs(E)) a = V[:,n] return a def ellipse_center(a): b,c,d,f,g,a = a[1]/2, a[2], a[3]/2, a[4]/2, a[5], a[0] num = b*ba*c x0=(c*db*f)/num y0=(a*fb*d)/num return np.array([x0,y0]) def ellipse_angle_of_rotation( a ): b,c,d,f,g,a = a[1]/2, a[2], a[3]/2, a[4]/2, a[5], a[0] return 0.5*np.arctan(2*b/(ac)) def ellipse_axis_length( a ): b,c,d,f,g,a = a[1]/2, a[2], a[3]/2, a[4]/2, a[5], a[0] up = 2*(a*f*f+c*d*d+g*b*b2*b*d*fa*c*g) down1=(b*ba*c)*( (ca)*np.sqrt(1+4*b*b/((ac)*(ac)))(c+a)) down2=(b*ba*c)*( (ac)*np.sqrt(1+4*b*b/((ac)*(ac)))(c+a)) res1=np.sqrt(up/down1) res2=np.sqrt(up/down2) return np.array([res1, res2]) def ellipse_angle_of_rotation2( a ): b,c,d,f,g,a = a[1]/2, a[2], a[3]/2, a[4]/2, a[5], a[0] if b == 0: if a > c: return 0 else: return np.pi/2 else: if a > c: return np.arctan(2*b/(ac))/2 else: return np.pi/2 + np.arctan(2*b/(ac))/2
And here's my full dataset. Does anyone have any idea why it's not being fitted correctly?
#  x = np.array([ 5727.53135, 7147.62235, 10330.93573, 8711.17228, 7630.40262, 4777.24983, 4828.27655, 9449.94416, 5203.81323, 6299.44811, 6494.21906]) y = np.array([67157.77567 , 66568.50068 , 55922.56257 , 54887.47348 , 65150.14064 , 66529.91705 , 65934.25548 , 55351.57612 , 63123.5103 , 67181.141725, 56321.36025 ]) #  a = fitEllipse(x,y) center = ellipse_center(a) #phi = ellipse_angle_of_rotation(a) phi = ellipse_angle_of_rotation2(a) axes = ellipse_axis_length(a) # get the individual axes a, b = axes from matplotlib.patches import Ellipse import matplotlib.pyplot as plt ell = Ellipse(center, a, b, phi) fig, ax = plt.subplots(subplot_kw={'aspect': 'equal'}) ax.add_artist(ell) ell.set_clip_box(ax.bbox) ax.set_xlim(0, 100000) ax.set_ylim(0, 100000) plt.show() scat = plt.scatter(x, y, c = "r")

Random Gaussian with standard deviation
I want to intialize a array with random gaussian with standard deviation of 0.08 in python. I was able to find something like this
self.biases = [np.random.randn(y, 1) for y in sizes[1:]]
However it for standard deviation of 1.

Natural spline basis in R
Hi all, I hope you can help me on this problem. I am using the
ns(...)
command in R to generate a base of natural splines, including 1 internal knot
 2 boundary knots
 Intercept equal to
T
Let's say as an example,
ns(1:20, knots=9, Boundary.knots=c(1,15), intercept=T)
. I have some questions that I'm struggling a lot to solve by myself, namely: R returns a basis of 3 polynomials: why 3?
 What is the equation of the 3 polynomials of this basis?
 Natural splines provide the constraint that after the boundary knots the extrapolation is linear (which is not exactly what I would like for my purpose): how should I instead impose the constraint that the extrapolation is actually constant (i.e. flat) after my rightmost boundary knot?
Thank you very much for all your help with this, you would help me a lot!
Best, EM

Accurev undo a revert to basis
In Accurev I have accidently purge a stream by doing a revert to basis while trying to pull the latest from the stream. This removed all the files within the stream. Is it possible to undo it? and how?