Faster way to patchify a picture to overlapping blocks

2018-11-08 06:33 Moran Reznik imported from Stackoverflow

I'm looking for a FAST (and if possible memory afficiant) way to rewrite a function I crerated as part of Visual bag of words algorithm:

def get_pic_patches(pic, l, s):  # "s" stands for stride

    r, c = pic.shape
    i, j = [0, 0]
    x_range = list(range(0, r, s  ) )
    y_range = list(range(0, c ,  s ) )
    patches = []
    patches_location = []
    for x in x_range:  # without last two since it will exceed dimensions
        for y in y_range:  # without last two since it will exceed dimensions
            if x+ l<= r and y+l <= c:
                patch = pic[x:x +  l , y:y + l ]
                patches_location.append([x, y])  # patch location is the upper left pixel
                patches.append(  patch   )

    return patches, patches_location

it takes a grayscale image (NOT RGB!), desired patch length and stride value, and gives back all patches as a list of numpy array.

On other qestions, I found this:

def patchify(img, patch_shape):
    img = np.ascontiguousarray(img)  # won't make a copy if not needed
    X, Y = img.shape
    x, y = patch_shape
    shape = ((X-x+1), (Y-y+1), x, y) # number of patches, patch_shape 
    strides = img.itemsize*np.array([Y, 1, Y, 1])
    return np.lib.stride_tricks.as_strided(img, shape=shape, strides=strides)

in order to get to return a list, I used it like this:

def patchify(img, patch_shape):
    img = np.ascontiguousarray(img)  # won't make a copy if not needed
    X, Y = img.shape
    x, y = patch_shape
    shape = ((X-x+1), (Y-y+1), x, y) # number of patches, patch_shape 
    strides = img.itemsize*np.array([Y, 1, Y, 1])
    patches = np.lib.stride_tricks.as_strided(img, shape=shape, strides=strides)
    a,b,c,d = patches.shape
    patches = patches.reshape(((a*b),c,d))
    patches = patches.tolist()
    return

but this was actually much slower than my original function! another problem is that is only works with stride = 1, and I want to be able to use all sorts of stride values.

See also questions close to this topic

Python spline with non-regular x values?
I have a set of x and y coordinates
```
x = [1,2,4,9,10]
y = [3,4,5,10,1]
```
And I plot them using:
```
import matplotlib.pyplot as plt
import numpy as np
plt.plot(x,y)
plt.show()
```
But I want to have a smooth image, so I can use spline:
```
from scipy.interpolate import BSpline
new_x = np.linspace(min(x),max(x),100)
new_y = BSpline(y, new_x, 2)
```
But this assumes the x coordinates are uniformly distributed (they lie on equal distances apart), but mine do not. How do I fix that?

Operands could not broadcast together

I'm trying to train a model using mini batches but I'm having a .... error.

I'm using the same function I've already used (and it worked) with other models, but this time is crashing.

def random_mini_batches(X, Y, mini_batch_size = 64):
"""
Creates a list of random minibatches from (X, Y)

Arguments:
X -- input data, of shape (input size, number of examples)
Y -- true "label" vector (1, number of examples)
mini_batch_size - size of the mini-batches, integer

Returns:
mini_batches -- list of synchronous (mini_batch_X, mini_batch_Y)
"""

m = X.shape[1]                  # number of training examples
mini_batches = []

# Step 1: Shuffle (X, Y)
permutation = list(np.random.permutation(m))
shuffled_X = X.iloc[:, permutation]
shuffled_Y = Y[:, permutation].reshape((Y.shape[0],m))

# Step 2: Partition (shuffled_X, shuffled_Y). Minus the end case.
num_complete_minibatches = math.floor(m/mini_batch_size) # number of mini batches of size mini_batch_size in your partitionning
for k in range(0, num_complete_minibatches):
    mini_batch_X = shuffled_X.iloc[:, k * mini_batch_size : k * mini_batch_size + mini_batch_size]
    mini_batch_Y = shuffled_Y[:, k * mini_batch_size : k * mini_batch_size + mini_batch_size]
    mini_batch = (mini_batch_X, mini_batch_Y)
    mini_batches.append(mini_batch)

# Handling the end case (last mini-batch < mini_batch_size)
if m % mini_batch_size != 0:
    mini_batch_X = shuffled_X.iloc[:, num_complete_minibatches * mini_batch_size : m]
    mini_batch_Y = shuffled_Y[:, num_complete_minibatches * mini_batch_size : m]
    mini_batch = (mini_batch_X, mini_batch_Y)
    mini_batches.append(mini_batch)

return mini_batches

I've used this function in a NN with 20 layers and this X and Y: And now I'm trying use it again with a 5 layer NN and shapes

However, I'm getting this error at this part of the code epoch_cost += minibatch_cost/num_minibatches.

The full code would be this:

        for epoch in range(num_epochs):

        epoch_cost = 0
        num_minibatches = int(m / minibatch_size) # number of minibatches of size minibatch_size in the train set

        minibatches = random_mini_batches(X_train, Y_train, minibatch_size)

        for minibatch in minibatches:

            #Select a minibatch
            (minibatch_X, minibatch_Y) = minibatch

            _, minibatch_cost = sess.run([optimizer, cost], feed_dict = {X: minibatch_X, Y: minibatch_Y})

            epoch_cost += minibatch_cost/num_minibatches

    # Print the cost every epoch
        if print_cost == True and epoch % 100 == 0:
            print("Cost after epoch %i: %f" % (epoch, epoch_cost))
        if print_cost == True and epoch % 5 == 0:
            costs.append(epoch_cost)

Thanks in advance

"Unknown label type" with numpy and scikit

X = np.array(...,dtype=np.float64).astype(np.float)
y = np.array(...,dtype=np.float64).astype(np.float)

print(X)
[[-0.5  0. ]
 [ 0.   0. ]
 [ 0.   0. ]
 [ 0.   0. ]
 [ 0.  -3.5]
 [-3.5 -3.5]
 [-3.5 -2. ]
 [-2.   0. ]
 [ 0.   0. ]
 [ 0.  -3. ]]
print(y)
[-3.5 -3.5 -3.5 -3.5 -3.5 -2.  -3.  -3.  -3.   1. ]
(10, 2)
(10,)


from sklearn.naive_bayes import GaussianNB
clf = GaussianNB()
clf.fit(X, Y)

No matter what I do I get

Traceback (most recent call last):
  File "scripts/wave-pool.py", line 174, in <module>
    clf.fit(X, y)
  File "/home/ben/.local/lib/python2.7/site-packages/sklearn/naive_bayes.py", line 192, in fit
    sample_weight=sample_weight)
  File "/home/ben/.local/lib/python2.7/site-packages/sklearn/naive_bayes.py", line 355, in _partial_fit
    if _check_partial_fit_first_call(self, classes):
  File "/home/ben/.local/lib/python2.7/site-packages/sklearn/utils/multiclass.py", line 320, in _check_partial_fit_first_call
    clf.classes_ = unique_labels(classes)
  File "/home/ben/.local/lib/python2.7/site-packages/sklearn/utils/multiclass.py", line 96, in unique_labels
    raise ValueError("Unknown label type: %s" % repr(ys))
ValueError: Unknown label type: (array([-3.5, -3. , -2. ,  1. ]),)

Python - machine learning: Creating a training and test set from a list of arrays

I would like to create a neural network trained on the RAVDESS dataset (https://smartlaboratory.org/ravdess/): the idea is to use this dataset to detect to mood of the person speaking to the microphone of my application.

Using librosa and the for loop below, I have extracted the labels and the features I want to use for the analysis.

# I started with only one folder to fasten the operations
oneActorPath = '/content/drive/My Drive/RAVDESS/Audio_Speech_Actors_01-24/Actor_01/'
lst = []

# Loop through each folder to find the wavs

for subdir, dirs, files in os.walk(oneActorPath):
  for file in files:
    if file == '.DS_Store':
      continue
    else:
      # Check if the format of the file is valid
      try:
        #Load librosa array
        data, rate = librosa.load(os.path.join(subdir,file))
        #Using the name of the file I can understand the emotion that contains
        file = file[6:8]
        arr = data, file
        lst.append(arr)
        #print(list)

      # If is not valid, skip it
      except ValueError:
        continue

The output of this loop is a list of arrays in the format below:

[(array([-8.1530527e-10,  8.9952795e-10, -9.1185753e-10, ...,
          0.0000000e+00,  0.0000000e+00,  0.0000000e+00], dtype=float32),
  '08'),
 (array([0., 0., 0., ..., 0., 0., 0.], dtype=float32), '08'),
 (array([0., 0., 0., ..., 0., 0., 0.], dtype=float32), '06'),
 (array([-0.00050612, -0.00057967, -0.00035985, ...,  0.        ,
          0.        ,  0.        ], dtype=float32), '05'),
 (array([ 6.8139506e-08, -2.3837963e-05, -2.4622474e-05, ...,
          3.1678758e-06, -2.4535689e-06,  0.0000000e+00], dtype=float32),
  '05'),
 (array([ 0.0000000e+00,  0.0000000e+00,  0.0000000e+00, ...,
          6.9306935e-07, -6.6020442e-07,  0.0000000e+00], dtype=float32),
  '04'),
 (array([-7.30260945e-05, -1.18022966e-04, -1.08280736e-04, ...,
          8.83421380e-05,  4.97258679e-06,  0.00000000e+00], dtype=float32),
  '06'),
 (array([0., 0., 0., ..., 0., 0., 0.], dtype=float32), '07'),
 (array([ 2.3406714e-05,  3.1186773e-05,  4.9467826e-06, ...,
          1.2180173e-07, -9.2944845e-08,  0.0000000e+00], dtype=float32),
  '01'),
 (array([ 1.1845550e-06, -1.6399191e-06,  2.5565218e-06, ...,
         -8.7445065e-09,  5.9859917e-09,  0.0000000e+00], dtype=float32),
  '04'),
 (array([0., 0., 0., ..., 0., 0., 0.], dtype=float32), '03'),
 (array([-1.3284328e-05, -7.4090644e-07,  7.2679302e-07, ...,
          0.0000000e+00,  0.0000000e+00,  0.0000000e+00], dtype=float32),
  '07'),
 (array([ 0.0000000e+00,  0.0000000e+00,  0.0000000e+00, ...,
          5.0694009e-08, -3.4546797e-08,  0.0000000e+00], dtype=float32),
  '03'),
 (array([ 1.5591205e-07, -1.5845627e-07,  1.5362870e-07, ...,
          0.0000000e+00,  0.0000000e+00,  0.0000000e+00], dtype=float32),
  '01'),
 (array([0., 0., 0., ..., 0., 0., 0.], dtype=float32), '03'),
 (array([0.0000000e+00, 0.0000000e+00, 0.0000000e+00, ..., 1.1608539e-05,
         8.2463991e-09, 0.0000000e+00], dtype=float32), '03'),
 (array([-3.6192148e-07, -1.4590451e-05, -5.3999561e-06, ...,
         -1.9935460e-05, -3.4417746e-05,  0.0000000e+00], dtype=float32),
  '02'),
 (array([ 0.0000000e+00,  0.0000000e+00,  0.0000000e+00, ...,
         -2.5319534e-07,  2.6521766e-07,  0.0000000e+00], dtype=float32),
  '02'),
 (array([ 0.0000000e+00,  0.0000000e+00,  0.0000000e+00, ...,
         -2.5055220e-08,  1.2936166e-08,  0.0000000e+00], dtype=float32)
...

The second element of each element of the list above ('08' in the first line) represents the label of the dataset according to the dictionary below

emotions = {
    "neutral": "01",
    "calm": "02",
    "happy": "03",
    "sad": "04",
    "angry": "05", 
    "fearful": "06", 
    "disgust": "07", 
    "surprised": "08"
}

At this point, I have my labels and my data: how can I split this dataset to obtain a training and a test set?

EDIT1: I need to understand how to obtain X and y from this structure to use train_test_split on the data.

Multiclass Classification using Xtreme Gradient Boosting XGB in R
Error in 'params' of XGB for multi-class classification of variable 'NSP' into classes 1,2,3.

Dataset: https://archive.ics.uci.edu/ml/datasets/cardiotocography

label_error >= 0 && label_error < nclass SoftmaxMultiClassObj: label must be in [0, num_class), num_class=3 but found 3 in label.

objective used: multi:softmax
```
eval_metric = mlogloss

nc=3 # (for labels 1,2,3)
```
ValueError: No way to determine width or height from video. Need `-s` in `inputdict`. Consult documentation on I/O
the following below is my code,while i run the code , got an error like "ValueError: No way to determine width or height from video. Need -s in inputdict. Consult documentation on I/O."
```
vid_path = '/home/dataset/MSVD/YouTubeClips/'
vid = skvideo.io.vread(my_dir)
# Select 16 frames from video
vid = vid[40:56]
x = preprocess_input(vid) 
```
opencv NULL guiReceiver (please create a window) in function 'cvDestroyWindow

I'm trying to execute a training code for computer vision and I keep getting this error:

cv2.error: OpenCV(3.4.3) /io/opencv/modules/highgui/src/window_QT.cpp:539: error: (-27:Null pointer) NULL guiReceiver (please create a window) in function 'cvDestroyWindow

Any help will be great
Are there pre-trained model to recognize character?

I extracted from number-plate into individual characters as picture.

https://i.stack.imgur.com/k09UB.jpg

I wonder there are pre-trained CNN model to this recognize characters. If any, please recommend for me API to perform this. Thanks a lot.

Related questions

Faster way to patchify a picture to overlapping blocks

See also questions close to this topic