Keras for predicting the digit in a specific position in a number

2022-01-17 23:27 sm535 imported from Stackoverflow

I am using Keras to train and then predict the digit in the i-th place of a n-digit decimal number. For example, if the input is 123456 (i.e., n = 6), and i=2, then the program needs to find out a value of 4. I save the inputs in x_values and the training, testing and validation values corresponding to the elements of x_values are in y_values.

# Generate x_values by reading the first column from the csv
from io import StringIO
df = pd.read_csv(dst_path, skipinitialspace=True)

x_values = df['Input']
y_values = df['OutputA'] # OutputA indicates 0th element (i.e., i = 0)

I have two models. The first one is simple 'relu' which takes a scalar input and feeds it through 8 "neurons", followed by a final layer is a single neuron.

# Using Keras to create a simple model architecture
model_1 = tf.keras.Sequential()

# First layer takes a scalar input and feeds it through 8 "neurons". The
# neurons decide whether to activate based on the 'relu' activation function.
model_1.add(keras.layers.Dense(8, activation='relu', input_shape=(1,)))

# Final layer is a single neuron, since we want to output a single value
model_1.add(keras.layers.Dense(1))

# Compile the model using the standard 'adam' optimizer and the mean squared error or 'mse' loss function for regression.
model_1.compile(optimizer='adam', loss='mse', metrics=['mae'])

The second model is 16 neurons x 32 deep (considerably larger than the first one).

model = tf.keras.Sequential()

# First layer takes a scalar input and feeds it through 16 "neurons". The
# neurons decide whether to activate based on the 'relu' activation function.
NUM_NEURONS = 16
model.add(keras.layers.Dense(NUM_NEURONS, activation='relu', input_shape=(1,)))

# The new layers will help the network learn more complex representations
NUM_LAYERS = 30
for layer in range(NUM_LAYERS):
  model.add(keras.layers.Dense(NUM_NEURONS, activation='relu'))

# Final layer is a single neuron, since we want to output a single value
model.add(keras.layers.Dense(1))

# Compile the model using the standard 'adam' optimizer and the mean squared error or 'mse' loss function for regression.
model.compile(optimizer='adam', loss="mse", metrics=["mae"])

My overall training set has 15000 samples. I split the input set into training, testing then validation sets.

# Train the model on our training data while validating on our validation set
history_1 = model_1.fit(x_train, y_train, epochs=500, batch_size=64,
                        validation_data=(x_validate, y_validate))

After all these, when I calculate the Loss and the Mean Average Error, they are both very high (about 8.3 and 2.5 resp.).

What would be the best way to solve this problem? Playing with the width, depth of the network or the value of epoch did not have discernible effect. Any hint will be greatly appreciated.

See also questions close to this topic

Python File Tagging System does not retrieve nested dictionaries in dictionary

I am building a file tagging system using Python. The idea is simple. Given a directory of files (and files within subdirectories), I want to filter them out using a filter input and tag those files with a word or a phrase.

If I got the following contents in my current directory:

data/
   budget.xls
   world_building_budget.txt
a.txt
b.exe
hello_world.dat
world_builder.spec

and I execute the following command in the shell:

py -3 tag_tool.py -filter=world -tag="World-Building Tool"

My output will be:

These files were tagged with "World-Building Tool":
data/ world_building_budget.txt
hello_world.dat
world_builder.spec

My current output isn't exactly like this but basically, I am converting all files and files within subdirectories into a single dictionary like this:

def fs_tree_to_dict(path_):
    file_token = ''
    for root, dirs, files in os.walk(path_):
        tree = {d: fs_tree_to_dict(os.path.join(root, d)) for d in dirs}
        tree.update({f: file_token for f in files})
        return tree

Right now, my dictionary looks like this: key:''.

In the following function, I am turning the empty values '' into empty lists (to hold my tags):

def empty_str_to_list(d):
    for k,v in d.items():
        if v == '':
            d[k] = []
        elif isinstance(v, dict):
            empty_str_to_list(v)

When I run my entire code, this is my output:

hello_world.dat ['World-Building Tool']
world_builder.spec ['World-Building Tool']

But it does not see data/world_building_budget.txt. This is the full dictionary:

{'data': {'world_building_budget.txt': []},  
'a.txt': [],
'hello_world.dat': [], 
'b.exe': [], 
'world_builder.spec': []}

This is my full code:

import os, argparse


def fs_tree_to_dict(path_):
    file_token = ''
    for root, dirs, files in os.walk(path_):
        tree = {d: fs_tree_to_dict(os.path.join(root, d)) for d in dirs}
        tree.update({f: file_token for f in files})
        return tree


def empty_str_to_list(d):
    for k, v in d.items():
        if v == '':
            d[k] = []
        elif isinstance(v, dict):
            empty_str_to_list(v)


parser = argparse.ArgumentParser(description="Just an example", formatter_class=argparse.ArgumentDefaultsHelpFormatter)

parser.add_argument("--filter", action="store", help="keyword to filter files")
parser.add_argument("--tag", action="store", help="a tag phrase to attach to a file")
parser.add_argument("--get_tagged", action="store", help="retrieve files matching an existing tag")

args = parser.parse_args()

filter = args.filter
tag = args.tag
get_tagged = args.get_tagged
current_dir = os.getcwd()

files_dict = fs_tree_to_dict(current_dir)
empty_str_to_list(files_dict)

for k, v in files_dict.items():
    if filter in k:
        if v == []:
            v.append(tag)
            print(k, v)

        elif isinstance(v, dict):
            empty_str_to_list(v)

    if get_tagged in v:
        print(k, v)

Actaully i am working on a project and in it, it is showing no module name pip_internal plz help me for the same. I am using pycharm(conda interpreter

 File "C:\Users\pjain\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\pjain\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Users\pjain\AppData\Local\Programs\Python\Python310\Scripts\pip.exe\__main__.py", line 4, in <module>
  File "C:\Users\pjain\AppData\Local\Programs\Python\Python310\lib\site-packages\pip\_internal\__init__.py", line 4, in <module>
    from pip_internal.utils import _log

I am using pycharm with conda interpreter.

Looping the function if the input is not string

I'm new to python (first of all) I have a homework to do a function about checking if an item exists in a dictionary or not.

inventory = {"apple" : 50, "orange" : 50, "pineapple" : 70, "strawberry" : 30}

def check_item():
    x = input("Enter the fruit's name: ")
    if not x.isalpha():
        print("Error! You need to type the name of the fruit")
    elif x in inventory:
        print("Fruit found:", x)
        print("Inventory available:", inventory[x],"KG")
    else:
        print("Fruit not found")
check_item()

I want the function to loop again only if the input written is not string. I've tried to type return Under print("Error! You need to type the name of the fruit") but didn't work. Help

Only download certain label tf dataset
Looking to do some fine tuning. The dataset (found here: https://knowyourdata-tfds.withgoogle.com/#dataset=sun397&filters=kyd%2Fsun397%2Flabel:%2Fh%2Fhouse&tab=ITEM&select=kyd%2Fsun397%2Flabel&item=%2Fh%2Fhouse%2Fsun_blpzjomvikwtulrq.jpg&expanded_groups=sun397) that Im trying to finetune w is pretty large and i just want to use/download the images with label /h/house. Any tips on how I can best accomplish this? Thanks!
```
import tensorflow as tf
import tensorflow_hub as hub
import tensorflow_datasets as tfds
import numpy as np
import matplotlib.pyplot as plt
import functools
import pandas

(train_ds, valid_ds), info = tfds.load("sun397", split=["train", "validation"], as_supervised=True, with_info=True, label = "/h/house")
int_to_class_label = info.features['label'].int2str
```
TFF: How can I train any model using a server running tff-runtime and a client running tff-client?

I read all the tensorflow-federated tutorials, including this one https://www.tensorflow.org/federated/gcp_setup, but I couldn't understand how to use this for training a model.

I'm doing a graduation project, to start I need to do this POC using tensorflow-federated to train a model with one server and one client in order to apply cross-silo setup for recognition of organs affected by covid in the future. If anyone can point me a direction, I'd be very grateful.

Can't use Keras MeanIoU to train semantic segmentation model

I'm working on a binary semantic segmentation problem. I built an UNet model with MobileNetV2 backbone. Here is my model code:

def upsample(filters, size, apply_dropout=False):
    initializer = tf.random_normal_initializer(0., 0.02)
    layer = Sequential()
    layer.add(layers.Conv2DTranspose(filters, size, strides=2, padding='same', kernel_initializer=initializer,
                                     use_bias=False))
    layer.add(layers.BatchNormalization())
    if apply_dropout:
        layer.add(layers.Dropout(0.5))
    layer.add(layers.ReLU())
    return layer


def UNet(image_size, num_classes):
    inputs = Input(shape=image_size + (3,))

    base_model = applications.MobileNetV2(input_shape=image_size + (3,), include_top=False)
    layer_names = [
        'block_1_expand_relu',
        'block_3_expand_relu',
        'block_6_expand_relu',
        'block_13_expand_relu',
        'block_16_project',
    ]
    base_model_outputs = [base_model.get_layer(name).output for name in layer_names]
    down_stack = Model(inputs=base_model.input, outputs=base_model_outputs)
    down_stack.trainable = False

    up_stack = [
        upsample(512, 3),
        upsample(256, 3),
        upsample(128, 3),
        upsample(64, 3)
    ]

    skips = down_stack(inputs)
    x = skips[-1]
    skips = reversed(skips[:-1])

    for up, skip in zip(up_stack, skips):
        x = up(x)
        x = layers.Concatenate()([x, skip])

    outputs = layers.Conv2DTranspose(filters=num_classes, kernel_size=3, strides=2, padding='same')(x)

    return Model(inputs, outputs)

To load the images and masks for training, I built an image loader inherits from keras.Sequnce.

class ImageLoader(utils.Sequence):

    def __init__(self, batch_size, img_size, img_paths, mask_paths):
        self.batch_size = batch_size
        self.img_size = img_size
        self.img_paths = img_paths
        self.mask_paths = mask_paths

    def __len__(self):
        return len(self.mask_paths) // self.batch_size

    def __getitem__(self, idx):
        i = idx * self.batch_size
        batch_img_paths = self.img_paths[i:i + self.batch_size]
        batch_mask_paths = self.mask_paths[i:i + self.batch_size]

        x = np.zeros((self.batch_size,) + self.img_size + (3,), dtype='float32')
        for j, path in enumerate(batch_img_paths):
            img = utils.load_img(path, target_size=self.img_size)
            img = utils.img_to_array(img)
            x[j] = img

        y = np.zeros((self.batch_size,) + self.img_size + (1,), dtype='uint8')
        for j, path in enumerate(batch_mask_paths):
            img = utils.load_img(path, target_size=self.img_size, color_mode='grayscale')
            img = utils.img_to_array(img)
            # [0, 255] -> [0, 1]
            img //= 255
            y[j] = img

        return x, y

In my segmentation problem, all the labels are in the range [0, 1]. However, when I try to compile and then fit the model using Adam optimizer, Sparse categorical cross entropy loss and metric tf.keras.metrics.MeanIoU, I encountered with the following problem:

Node: 'confusion_matrix/assert_non_negative_1/assert_less_equal/Assert/AssertGuard/Assert'
2 root error(s) found.
  (0) INVALID_ARGUMENT:  assertion failed: [`predictions` contains negative values.  ] [Condition x >= 0 did not hold element-wise:] [x (confusion_matrix/Cast:0) = ] [-1 -1 -1...]
     [[{{node confusion_matrix/assert_non_negative_1/assert_less_equal/Assert/AssertGuard/Assert}}]]
     [[confusion_matrix/assert_less_1/Assert/AssertGuard/pivot_f/_31/_67]]
  (1) INVALID_ARGUMENT:  assertion failed: [`predictions` contains negative values.  ] [Condition x >= 0 did not hold element-wise:] [x (confusion_matrix/Cast:0) = ] [-1 -1 -1...]
     [[{{node confusion_matrix/assert_non_negative_1/assert_less_equal/Assert/AssertGuard/Assert}}]]

At first, I used accuracy as a metrics for training and I didn't encounter this problem, however when I changed to MeanIoU, this problem happened. Does anyone know how to fix this problem? Thank you very much!

UPDATE: I've searched on StackOverflow and found this question about a similar error, however the fix mentioned in that link (reduce learning rate) doesn't work in my case.

how to print all parameters of a keras model

I am trying to print all the 1290 parameters in dense_1 layer, but model.get_weights()[7] only show 10 parameters. How could I print all the 1290 parameters of dense_1 layer? What is the difference between model.get_weights() and model.layer.get_weights()

>model.get_weights()[7]
array([-2.8552295e-04, -4.3254648e-03, -1.8752701e-04,  2.3482188e-03,
       -3.4848123e-04,  7.6121779e-04, -2.7494309e-06, -1.9068648e-03,
        6.0777756e-04,  1.9550985e-03], dtype=float32)

>model.summary()
Model: "sequential"
_________________________________________________________________
Layer (type)                Output Shape              Param #   
=================================================================
 conv2d (Conv2D)             (None, 26, 26, 32)        320       
                                                                 
 conv2d_1 (Conv2D)           (None, 24, 24, 64)        18496     
                                                                 
 max_pooling2d (MaxPooling2D  (None, 12, 12, 64)       0         
 )                                                               
                                                                 
 dropout (Dropout)           (None, 12, 12, 64)        0         
                                                                 
 flatten (Flatten)           (None, 9216)              0         
                                                                 
 dense (Dense)               (None, 128)               1179776   
                                                                 
 dropout_1 (Dropout)         (None, 128)               0         
                                                                 
 dense_1 (Dense)             (None, 10)                1290      
_________________________________________________________________
                                                                 
=================================================================
Total params: 1,199,882
Trainable params: 1,199,882
Non-trainable params: 0
_________________________________________________________________

Training plot is not appearing properly for keras model

I have data where I need to train it with X and Y. Traning part is done but when I want to plot the prediction and actual data, it is appearing with so many lines instead of showing just non-linear regression line.

model= Sequential()
model.add(Dense(7,input_dim=1, activation="tanh"))
model.add(Dense(1))

model.compile(loss="mse", 
optimizer=tf.keras.optimizers.Adam(learning_rate=0.001), metrics= 
["mae"])
history=model.fit(X,Y,epochs=1000)

predict=model.predict(X)
plt.scatter(X, Y,edgecolors='g')
plt.plot(X, predict,'r')
plt.legend([ 'Predictated Y' ,'Actual Y'])
plt.show()

Please see the attached imageplotting image

Print accuracy for YOLOv5 model?

I want to print the accuracy of my trained model. I do understand when I use the model through the camera it shows me the accuracy on the box only, but I want to see the accuracy of my whole model rather than a single instance of my image.

cap  = cv2.VideoCapture(0)

# loop through labels
for label in labels:
    print('Collecting images for {}'.format(label)) #so that we can see when transitioning through images
    time.sleep(5)  #sleep or wait for 5 seconds when transitioning between image for other class
    
    # Loop through image range
    for img_num in range(number_imgs):
        print('Collecting images for {}, image number {}'.format(label, img_num))
        
        # webcam feed
        ret, frame = cap.read()   # read the feed from the web cam and store it in given vars
        
        # Naming image path
        imgname = os.path.join(IMAGES_PATH, label+'.'+str(uuid.uuid1())+'.jpg')
        
        #Writes out image to file
        cv2.imwrite(imgname,frame)
        
        # Render to screen
        cv2.imshow('Image Collection', frame)
        
        # 2 sec delay between diff captures
        time.sleep(2)
        
        if cv2.waitKey(10) & 0xFF == ord('q'):
            break
cap.release()
cv2.destroyAllWindows()

!cd yolov5 && python train.py --img 320 --batch 16 --epochs 500 --data dataset.yml --weights yolov5s.pt

model = torch.hub.load('ultralytics/yolov5', 'custom', path='yolov5/runs/train/exp/weights/last.pt', force_reload=True)
while cap.isOpened():
    ret, frame = cap.read()
    
    # Make detections 
    results = model(frame)
    
    cv2.imshow('YOLO', np.squeeze(results.render()))
    
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break
        
cap.release()
cv2.destroyAllWindows()
cv2.waitKey(1)

keyerror 0 during cpu trainning

unknow error

I am not sure what is the reason, probably my index exceed the limit of the input? But how can I solve this error? Here is the code I used https://github.com/wanyu-lin/ICML2021-Gem

how can I modify Dataset class to make the mask RCNN work with multiple objects?

I am currently working on instance segmentation. I follow these two tutorials:

However, these two tutorials work perfectly with one class like person + background. But in my case, I have two classes like a person and car + background. I didn't find any resources about making the Mask RCNN work with multiple objects.

Notice that:

I am using PyTorch ( torchvision ), torch==1.10.0+cu111 torchvision==0.11.0+cu111 torchaudio==0.10.0
I am using a Pascal VOC annotation
i used segmentation class (not the XML file) + the images

and this is my dataset class

class PennFudanDataset(torch.utils.data.Dataset):
def __init__(self, root, transforms=None):
    self.root = root
    self.transforms = transforms
    # load all image files, sorting them to
    # ensure that they are aligned
    self.imgs = list(sorted(os.listdir(os.path.join(root, "img"))))
    self.masks = list(sorted(os.listdir(os.path.join(root, "imgMask"))))

def __getitem__(self, idx):
    # load images ad masks
    img_path = os.path.join(self.root, "img", self.imgs[idx])
    mask_path = os.path.join(self.root, "imgMask", self.masks[idx])
    img = Image.open(img_path).convert("RGB")
    # note that we haven't converted the mask to RGB,
    # because each color corresponds to a different instance
    # with 0 being background
    mask = Image.open(mask_path)

    mask = np.array(mask)
    # instances are encoded as different colors
    obj_ids = np.unique(mask)
    # first id is the background, so remove it
    obj_ids = obj_ids[1:]

    # split the color-encoded mask into a set
    # of binary masks
    masks = mask == obj_ids[:, None, None]

    # get bounding box coordinates for each mask
    num_objs = len(obj_ids)
    boxes = []
    for i in range(num_objs):
        pos = np.where(masks[i])
        xmin = np.min(pos[1])
        xmax = np.max(pos[1])
        ymin = np.min(pos[0])
        ymax = np.max(pos[0])
        boxes.append([xmin, ymin, xmax, ymax])

    boxes = torch.as_tensor(boxes, dtype=torch.float32)
    # there is only one class
    labels = torch.ones((num_objs,), dtype=torch.int64) 
    masks = torch.as_tensor(masks, dtype=torch.uint8)

    image_id = torch.tensor([idx])
    area = (boxes[:, 3] - boxes[:, 1]) * (boxes[:, 2] - boxes[:, 0])
    # suppose all instances are not crowd
    iscrowd = torch.zeros((num_objs,), dtype=torch.int64)

    target = {}
    target["boxes"] = boxes
    target["labels"] = labels
    target["masks"] = masks
    target["image_id"] = image_id
    target["area"] = area
    target["iscrowd"] = iscrowd
    if self.transforms is not None:
        img, target = self.transforms(img, target)

    return img, target

def __len__(self):
    return len(self.imgs)

anyone can help me?

Will ReLU lose negative value of data?

I am building a neural network with Keras and the input data is standard using StandardScaler(). I was wonder if the first layer I used has an activation function RELU, will the negative part of value from StandardScaler lost in training? (because it basically set all negative values to zero) I understand that it will first go through WX+b function before go into RELU, but just curious. Or is there any general rules for activation functions between dense layers?

pytorch Error: module 'torch.nn' has no attribute 'ReLu'

i am working in google colab, so i assume its the current version of pytorch. I tried this:

class Fc(nn.Module):
    def __init__(self):
        super(Fc, self).__init__()
        self.flatt = nn.Flatten()
        self.seq = nn.Sequential(nn.Linear(28*28, 512),
                                 nn.ReLU(),
                                 nn.Linear(512, 512),
                                 nn.ReLu(),
                                 nn.Linear(512, 10), nn.ReLu())


    def forward(x):
        p = self.flatt(x)
        p = self.seq(p)
        return p
m1 = Fc()

and got:

<ipython-input-85-142a1e77b6b6> in <module>()
----> 1 m1 = Fc()
<ipython-input-84-09df3be0b613> in __init__(self)
  4         self.flatt = nn.Flatten()
  5         self.relu = torch.nn.modules.activation.ReLU()
----> 6         self.seq = nn.Sequential(nn.Linear(28*28, 1012), nn.ReLU(), 
nn.Linear(1012, 512), nn.ReLu(), nn.Linear(512, 10), nn.ReLu())
AttributeError: module 'torch.nn' has no attribute 'ReLu'

What I am doing wrong here?

Dying relu produces weight and bias gradients stuck at 0?

I am trying to understand the issue of dying relu neurons. If I use a learning rate which is too high, my model produces weight and bias gradients which remain at 0 throughout training.

I understand that during gradient descent, the new weight and bias values are pushed into the negative value range, as old weight values are subtracted from the learning rate.

However, I am failing to understand how this leads to the weights not updating. Is it because they don't pass the threshold for neuron activation, and therefore not updated?

Related questions

Keras for predicting the digit in a specific position in a number

See also questions close to this topic