Training plot is not appearing properly for keras model
I have data where I need to train it with X and Y. Traning part is done but when I want to plot the prediction and actual data, it is appearing with so many lines instead of showing just non-linear regression line.
model= Sequential()
model.add(Dense(7,input_dim=1, activation="tanh"))
model.add(Dense(1))
model.compile(loss="mse",
optimizer=tf.keras.optimizers.Adam(learning_rate=0.001), metrics=
["mae"])
history=model.fit(X,Y,epochs=1000)
predict=model.predict(X)
plt.scatter(X, Y,edgecolors='g')
plt.plot(X, predict,'r')
plt.legend([ 'Predictated Y' ,'Actual Y'])
plt.show()
Please see the attached imageplotting image
do you know?
how many words do you know
See also questions close to this topic
-
Python File Tagging System does not retrieve nested dictionaries in dictionary
I am building a file tagging system using Python. The idea is simple. Given a directory of files (and files within subdirectories), I want to filter them out using a filter input and tag those files with a word or a phrase.
If I got the following contents in my current directory:
data/ budget.xls world_building_budget.txt a.txt b.exe hello_world.dat world_builder.specand I execute the following command in the shell:
py -3 tag_tool.py -filter=world -tag="World-Building Tool"My output will be:
These files were tagged with "World-Building Tool": data/ world_building_budget.txt hello_world.dat world_builder.specMy current output isn't exactly like this but basically, I am converting all files and files within subdirectories into a single dictionary like this:
def fs_tree_to_dict(path_): file_token = '' for root, dirs, files in os.walk(path_): tree = {d: fs_tree_to_dict(os.path.join(root, d)) for d in dirs} tree.update({f: file_token for f in files}) return treeRight now, my dictionary looks like this:
key:''.In the following function, I am turning the empty values
''into empty lists (to hold my tags):def empty_str_to_list(d): for k,v in d.items(): if v == '': d[k] = [] elif isinstance(v, dict): empty_str_to_list(v)When I run my entire code, this is my output:
hello_world.dat ['World-Building Tool'] world_builder.spec ['World-Building Tool']But it does not see
data/world_building_budget.txt. This is the full dictionary:{'data': {'world_building_budget.txt': []}, 'a.txt': [], 'hello_world.dat': [], 'b.exe': [], 'world_builder.spec': []}This is my full code:
import os, argparse def fs_tree_to_dict(path_): file_token = '' for root, dirs, files in os.walk(path_): tree = {d: fs_tree_to_dict(os.path.join(root, d)) for d in dirs} tree.update({f: file_token for f in files}) return tree def empty_str_to_list(d): for k, v in d.items(): if v == '': d[k] = [] elif isinstance(v, dict): empty_str_to_list(v) parser = argparse.ArgumentParser(description="Just an example", formatter_class=argparse.ArgumentDefaultsHelpFormatter) parser.add_argument("--filter", action="store", help="keyword to filter files") parser.add_argument("--tag", action="store", help="a tag phrase to attach to a file") parser.add_argument("--get_tagged", action="store", help="retrieve files matching an existing tag") args = parser.parse_args() filter = args.filter tag = args.tag get_tagged = args.get_tagged current_dir = os.getcwd() files_dict = fs_tree_to_dict(current_dir) empty_str_to_list(files_dict) for k, v in files_dict.items(): if filter in k: if v == []: v.append(tag) print(k, v) elif isinstance(v, dict): empty_str_to_list(v) if get_tagged in v: print(k, v) -
Actaully i am working on a project and in it, it is showing no module name pip_internal plz help me for the same. I am using pycharm(conda interpreter
File "C:\Users\pjain\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "C:\Users\pjain\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code exec(code, run_globals) File "C:\Users\pjain\AppData\Local\Programs\Python\Python310\Scripts\pip.exe\__main__.py", line 4, in <module> File "C:\Users\pjain\AppData\Local\Programs\Python\Python310\lib\site-packages\pip\_internal\__init__.py", line 4, in <module> from pip_internal.utils import _logI am using pycharm with conda interpreter.
-
Looping the function if the input is not string
I'm new to python (first of all) I have a homework to do a function about checking if an item exists in a dictionary or not.
inventory = {"apple" : 50, "orange" : 50, "pineapple" : 70, "strawberry" : 30} def check_item(): x = input("Enter the fruit's name: ") if not x.isalpha(): print("Error! You need to type the name of the fruit") elif x in inventory: print("Fruit found:", x) print("Inventory available:", inventory[x],"KG") else: print("Fruit not found") check_item()I want the function to loop again only if the input written is not string. I've tried to type return Under print("Error! You need to type the name of the fruit") but didn't work. Help
-
Only download certain label tf dataset
Looking to do some fine tuning. The dataset (found here: https://knowyourdata-tfds.withgoogle.com/#dataset=sun397&filters=kyd%2Fsun397%2Flabel:%2Fh%2Fhouse&tab=ITEM&select=kyd%2Fsun397%2Flabel&item=%2Fh%2Fhouse%2Fsun_blpzjomvikwtulrq.jpg&expanded_groups=sun397) that Im trying to finetune w is pretty large and i just want to use/download the images with label /h/house. Any tips on how I can best accomplish this? Thanks!
import tensorflow as tf import tensorflow_hub as hub import tensorflow_datasets as tfds import numpy as np import matplotlib.pyplot as plt import functools import pandas (train_ds, valid_ds), info = tfds.load("sun397", split=["train", "validation"], as_supervised=True, with_info=True, label = "/h/house") int_to_class_label = info.features['label'].int2str -
TFF: How can I train any model using a server running tff-runtime and a client running tff-client?
I read all the tensorflow-federated tutorials, including this one https://www.tensorflow.org/federated/gcp_setup, but I couldn't understand how to use this for training a model.
I'm doing a graduation project, to start I need to do this POC using tensorflow-federated to train a model with one server and one client in order to apply cross-silo setup for recognition of organs affected by covid in the future. If anyone can point me a direction, I'd be very grateful.
-
Can't use Keras MeanIoU to train semantic segmentation model
I'm working on a binary semantic segmentation problem. I built an UNet model with MobileNetV2 backbone. Here is my model code:
def upsample(filters, size, apply_dropout=False): initializer = tf.random_normal_initializer(0., 0.02) layer = Sequential() layer.add(layers.Conv2DTranspose(filters, size, strides=2, padding='same', kernel_initializer=initializer, use_bias=False)) layer.add(layers.BatchNormalization()) if apply_dropout: layer.add(layers.Dropout(0.5)) layer.add(layers.ReLU()) return layer def UNet(image_size, num_classes): inputs = Input(shape=image_size + (3,)) base_model = applications.MobileNetV2(input_shape=image_size + (3,), include_top=False) layer_names = [ 'block_1_expand_relu', 'block_3_expand_relu', 'block_6_expand_relu', 'block_13_expand_relu', 'block_16_project', ] base_model_outputs = [base_model.get_layer(name).output for name in layer_names] down_stack = Model(inputs=base_model.input, outputs=base_model_outputs) down_stack.trainable = False up_stack = [ upsample(512, 3), upsample(256, 3), upsample(128, 3), upsample(64, 3) ] skips = down_stack(inputs) x = skips[-1] skips = reversed(skips[:-1]) for up, skip in zip(up_stack, skips): x = up(x) x = layers.Concatenate()([x, skip]) outputs = layers.Conv2DTranspose(filters=num_classes, kernel_size=3, strides=2, padding='same')(x) return Model(inputs, outputs)To load the images and masks for training, I built an image loader inherits from
keras.Sequnce.class ImageLoader(utils.Sequence): def __init__(self, batch_size, img_size, img_paths, mask_paths): self.batch_size = batch_size self.img_size = img_size self.img_paths = img_paths self.mask_paths = mask_paths def __len__(self): return len(self.mask_paths) // self.batch_size def __getitem__(self, idx): i = idx * self.batch_size batch_img_paths = self.img_paths[i:i + self.batch_size] batch_mask_paths = self.mask_paths[i:i + self.batch_size] x = np.zeros((self.batch_size,) + self.img_size + (3,), dtype='float32') for j, path in enumerate(batch_img_paths): img = utils.load_img(path, target_size=self.img_size) img = utils.img_to_array(img) x[j] = img y = np.zeros((self.batch_size,) + self.img_size + (1,), dtype='uint8') for j, path in enumerate(batch_mask_paths): img = utils.load_img(path, target_size=self.img_size, color_mode='grayscale') img = utils.img_to_array(img) # [0, 255] -> [0, 1] img //= 255 y[j] = img return x, yIn my segmentation problem, all the labels are in the range [0, 1]. However, when I try to compile and then fit the model using Adam optimizer, Sparse categorical cross entropy loss and metric
tf.keras.metrics.MeanIoU, I encountered with the following problem:Node: 'confusion_matrix/assert_non_negative_1/assert_less_equal/Assert/AssertGuard/Assert' 2 root error(s) found. (0) INVALID_ARGUMENT: assertion failed: [`predictions` contains negative values. ] [Condition x >= 0 did not hold element-wise:] [x (confusion_matrix/Cast:0) = ] [-1 -1 -1...] [[{{node confusion_matrix/assert_non_negative_1/assert_less_equal/Assert/AssertGuard/Assert}}]] [[confusion_matrix/assert_less_1/Assert/AssertGuard/pivot_f/_31/_67]] (1) INVALID_ARGUMENT: assertion failed: [`predictions` contains negative values. ] [Condition x >= 0 did not hold element-wise:] [x (confusion_matrix/Cast:0) = ] [-1 -1 -1...] [[{{node confusion_matrix/assert_non_negative_1/assert_less_equal/Assert/AssertGuard/Assert}}]]At first, I used accuracy as a metrics for training and I didn't encounter this problem, however when I changed to MeanIoU, this problem happened. Does anyone know how to fix this problem? Thank you very much!
UPDATE: I've searched on StackOverflow and found this question about a similar error, however the fix mentioned in that link (reduce learning rate) doesn't work in my case.
-
How to deal with the colorbar axis space in matplotlib subplots
I am plotting seven different parameters over four seasons, as shown in below image. but on last column (Post-Monsoon) sub_plots axis compromised with colorbar axis, that is really awkward!!
import matplotlib.pyplot as plt import cartopy import cartopy.crs as ccrs from cartopy.mpl.gridliner import LONGITUDE_FORMATTER, LATITUDE_FORMATTER import warnings warnings.filterwarnings('ignore') k = [1,5,9,13,17,21,25] k1= [25,26,27,28] k2= [4,8,12,16,20,24,28] k3= [1,2,3,4] k4=[25] S=['Winter','Pre-monsoon','Monsoon','Post-Monsoon'] fig=plt.figure(figsize=(13,11), dpi=300) for i in range(1,29): ax = fig.add_subplot(7,4,i, projection=ccrs.PlateCarree()) ax.set_extent([39.9,100.5,-0.5,25.5],ccrs.PlateCarree()) ax.add_feature(cartopy.feature.COASTLINE) ax.add_feature(cartopy.feature.BORDERS, linestyle='-') ax.add_feature(cartopy.feature.LAND, zorder=100, edgecolor='k') ax.set_xticks([40,50,60,70,80,90,100], crs=ccrs.PlateCarree()) ax.set_yticks([0,5,10,15,20,25], crs=ccrs.PlateCarree()) ax.tick_params(axis='x', length=7, width=1, bottom=True, top=True) ax.tick_params(axis='y', length=7, width=1, right=True, left=True) ax.yaxis.set_major_formatter(plt.NullFormatter()) ax.xaxis.set_major_formatter(plt.NullFormatter()) A=D[i-1].plot.pcolormesh(ax=ax, cmap='seismic', transform=ccrs.PlateCarree(), add_colorbar=False,add_labels=False) # D contains list of parameters to be plotted if i in k: gl = ax.gridlines(draw_labels=True, linestyle='--') gl.xlabels_top=False gl.xlabels_bottom=False gl.ylabels_right=False gl.xformatter=LONGITUDE_FORMATTER gl.yformatter=LATITUDE_FORMATTER gl.xlabel_style={'size':10,} gl.ylabel_style={'size':10,} ax.tick_params(axis='y', length=7, width=1, left=False) if i in k1: gl = ax.gridlines(draw_labels=True, linestyle='--') gl.xlabels_top=False gl.xlabels_bottom=True gl.ylabels_right=False gl.ylabels_left=False gl.xformatter=LONGITUDE_FORMATTER gl.yformatter=LATITUDE_FORMATTER gl.xlabel_style={'size':10,} gl.ylabel_style={'size':10,} ax.tick_params(axis='x', length=7, width=1, bottom=False, top=True) ax.tick_params(axis='y', length=7, width=1, right=True, left=True) if i in k4: ax.tick_params(axis='y', length=7, width=1, right=True, left=False) if i in k2: ax.tick_params(axis='y', length=7, width=1, right=False) fig.colorbar(A,ax=ax, shrink=0.5) # Here is the Colorbar option if i in k3: ax.tick_params(axis='x', length=7, width=1,top=False) ax.title.set_text(S[i-1]) fig.tight_layout(h_pad=0) plt.show()How to adjust colorbar without distortion of the last axis, Any remark from the community🙏 Thanks
-
Place ipywidget Dropdown on screen in matplotlib.pyplot axes ax1
When I run this code, the Dropdown is nowhere to be found. How do I make it appear in ax1? (I think I would like to avoid Tkinter.)
#!/usr/bin/env python3 #Place ipywidget Dropdown on screen in matplotlib.pyplot axes ax1 import matplotlib.pyplot as plt from ipywidgets.widgets import Dropdown, Layout fig = plt.figure() axcolor = 'lightgoldenrodyellow' # I want a fig with 1 axes--left rectangle with Dropdown (later I will have 4 other axes) # instance the axes [left, bottom, width, height]. ax1 = fig.add_axes([.1, .4, .4, .4]) #([.92, .97, .08, .03]) # Dropdown is from ipywidgets def STypeGrp_event_handler(change): print(change.new) STypeGrp.value = "" STypeGrp = Dropdown( options=list(['E1', 'E2', 'F1']), value='E1', description='Type-Grp:', layout=Layout(width='100px', margin = '20px 20px 20px 20px') ) STypeGrp.observe(STypeGrp_event_handler) #widgets.interact(STypeGrp_event_handler, value=STypeGrp) plots = [STypeGrp] # ?? How do I put STypeGrp dropdown into the ax1 frame?? figManager = plt.get_current_fig_manager() figManager.window.showMaximized() plt.show() -
how to print all parameters of a keras model
I am trying to print all the 1290 parameters in
dense_1layer, butmodel.get_weights()[7]only show 10 parameters. How could I print all the 1290 parameters ofdense_1layer? What is the difference betweenmodel.get_weights()andmodel.layer.get_weights()>model.get_weights()[7] array([-2.8552295e-04, -4.3254648e-03, -1.8752701e-04, 2.3482188e-03, -3.4848123e-04, 7.6121779e-04, -2.7494309e-06, -1.9068648e-03, 6.0777756e-04, 1.9550985e-03], dtype=float32) >model.summary() Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= conv2d (Conv2D) (None, 26, 26, 32) 320 conv2d_1 (Conv2D) (None, 24, 24, 64) 18496 max_pooling2d (MaxPooling2D (None, 12, 12, 64) 0 ) dropout (Dropout) (None, 12, 12, 64) 0 flatten (Flatten) (None, 9216) 0 dense (Dense) (None, 128) 1179776 dropout_1 (Dropout) (None, 128) 0 dense_1 (Dense) (None, 10) 1290 _________________________________________________________________ ================================================================= Total params: 1,199,882 Trainable params: 1,199,882 Non-trainable params: 0 _________________________________________________________________ -
Is there a way in scikit library to calculate the MSR of the model?
I have trained a regression multioutput neural network model predicting two dependant variables and have estimated the quality of the prediction by calculating the MSE using the scikit provided mean_squared_error metric. I was asked to calculate the MSR too. Reading about it I understood that while MSE is SSE/number of degree of freedom, MSR is SSR/number of degree of freedom. I was wondering if a similar function or working function exists to calculate MSR.
-
Predicted values for conditional logistic regression greater than 1
I have a multivariate conditional logistic regression model. Case and controls are matched on a 1 to many basis.
I want to make predictions using the model. However, the predicted values I keep getting are between 0 and 3 when they should be binary (0 or 1). Why don't I get binary values?
This is my data: survival1 is binary. IC is also binary. Test_intensity-cat is categorical with 4 levels. Herd_size_cat is also categorical with 4 levels. Group has groups in the original data set. I've just included 11 here. The subset of data doesn't converge but the original set does.
n survival1 IC test_intensity_cat herd_size_cat group 1 0 none 1.2 < test/yr <= 1.5 medium 628 2 0 none <= 1.2 test/yr medium 629 3 0 none >= 2 tests/yr very large 627 4 1 IC >= 2 tests/yr very large 628 5 0 none <= 1.2 test/yr large 627 6 1 IC >= 2 tests/yr very large 627 7 1 none >= 2 tests/yr very large 627 8 0 none 1.5 < test/yr <= 2 large 629 9 0 IC 1.5 < test/yr <= 2 very large 629 10 0 none 1.5 < test/yr <= 2 large 628 11 0 none <= 1.2 test/yr large 628 12 0 none 1.5 < test/yr <= 2 small 231 13 0 none 1.5 < test/yr <= 2 very large 231 14 0 none 1.2 < test/yr <= 1.5 very large 231 15 0 IC 1.5 < test/yr <= 2 very large 231 16 1 none >= 2 tests/yr very large 170 17 0 none 1.2 < test/yr <= 1.5 very large 170 18 0 none >= 2 tests/yr very large 170 19 1 none >= 2 tests/yr medium 582 20 0 none 1.5 < test/yr <= 2 small 583 21 0 IC 1.5 < test/yr <= 2 large 582 22 1 none >= 2 tests/yr large 583 23 0 none 1.2 < test/yr <= 1.5 very large 134 24 0 none 1.2 < test/yr <= 1.5 very large 134 25 0 none <= 1.2 test/yr small 134 26 0 IC 1.5 < test/yr <= 2 very large 134 27 0 none 1.2 < test/yr <= 1.5 very large 484 28 0 none >= 2 tests/yr very large 485 29 0 IC 1.5 < test/yr <= 2 medium 484 30 0 none 1.5 < test/yr <= 2 large 485 31 0 none 1.5 < test/yr <= 2 small 484 32 0 IC <= 1.2 test/yr very large 485 33 0 none 1.2 < test/yr <= 1.5 very large 484 34 0 none 1.5 < test/yr <= 2 very large 485 35 0 none <= 1.2 test/yr medium 76 36 0 none >= 2 tests/yr very large 76 37 0 none >= 2 tests/yr large 76 38 0 IC >= 2 tests/yr medium 76 39 0 none <= 1.2 test/yr very large 629 40 0 none 1.5 < test/yr <= 2 medium 582 41 0 IC >= 2 tests/yr large 170 42 1 IC 1.2 < test/yr <= 1.5 small 583 43 0 none 1.5 < test/yr <= 2 small 582 44 0 none <= 1.2 test/yr small 583This is my code is in R studio:
library(tidyverse) library(broom) library(survival) model_IC_intensity_size <- clogit(survival1 ~ IC + test_intensity_cat + herd_size_cat + strata(group), method = "exact", data = LCT_herd_matched) actual <- LCT_herd_matched$survival1 predicted <- round(predict(model_IC_intensity_size, type = "expected")) table(predicted, actual)This is the output with the original dataset. The subset gives a smaller version that includes an aberrant 2.
actual predicted 0 1 0 9271 641 1 185 434 2 6 42 3 0 2I'm also want to calculate leverage, delta chi squared and the delta beta statistics (p 425 Veterinary Epidemiologic Research by Dohoo et al). How would I go about determining these diagnostics for a conditional logistic regression model in R?
