Combine rows of a single file based on multiple matched columns, while retaining unique content by appending additional columns
I'm sure this code exists and I've read through a LOT of pandas / python documentation, in fact perhaps my answer is contained within Pandas, append column based on unique subset of column values but we can't seem to get it to work as below.
Using the example below, if both the Company and Place match, we want to combine the rest of the columns. IF there is unique data, then we would to retain the data in that column and append it as an additional column.
Here is the visual representation of what we need:
See also questions close to this topic
How to train a multi-class image classifier with Keras
I was following the 2 class image classification tutorial here and wanted to convert it into a multi-class classifier.
I am trying to train a model to predict the brand of a watch from 17 classes. My accuracy after 50 epochs is only 21.88% so I'm not really sure where I am going wrong or if I am even doing this right.
Here is my code:
All the images are in their own separate folders under the /data or /valid folders.
Ex: ../watch finder/data/armani
Ex2: ../watch finder/data/gucci
import numpy as np import matplotlib.pyplot as plt import os import cv2 from keras.utils import np_utils from keras.preprocessing.image import ImageDataGenerator from keras.models import Sequential from keras.layers import Conv2D, MaxPooling2D from keras.layers import Activation, Dropout, Flatten, Dense from keras import backend as K import keras.optimizers img_width, img_height = 210, 210 train_data_dir = 'C:/Users/Adrian/Desktop/watch finder/data' validation_data_dir = 'C:/Users/Adrian/Desktop/watch finder/valid' nb_train_samples = 4761 nb_validation_samples = 612 epochs = 50 batch_size = 16 model = Sequential() model.add(Conv2D(32, (3, 3), input_shape=input_shape)) model.add(Activation('relu')) model.add(MaxPooling2D(pool_size=(2, 2))) model.add(Conv2D(32, (3, 3))) model.add(Activation('relu')) model.add(MaxPooling2D(pool_size=(2, 2))) model.add(Conv2D(64, (3, 3))) model.add(Activation('relu')) model.add(MaxPooling2D(pool_size=(2, 2))) model.add(Flatten()) model.add(Dense(64)) model.add(Activation('relu')) model.add(Dropout(0.5)) model.add(Dense(17)) model.add(Activation('softmax')) model.compile(loss='categorical_crossentropy', optimizer='rmsprop', metrics=['accuracy']) train_datagen = ImageDataGenerator( rescale=1. / 255, shear_range=0.2, zoom_range=0.2, horizontal_flip=True) test_datagen = ImageDataGenerator(rescale=1. / 255) train_generator = train_datagen.flow_from_directory( train_data_dir, target_size=(img_width, img_height), batch_size=batch_size, class_mode='categorical') validation_generator = test_datagen.flow_from_directory( validation_data_dir, target_size=(img_width, img_height), batch_size=batch_size, class_mode='categorical') model.fit_generator( train_generator, samples_per_epoch=nb_train_samples // batch_size, epochs=epochs, validation_data=validation_generator, validation_steps=nb_validation_samples // batch_size)
This is my first epoch:
Epoch 1/50 18/18 [==============================] - 8s 422ms/step - loss: 4.1104 - accuracy: 0.0833 - val_loss: 2.8369 - val_accuracy: 0.0592
And this is my 50th/last epoch:
Epoch 50/50 18/18 [==============================] - 7s 404ms/step - loss: 2.4840 - accuracy: 0.2188 - val_loss: 3.0823 - val_accuracy: 0.1795
I am fairly certain I am doing something wrong here but I am really new to deep learning so I'm not sure what that something is. All the help is appreciated.
Also, each brand of watch has between 300-400 images and each image size is the same at 210x210.
Find all possible combinations of even or odd numbers and perform mathematical operations on the result
I am trying to find all possible combinations from taking one integer from each of three lists. The combinations must contain either all even or all odd integers.
I would then like to find the sum of the squares of the integers in each combination.
Finally I would like to create a dictionary that uses this value as the key and the combination it used as its stored value.
E.g. for the combination (1,3,1): the key would be the integer 11 (from the sum (1^2) + (3^2) + (1^2)) and the value stored would be (1,3,1)
My code so far:
lists = [[0,1,2,3,4,5,6],[0,1,2,3,4,5,6],[0,1,2,3,4,5,6]] combos = np.array([list(i) for i in np.array(np.meshgrid(*values)).T.reshape(-1,len(values))])
This takes three lists that each contain the integers 0-6
and uses an element from each list to create a combination of three integers
All possible combinations result: [0,0,0], [0,1,0], [0,2,0], [1,0,0] ... [6,6,6]
Is it possible to import a variable without all other data?
If i want to import a variable from another file , lets say file name is 'USA' and i want to import only the 'population' variable.
I do it by
From USA import population
All is good, all is working. But it also imports everything else from the 'USA' file, which means that if i had print Function variables it will print them too at the file i imported the single variable to..
Is it possible to import a variable without importing all the functions, or at least hide the print functions somehow?
How to apply value_counts to all elments of a DataFrame?
It is easy to apply value_counts to a Series:
In : import pandas as pd In : a = pd.DataFrame([[2,3],[2,2],[3,2],[2,1]]) In : a Out: 0 1 0 2 3 1 2 2 2 3 2 3 2 1 In : a.value_counts() Out: 2 3 3 1 Name: 0, dtype: int64
I need something like
In : a.value_counts() Out: 2 5 3 2 1 1 dtype: int64
'DataFrame' object has no attribute 'value_counts'.
How could I apply value_counts to the element of a DataFrame?
Does it make sense to replace nan values by -99999?
how does it make sense to replace
nanvalues in the dataframe by the value
-99999? I found it here, example 3: https://www.geeksforgeeks.org/python-pandas-dataframe-replace/
df.replace(to_replace = np.nan, value =-99999)
-99999should simply represent
-infinite, but which intention could be behind this operation? Any ideas, guesses? :/
I'd appreciate any advise!
Python datetime problem converting time format
I would like to convert the following time format which is located in a panda dataframe column
100 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400 1500 1600 1700 1800 1900 2000 2100 2200 2300 2400
I would like to transform the previous time format into a standard time format of HH:MM as follow
01:00 02:00 03:00 ... 15:00 16:00 ... 22:00 23:00 00:00
How can I do it in python?
Thank you in advance