Mapping the columns of user uploaded excel files
I have a trained model that I want to expose via a webapp to make new predictions based on user uploaded files.
I need the columns of the user's uploaded file (excel, csv) to conform to a set of assumptions/expectations.
I was imagining a sort of wizard that asks the user questions like: "Thanks for uploading your file, now ..."
- What is the name of the column for account balance?
- What is the name of the column for date the account was opened?
This workflow of "User uploads data" -> "Collect additional user input" -> "Do something" seems common enough.
I found this thing:
- and google maps: http://www.cartagram.com/5648/from-excel-to-google-maps/
- and this isn't bad either: https://batchgeo.com/
Any other thoughts on how to keep this relatively painless and maximize user experience?
See also questions close to this topic
Loop Excel Files and Save as PDF - Excel Macro
I am trying to loop through a folder of excel files and for each one re-save as PDF using a VBA excel macro.
The code runs and saves a PDF file however whenever I try to open the file it is unable to open.
To give credit, the original source of the code outline is from www.TheSpreadsheetGuru.com. I have input my code for desired function under the indicated "*****" area.
'Loop through each Excel file in folder Do While myFile <> "" 'Set variable equal to opened workbook Set wb = Workbooks.Open(Filename:=myPath & myFile) 'Ensure Workbook has opened before moving on to next line of code DoEvents *******'Do this to Opened Worksheet***** wb.Activate Dim strFilename, strDirname, strPathname, strDefpath As String On Error Resume Next ' If directory exist goto next line strDirname = Range("A6").Value ' New directory name strFilename = Range("A6").Value & " Rater" & Range("C15").Value 'New file name strDefpath = "C:\Users\Kael\Documents" 'Default path name If IsEmpty(strDirname) Then Exit Sub If IsEmpty(strFilename) Then Exit Sub MkDir strDefpath & "\" & strDirname strPathname = strDefpath & "\" & strDirname & "\" & strFilename 'create total string ActiveWorkbook.SaveAs Filename:=strPathname & ".pdf", _ FileFormat:=xlOpenXMLWorkbookMacroEnabled, Password:="", WriteResPassword:="", _ ReadOnlyRecommended:=False, CreateBackup:=False 'Save and Close Workbook wb.Close SaveChanges:=True 'Ensure Workbook has closed before moving on to next line of code DoEvents 'Get next file name myFile = Dir Loop
If anyone has any input it would be really appreciated and if anymore information/snippet of code is required, let me know!
Excel VBA Skip Variable In Array If Variable Is Nothing When Plotting Graph
I'm trying to make a versatile worksheet that scans for available data in columns and then plots several graphs.
It will first scan for column headers and then inputs its column number as variable for plotting graph later. For example:
A B C D E A C E - - - - - Sometimes, there - - - 1 4 7 1 4 are less data, 1 7 4 2 5 8 2 5 like --> 2 8 5 3 6 9 3 6 3 9 6
I get the column number needed using
Dim findA As Range Set findA = Range("A1:G1").Find(What:="A", _ lookat:=xlWhole, MatchCase:=False, SearchFormat:=False) If Not (findA Is Nothing) Then findA = findA.Column End If
The same goes for
findE. If there are no header "B",
findBwill be return as
I then use array to plot graphs in loop.
v(0) = Array(4, 21, 38, 55, 72) 'graphs' row-location v(1) = Array(findA, findB, findB, findA, findB) 'xaxis column v(2) = Array(findC, findC, findA, findD, findE) 'yaxis column v(3) = Array("A", "B", "B", "A", "B") 'xaxis title v(4) = Array("C", "C", "A", "D", "E") 'yaxis title For i = 0 To 4 Step 1 Set c = ActiveSheet.Shapes.AddChart.Chart With c .Parent.Top = Cells(v(0)(i), 8).Top .Parent.Left = Cells(v(0)(i), 8).Left .ChartType = xlXYScatterSmoothNoMarkers Do While .SeriesCollection.Count > 0 .SeriesCollection(1).Delete Loop .SeriesCollection.NewSeries .SeriesCollection(1).Name = Data -->.SeriesCollection(1).XValues = Range(Cells(3, v(1)(i)), Cells(5, v(1)(m))) -->.SeriesCollection(1).Values = Range(Cells(3, v(2)(i)), Cells(5, v(2)(m))) .Legend.Delete .ChartTitle.Delete .Axes(xlCategory, xlPrimary).HasTitle = True 'x-axis .Axes(xlCategory, xlPrimary).AxisTitle.Caption = v(3)(i) .Axes(xlValue, xlPrimary).HasTitle = True 'y-axis .Axes(xlValue, xlPrimary).AxisTitle.Caption = v(4)(i) .ChartArea.Border.LineStyle = xlNone .PlotArea.Border.LineStyle = msoLineSingle End With Next
Here's the problem. When
Nothing, the highlighted
.SeriesCollectionwill return an error. Is there a method to skip the variable
Nothingor more preferably, leave the graph blank and proceed to the next element in the array?
Create additional columns from data in existing column in Excel
I would like to create additional columns to an existing Excel table while keeping all the data in the other columns same. The new columns I would like to create would come from the data contained in an existing column (the unique values in this column)
For example, I have a "Business Unit" column that contains 5 different business units (e.g. Corporate, North, East, South, West). I would like create five new columns (one for each unit) while retaining the same data attributes associated with the business units in their respective rows. Also would like to note that multiple business units (two - five) could be contained in a row in the Business Unit column (i.e. one row has "East, West" , another row has "Corporate", another has all five). This "Business Unit" column is deliminated by a comma so I could break it out via text to columns.
How to add a custom convert to ruby csv?
Consider a CSV file like below. I would like to change the Credit/Debit field to 0 if they are empty strings and then do some processing.
The roundabout way that I came up with seems inefficient since I read in the csv file, apply the converter and then convert it to CSV::Table for processing. Is there a way to directly apply the converter to parse method ?
csv_data = <<_ Description,Debit,Credit Eating Out,10,"" _ csv = CSV.new(csv_data, headers: true, converters: :integer) csv.convert do |field, info| if info.index == 1 field == "" ? 0 : field end if info.index == 2 field == "" ? 0 : field end end table = CSV::Table.new(csv.to_a) #Seems wrong to convert to array puts table['Debit'] #10 puts table['Credit'] #0
How to convert .csv to.ctt in Windows
I need to convert exported outlook contacts (.csv) to .ctt format in Windows 10. Is there a way to do it? I have change the file extension to .ctt,but to no valid. I see this webspag (http://www.rhys.cx:80/cttconvert/csv2ctt.php) used to do this kind of stuff, but now it is turned off. Thanks.
Unable to write to CSV file in Python
def sampleChecker(inputFile): with open(inputFile, 'r') as infile: reader = csv.DictReader(infile) for row in reader: # row is dictionary for (key, value) in row.items(): # columns dictionary columns[key].append(value) sampleIDs = columns['SampleID'] count = Counter(sampleIDs) incompleteSamples =  incr = 0 reqFields = [['FCID', 'Lane', 'SampleID', 'SampleRef', 'Index', 'Description', 'Control', 'Recipe', 'Operator', 'SampleProject']] for key in count: if count[key] == 1: incompleteSamples.append(key) incr = incr+1 for x in incompleteSamples: with open(inputFile, 'r') as infile: reader = csv.DictReader(infile) for row in reader: for (key, value) in row.items(): if row[key] == x: if row['Lane'] == '2': reqFields.append([row['FCID'], '1', row['SampleID'], row['SampleRef'], row['Index'], row['Description'], row['Control'], row['Recipe'], row['Operator'], row['SampleProject']]) elif row['Lane'] == '1': reqFields.append([row['FCID'], '2', row['SampleID'], row['SampleRef'], row['Index'], row['Description'], row['Control'], row['Recipe'], row['Operator'], row['SampleProject']]) with open(inputFile, 'a') as infile: writer = csv.writer(infile) writer.writerows(reqFields)
Here I am taking a file as input, reading it, creating some missing fields as "reqFields" and trying to write the rec fields to the same file. But at the end of the line "writer.writerows(reqFields)", I am getting the closed file error, any thoughts?
To test replace "inputfile" with a csv file provided in the link below.
Rotate overlaid transparent image on another transparent image in Python 3.6 and Tkinter
I'm working on a simple raspberry pi digital dashboard for a car and i have chosen Python 3.6 as my programming language, and Tkinter as the main GUI library - i am relatively new to both of them.
import tkinter as tk from PIL import Image, ImageTk root = tk.Tk() def currentData(): print("pretend to get values from car") return 99 def my_mainloop(): # get data from car currentValues = currentData() #do more stuff i suppose root.after(my_mainloop) homeFrame = tk.Frame(root, height="600", width="800", bg="black") homeFrame.grid(row=0, column=0) dial_1_path = "gauge1.png" needle_1_path = "needle2.png" '''Open image (GUAGE1) and resize it to proper length''' img = Image.open(dial_1_path, 'r') #open image img2 = img.resize((400, 288), 1) img2 = ImageTk.PhotoImage(img2) '''put image on label and display it on homeFrame''' image_panel = tk.Label(homeFrame, image=img2, bg="black") image_panel.grid(row=0, column=0) '''----------------------------------------------------------------------------------------''' '''Open image (NEEDLE1) and resize it to proper length''' img_needle = Image.open(needle_1_path, 'r') #open image img2_needle = img_needle.resize((200, 17), 1) img2_needle = ImageTk.PhotoImage(img2_needle) #put image on label and dispaly it on homeFrame image_panel2 = tk.Label(homeFrame, image=img2_needle) image_panel2.grid(row=0, column=0) #does not produce desired effect! root.after(100, my_mainloop) root.mainloop()
The way i currently structure the layers of my window is as follows:
- The very bottom layer is the
rootwith a standard background
homeFrameframe is layered above that with a black background
- then a label containing the image of the transparent-background speed gauge is overlaid ontop
homeFramewith a black background
- then finally a label containing the image of the transparent-background gauge needle is overlaid ontop of the gauge in
homeFrameby using grid coordinates
I realize that labels do not support transparency, so:
Q1: are there any containers that support images and transparent backgrounds?
What i simply want is to have the gauge and needle image be two separate objects which can be controlled independently from one another. I need the needle to be able to rotate over the dial but have its transparency work.
What i could do is have the two objects separate in code and rotate the needle individually then merge the images together HOWEVER i need this application to run at 1080p 60Hz on a monitor therefore
Q2: is the merge method i described above an efficient way of creating the effect of a transparency working speed gauge? Multiple gauges need to run on a raspberry pi 3 model B (Quad Core 1.2GHz Broadcom BCM2837 64bit CPU on 1080p 60Hz monitor)
Q3: is Tkinter even the right library for the task? require something that i can program in python. Any other good choice?
I realize that i'm a little all over the place, but im really just trying to wrap my brain around this so any help and guidance at all is greatly appreciated!
- The very bottom layer is the
Programming language that compiles to native code with good job opportunity
I would like to hear from you guys what language to learn that compiles to native code and has good job opportunity. I want to do GUI based programming, not console. Suggestions appreciated. Thanks!
circular div inner shadow on bottom
I want to have inner shadow on bottom semicircle of my circular div but shadow seems moving on wrong edges.
important part from code which does not work:
box-shadow: inset 3px 3px 3px -1px #000;
What I want is slightly different from those in fiddle :
Modeling movement more efficiently - Integrating
In order to construct a robot simulator, I am modeling differential motion. Basically, I need to construct a function which takes input of left and right wheel data (speeds and accelerations) and produces position data. This simulator is going to be used to write code for a real robot so it needs to be as accurate as possible. My current approach is computationally intensive. Is there a more efficient method? How do game engines or physics simulators handle this kind of problem?
Here is the approach I am currently taking:
dt = .05 // sec While simulation is running: speed_left = get speed from user // m/s speed_right = get speed from user // m/s acceleration_left = get acceleration from user // m/s^2 acceleration_right = get acceleration from user // m/s^2 // Integrate left and right accelerations speed_left += acceleration_left * dt speed_right += acceleration_right * dt // Integrate angular velocity to get theta // Using the equation w (angular velocity) = (l - r)/w theta += dt * (speed_left - speed_right)/w // Get linear velocity // Using the equation v = (l + r) / 2 linear_velocity = (speed_left + speed_right) / 2 // Find x and y by breaking theta a y += dt * linear_velocity * sin(theta) x += dt * linear_velocity * cos(theta) update_robots_coordinates_on_screen(x, y) sleep_ms(dt * 1000)
Construct a model without labeled output using machine learning?
I have a data set x, p, q, h, k. I would like to use x as independent variable and a combination (to be found) of p,q,h,k to form a y which is the dependent variable. How can I use machine learning to get a model (combination of p,q,h,k) so that this output y has smallest confidence interval?
I know in supervised learning there must be a labeled y output to train the machine so that a model is formed for prediction. But in my case I don't even know how y should be constructed, because there can be many different combination of p,q,h,k, linear or non-linear. I just want to have a model that the y output has smallest confidence interval(variation). Are there any possible way to use machine learning to solve this problem? Thank you.
Logic Programming modelling relations
I need help trying to sort these relationship out. Its a feature of logic programming
P = NP problem NC = P problem NP = co-NP problem P = BPP problem P = PSPACE problem L = NL problem L = P problem L = RL problem What is the relationship between BQP and NP?