Python : Multiple excel files in folders and sub-folders
I have a folder with the following structure,
Folder |_ File1.xlsx |_ File1.xlsx ... |_ Folder2.xlsx |_File3.xlsx |_File4.xlsx
There are three aims that I want to fulfill,
List out all files in folders and sub-folders.
Open all files one-by-one.
- Copy the columns 1, 3 and 8 and store it in another excel file.
As of now, I have successfully done (1) and the code is as follows, import os
rootdir = os.getcwd()
for subdir, dirs, files in os.walk(rootdir): for file in files: filepath = subdir + os.sep + file if filepath.endswith(".xlsx"): print (filepath)
However, I am stuck at (2), I have tried to think of ways (and google) to get around it and have failed.
Any kind of help will be appreciated.
Thanks in advance.
See also questions close to this topic
How to Delete COMPLETELY Blank Rows VBA
In my macro, I want to delete completely blank Rows. Not Rows that has something in one column because that is still valuable information.
I have looked all over the internet but if someone could come up with a function that deletes ENTIRELY blank rows and not just rows with something missing in a few columns that would be awesome.
(Information is only on Columns A through N and roughly 7 thousand rows). I haven't developed any code for this because I am really stumped.
Summing csv columns
I have a csv file with a couple columns and about 100k rows. One of the columns is a date, and I was wondering what the easiest way was to count the number of rows that have a certain date for all the possible dates and make a new csv file with just the date and the number of rows that have that date in the specific column. Any language or method is fine!
How do I import Subscript and Superscript from an excel file to mySQL Database?
We have a lot of chemistry MCQs in an excel file with lot of chemistry formulas and equations. When we tried to import these MCQs into mySQL based database all subscripts used in formulas lost and taken as normal character e.g, H2SO4, CO2, CH3 etc and makes our formulas and elements name nonsense. Is there any solution by which mySQL can import subscript and superscript as it is from excel? Or Is there any solution so we can force excel to consider subscript and superscript as inserted symbols because mySQL can pick symbols as it is?
gcloud ml-engine with python3.5 tkinter import error on run
I'm trying to run my model on Google cloud ml-engine with:
gcloud ml-engine jobs submit training $NAME --module-name train.task_w2v \ --package-path train --runtime-version 1.8 --python-version 3.5 \ --scale-tier BASIC --staging-bucket $BUCKET --region $REGION
And this is my setup.py:
from setuptools import find_packages from setuptools import setup REQUIRED_PACKAGES = ['numpy', 'tensorflow', 'pandas', 'matplotlib', 'opencv-python', 'PyYAML', 'coloredlogs', 'scikit-learn', 'scipy', 'matplotlib'] setup( name='ConvMultiAttention', version='0.9', author='name', install_requires=REQUIRED_PACKAGES, packages=find_packages(), include_package_data=True, )
The model runs fine locally and gets successfully build:
I master-replica-0 Successfully installed model-0.9 coloredlogs-10.0 cycler-0.10.0 humanfriendly-4.15.1 kiwisolver-1.0.1 matplotlib-2.2.2 opencv-python-188.8.131.52 pyparsing-2.2.0 master-replica-0
I master-replica-0 Running command: python3 -m train.task_w2v master-replica-0
But then it gets this exception:
master-replica-0 Traceback (most recent call last): File "/usr/lib/python3.5/tkinter/__init__.py", line 36, in import _tkinter ImportError: No module named '_tkinter'
master-replica-0 Command '['python3', '-m', 'train.task_w2v']' returned non-zero exit status 1
Since my understanding is that tkinter is part of python3.5 I don't really know what goes wrong here, or what to do. I tried to run it without matplotlib and with a lower tf version, but the problems consisted.
Also I get these warnings:
google-cloud-spanner 0.29.0 has requirement requests<3.0dev,>=2.18.4, but you'll have requests 2.13.0 which is incompatible.
The script humanfriendly is installed in '/root/.local/bin' which is not on PATH.
Which I don't really know how to handle, or if I even need to.
Getting Youtube Favorites/Likes of a channel with API
By using Youtube API, in Python 3.5.2, I want to fetch the likes/favorites of a giving channel id. Here the code I used:
channel_details = youtube.channels().list(id=channel_id, part='snippet, contentDetails').execute() for c_detail in channel_details['items']: c_upload_list_id = c_detail['contentDetails']['relatedPlaylists']['Favorites']
According the doc:https://developers.google.com/youtube/v3/docs/channels "favorites"/"Likes" properties are present in the "contentDetails" object. However, I'm getting the following error:
I get the same error when try to fetch the likes/favorites of my own Youtube Channel (even if they are set to public).
Can someone help me ?
Using Scipy's deconvolve function to deconvolve electrodermal activity data
I wish to deconvolve an EDA (electrodermal activity) signal using a Bateman function as the filter as described here, using Scipy's deconvolve function.
However, when I attempt this, the deconvolution graph does not look how I expect it to. Namely, it generally takes the shape of a mostly flat line, sometimes with spikes at multiples of the filter length:
What am I missing here? Should I be smoothing the EDA curve? Am I hoping for too much from
deconvolve? My code is below:
import csv import numpy as np import matplotlib.pyplot as plt import scipy.signal as signal import math with open('test session 1.csv', newline='') as csvfile: filereader = csv.reader(csvfile, delimiter=' ') i = 0 timestamps =  conductances =  for row in filereader: i += 1 fields = ' '.join(row).split() if i > 3: timestamps.append(float(fields)) conductances.append(float(fields)) timestamps = [timestamp - timestamps for timestamp in timestamps] c = 10. tau1 = 300 tau2 = 2000 bateman = [c * ( math.exp(-time / tau2) - math.exp(-time / tau1)) for time in timestamps] bateman = bateman[3:1700] deconv, remain = signal.deconvolve(conductances, bateman) fig, ax = plt.subplots(nrows=4) ax.plot(conductances, label="EDA Signal") ax.plot(bateman, label="Bateman Function") ax.plot(deconv, label="Deconvolution Result") ax.plot(remain, label="Remainder") for i in range(len(ax)): ax[i].legend(loc=4) plt.show()