Looking for a BSS algorithm that works with Real World audio recordings
I am looking for a Blind Audio Source Separation algorithm that I can use to reconstruct human voices from real world recordings. My application is simultaneously recording 2 people speaking using 2 microphones and saving the recordings as mono audio files.
I have tried several implementations of FastICA (including FastICA Package for MATLAB from Aalto University, scikit learn BSS using FastICA, and several other university professors’ implementations) and a couple DUET implementations, but none have worked to separate real world recordings of 2 speakers. I know that FastICA fails when there is delay between the sources, which would be true in the real world so I am not surprised that FastICA did not work for me. Also, DUET makes the assumption that the sources are in separate frequency bands, which is not true of human voices.
Several of the algorithms have been able to separate the “manual mixing” case (speakers are recorded separately and then linearly mixed together using MATLAB). However my project requires that the voices be recorded simultaneously in real time.
I have come across others who seem to be looking for similar algorithms (Real World Blind Source Separation), and I am aware that there are other algorithms (Jade, AMUSE). My understanding is that JADE is another ICA algorithm so I expect that I would get results similar to the results I got from FastICA.
I was wondering if anyone had a BSS algorithm that has worked for reconstructing real world audio samples and could post a link to it. My project is running on a Unixbased OS that does not have MATLAB so preferably the algorithm is not coded for MATLAB, but I have access to MATLAB on another computer so I could still work with it.
Thanks!
See also questions close to this topic

Divide each value in list array
I am trying to divide by 80 of each array value in the list. What I have tried is,
dfs = pd.read_excel('ff1.xlsx', sheet_name=None) dfs1 = {i:x.groupby(pd.to_datetime(x['date']).dt.strftime('%Y%m%d'))['duration'].sum() for i, x in dfs.items()} d = pd.concat(dfs1).groupby(level=1).apply(list).to_dict() print(d)
OP :
{'20170506': [197, 250], '20170507': [188, 80], '20170508': [138, 138], '20170509': [216, 222], '20170609': [6]}
But Expected OP :
1 : Divide by 80 {'20170506': [2, 3], '20170507': [2, 1], '20170508': [2, 2], '20170509': [2, 2], '20170609': [0]} 2 : total of each array and subtract each value (3+2 = 53 and 52) {'20170506': [3, 2], '20170507': [1, 2], '20170508': [2, 2], '20170509': [2, 2], '20170609': [0]}
How to do this using python?

Export list of data frames to CSVs in python
I've got a list of data frames I'm trying to preform a function on and export the results. The function spits out a result, and I then want to turn the results into a data frame and export to a .CSV. Here's what I currently have:
for df, filename in zip(df_list, filename_list): function(df) results_df = pd.DataFrame(function_results) results_df.to_csv(filename)
The error occurs when I try to export the .csv. If I just run the loop with the function and print results to the console like so:
for df in df_list: function(df)
It works fine. When I try to loop the .csv export though I get
Attribute Error: 'list' object has no attribute 'close'
Any ideas?

Tips on saving distribution shape parameters to sqlite3
I have been playing around with distributions in Python using scipy. I have some code which produces a dataframe containing fit parameters for my various data fits. Example is shown below
fit = pd.DataFrame({'domain': ['T1','T1', 'T1'], 'type':['A1', 'A2', 'A3'], 'dist':['triang', 'triang', 'trapz'], 'shp':[(0.1, None), (0.1, None), (0.2, 0.8)], 'loc':[3, 100, 85], 'scale':[60, 50, 95]})
I have a range of these inputs using different distributions, so the shape parameters change and need to be kept in the right order. Therefore, when It is retrieved from the database, it will fit in with the distribution order.So for example, using the Triangle distribution: (c, d, loc, scale).
I have tried to convert the tuple to a string (keeping the brackets) so it can be viewed in the database. For example, for the 3rd row.
a = str(fit['shp'][2])
which produces
'(0.2, 0.8)'
. This can be saved to the Sqlite3 database.However, I am not to sure how to convert it back to a tuple?
I did try,
b = tuple(a)
But this does not seem to produce the desired result, as shown below.
('(', '0', '.', '2', ',', ' ', '0', '.', '8', ')')
What would you guys recommend about saving the shape tuple to a database?
BJR

shifting each character by the second parameter
Encrypt all characters of the first parameter by shifting each character by the second parameter
Input string will always be in uppercase. No need to shift any nonalpha characters.
Example :
function encrypt(str, index) { var encryptedstr = ''; var code = 0; for(var i = 0; i < str.length; i++) { var k = str[i].charCodeAt(); if (k >= 65 && k <= 90) { code = k + index; encryptedstr += String.fromCharCode(code); } else { encryptedstr += str[i]; } } return encryptedstr; } console.log( encrypt('ABC', 4) );
but my code is statisfying for following instances
1.The return type should be a string.
2.encrypt("ABC", 4) Should equal to “EFG”
3.encrypt("AB C", 2) Should equal to “CD E”
4.encrypt("ABC DEF", 2) Should equal to “CDE FGH”
========================================================================== but i am having following error
The answer should be valid for any given input.
any suggestion ??

My code stop when first match and I need to search whole text
Looks like problem here but I could not find out how to continue search after first match. I need to make a program of Horspool Algorithm a I'm stuck in there. It is supposed to search all text but when matches for first time it stops.
public static class BoyerMooreHorspool { public static int Find(string haystack, string needle) { // If substring is bigger than string, no match exists if (needle.Length > haystack.Length) return 1; // If the substring has characters not in the string, no match exists if (needle.Except(haystack).Any()) return 1; Dictionary<char, int> BadMatchTable = new Dictionary<char, int>(); // Initializes every letter with the default value foreach (char c in haystack) { if (!BadMatchTable.Keys.Contains(c)) BadMatchTable.Add(c, needle.Length); } // The formula for every letter in the needle (except the last) is lenght  index  1 for (int i = 0; i < needle.Length  1; i++) { BadMatchTable[needle[i]] = needle.Length  i  1; } int index = 0; while (index <= haystack.Length  needle.Length) { bool match = true; for (int i = needle.Length  1; i >= 0; i) { if (needle[i] != haystack[index + i]) { match = false; index = index + BadMatchTable[haystack[index + needle.Length  1]]; } } if (match) { return index; } if (index==haystack.Length) { return 1; } } return 1; } }

Cannot short string array having large strings numbers
I have to sort large datasets having long strings(length up to 1 million char) containing only digits. Also considering all the strings as large positive numbers only.
I have modified mergesort code that works really fine for large data sets (with array size of 200000) if the length of string is within 18 (so I could convert it to long number for comparision).
I have also implemented a logic that should work theoretically for any length of string (string of numbers). But there is some glitch in my code that is not allowing to sort array with long (length > 18) strings. I have added a capitallatter comment in that code block below.
Note: the code is successfully executing within a few seconds for all length of data sets and gives not so correct output as shown at the end.
Below is my code :
package algorithms; import java.math.BigInteger; import java.util.Scanner; public class BigSort { static void merge(String arr[], int l, int m, int r) { int n1 = m  l + 1; int n2 = r  m; String L[] = new String [n1]; String R[] = new String [n2]; for (int i=0; i<n1; ++i) L[i] = arr[l + i]; for (int j=0; j<n2; ++j) R[j] = arr[m + 1+ j]; int i = 0, j = 0; int k = l; while (i < n1 && j < n2){ if (L[i].length() <= R[j].length()){ if(L[i].length()<=18 && R[j].length() <=18) { if(BigInteger.valueOf(Long.parseLong(L[i])).compareTo(BigInteger.valueOf(Long.parseLong(R[j]))) <=0){ //this will convert strings to numbers and compare them. //I have used it just to possibly decrease load of //comparing each characters for sorting smaller strings. arr[k] = L[i]; i++; }else{ arr[k] = R[j]; j++; } }else{//THIS ELSE PART IS HAVING SOME PROBLEM. //if length of string is greater than 18digits //it will compare two string character by character to find //the larger string or if they are equal. char[] c1 = L[i].toCharArray(); char[] c2 = R[j].toCharArray(); int c1leng= c1.length; int c2leng= c2.length; //int shorter= c1leng < c2leng ? c1leng : c2leng ; if(c1leng==c2leng){ for(int p=0; p<c1leng; p++){ if(c1[p]==c2[p]){ if(p == c1leng1) { arr[k] = L[i]; i++; break; } continue; }else if(c1[p]<c2[p]){ arr[k] = L[i]; i++; break; }else if(c1[p]>c2[p]){ arr[k] = R[j]; j++; break; } } }else{ arr[k] = R[j]; j++; } } }else{ arr[k] = R[j]; j++; } k++; } while (i < n1){ arr[k] = L[i]; i++; k++; } while (j < n2){ arr[k] = R[j]; j++; k++; } } static void sort(String arr[], int l, int r) { if (l < r){ int m = (l+r)/2; sort(arr, l, m); sort(arr , m+1, r); merge(arr, l, m, r); } } static String[] bigSorting(String[] arr) { sort(arr, 0, arr.length1); return arr; } public static void main(String[] args){ Scanner in = new Scanner(System.in); int n = in.nextInt(); String[] arr = new String[n]; for(int arr_i = 0; arr_i < n; arr_i++){ arr[arr_i] = in.next().trim(); } System.out.println("result is:"); String[] result = bigSorting(arr); for (int i = 0; i < result.length; i++) { System.out.print(result[i] + (i != result.length  1 ? "\n" : "")); } in.close(); } }
these are the inputs that I was using (first row takes number of strings and then follows all the strings to be sorted. output is the sorted numberstrings in each line):
input(1) 10 5454545454 212101225515 51212 5141215 52 521 52145 5 5 5 Output(1)//correct 5 5 5 52 521 51212 52145 5141215 5454545454 212101225515 Input(2) 10 5454545454 212101225515 51212 5141215 52 5465156165164215612616546954512202496421 2121564 216451564561564651564561256065 11 55 Output(2)//incorrect 11 52 55 216451564561564651564561256065 51212 2121564 5465156165164215612616546954512202496421 5141215 5454545454 212101225515

How to properly extract an embedded watermark from a photo
I have an original, grayscale image that is 256 x 256. My watermark image is 110 x 100. It was originally a colored image but I converted it to a binary:
originalImage = imread('dog.png'); petLogo = imread('petLogo.png'); petLogoBinarized = im2bw(petLogo);
In order to extract the watermark, I had to actually embed it in the original image itself at the top left corner of the original image. I did so here:
firstWatermarkCopy = originalImage; // make a copy of the original image for column = 1 : watermarkCols for row = 1 : watermarkRows firstWatermarkCopy(row, column) = bitset(originalImage(row, column), 1, petLogoBinarized(row, column)); end end
Where watermarkCols refers to the columns in petLogoBinarized watermarkRows refers to the rows in petLogoBinarized.
This works perfectly fine, I believe. The copy is a 256 x 256 image with a watermark at the top left corner.
However, I want to extract this watermark and display it separately but the problem is, my extracted watermark now reads as a 110 x 110 image. Shouldn't it be a 110 x 100 image, just like the original?
recoveredWatermark = zeros(size(petLogoBinarized); // allocate an empty vector the size of the original watermark for column = 1 : size(petLogoBinarized, 1) for row = 1 : size(petLogoBinarized, 2) recoveredWatermark(row, column) = bitget(firstWatermarkCopy(row, column), 1); end end
For reference, 1 is the bit plane I am extracting. I am suppose to be seeing a 110 x 100 image, correct, and not a 110 x 110 image? The image itself looks okay but the padded size is added noisy space on the image on the righthand side.

Degree of Irregularity (DOI)
As part of my research, I need to implement a path loss model based on the degree of irregularity (DOI).
The model uses Weibull distribution.
I have the collected data for every 5 degree (72 degrees in total) and from this data I need to find out the overall DoI value for the entire rotation.
I need to implement this in Matlab and I was wondering if somebody could help me by explaining how to compute the value. Sample dataset
References http://behnam.dezfouli.com/publication/MLPWCJNCA2014.pdf (page 109)
https://www.usenix.org/legacy/publications/library/proceedings/mobisys04/pdf/p125zhou.pdf (page 129)

How to change the size of bar in bar3 plot in MATLAB?
In the attached bar3 plot, in 'kanal 1' and 'kanal 2' axis values are not equidistant. But in plot all the bar are same square sized only height is different. Is there and command to change the size for better representation? A sample code is given here:
bar3(rand(3,4)) set(gca,'XTickLabel',[10 20 30 40]) set(gca,'YTickLabel',[100 200 300])
I just need non square sized bar. Bar size need to be changed according to axis label value. If there any other plotting suggestions, it will be warmly welcomed. Thanks

extrapolation of the curve
I am trying to extrapolate the fitting curve. There are many questions posted with a similar problem, but I don't understand any of them. I tried the simplest of the solutions, however, it is not working. Can anyone of you review the code please?
import numpy as np import matplotlib.pyplot as plt from scipy.optimize import curve_fit old_settings = np.seterr(all='ignore') x=np.array([0.21, 0.43, 0.50, 0.65, 0.86, 1.0, 1.5, 2.0, 2.5, 3.0, 4.0]) y=[43, 33, 30, 24, 18, 16, 14, 13, 14, 13, 13] yerr= [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] xerr=[0.01,0.01,0.01,0.01,0.01,0.01,0.01,0.01, 0.01,0.01,0.01] def func(x, a, b, c, d): return a * np.exp(b * x) + c * np.exp(d * x) def func1(x,m,n): return m *np.exp (n*x) def func2(x, s, t): return s *np.exp(t*x) # Here you give the initial parameters for a,b,c which Python then iterates over # to find the best fit popt, pcov = curve_fit(func,x,y,p0=(1, 1e6, 0.5, 1)) print(popt) # This contains your three best fit parameters p1 = popt[0] # This is your a p2 = popt[1] # This is your b p3 = popt[2] # This is your c p4 = popt[3] # This is your d residuals = y  func(x,p1, p2, p3,p4) fres = sum( (residuals**2)/func(x,p1, p2, p3,p4) ) # The chisqaure of your fit print(fres) """ Now if you need to plot, perform the code below """ curvey = func(x, p1, p2,p3,p4) # This is your y axis fitline curvey1 = func1(x, p1,p2) # This is your y axis fitline curvey2 = [func2(i,p3,p4) for i in x] # This is the curve I want to extrapolate to x=0 plt.plot(x, curvey, 'red', label= r"Fit: $A \cdot e^{a \cdot x}+ B \cdot e^ {b \cdot x}$" "\n" r"$\chi ^2: 0.52 $" ) plt.plot(x, curvey1, 'blue', label= r"Fit: $A \cdot e^{a \cdot x}$") plt.plot([0]+x, curvey2, 'green', label= r"Fit: $B \cdot e^{b \cdot x}$") plt.errorbar(x,y, yerr=yerr, xerr=xerr, fmt='.',label='experimental data') plt.legend(loc='best') plt.ylim(0,45) plt.xlabel('x') plt.ylabel('y') plt.show()
The curve I want to extrapolate is given in the line 42.

Numpy/Scipy: Singular Matrix Calculating probability of multivariate observation
I'm trying to calculate probabilities for observations in matrices where my rows are observations and my columns are features using python. I always get singular matrix errors, even when using random matrices, so I suspect something is wrong with my code:
from scipy.stats import multivariate_normal import numpy as np def get_mean(x, axis=0): return x.mean(axis=axis) def get_sigma(x): return np.cov(x, rowvar=False) def get_probabilities(x, mu, sigma): return multivariate_normal.pdf(x, mean=mu, cov=sigma) x = np.random.rand(10,10) t = np.random.rand(1, 10) mu = get_mean(x) sigma = get_sigma(x) p = get_probabilities(t, mu, sigma)
This results in:
Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 2, in get_probabilities File "/usr/local/lib/python3.5/distpackages/scipy/stats/_multivariate.py", line 512, in pdf psd = _PSD(cov, allow_singular=allow_singular) File "/usr/local/lib/python3.5/distpackages/scipy/stats/_multivariate.py", line 159, in __init__ raise np.linalg.LinAlgError('singular matrix') numpy.linalg.linalg.LinAlgError: singular matrix
What am I doing wrong?

Are these frequent calls to a function that returns a fixed value in dblquad avoidable?
This script calculates the electric field at (x0, y0, z0) from a uniformly charged annulus using SciPy's dblquad.
My question is about the use of
gfun
andhfun
which define the functional dependence of the inner integral limits on the outer integral variable. This would be a cookiecutter shape if I'd chosen to integrate in cartesian coordinates, but when I use cylindrical coordinates the functions return constant floats.Is there a way to eliminate these function calls that simply return constants in order to avoid the time penalty of the function calls?
The example may not be optimized in other ways, but it's just a simple example to show the use of
gfun
andhfun
.def Excalc(r, th): x, y, z = r*np.cos(th), r*np.sin(th), 0.0 return (x0x) * ((x0x)**2 + (y0y)**2 + (z0z)**2)**1.5 def Eycalc(r, th): x, y, z = r*np.cos(th), r*np.sin(th), 0.0 return (y0y) * ((x0x)**2 + (y0y)**2 + (z0z)**2)**1.5 def Ezcalc(r, th): x, y, z = r*np.cos(th), r*np.sin(th), 0.0 return (z0z) * ((x0x)**2 + (y0y)**2 + (z0z)**2)**1.5 def gfun(x): return rmin def hfun(x): return rmax import numpy as np import matplotlib.pyplot as plt from scipy.integrate import dblquad twopi = 2.*np.pi # annulus of uniform, unit charge density rmin, rmax = 0.8, 1.2 thmin, thmax = 0, twopi # point to evaluate the field x0, y0, z0 = 1.5, 0, 1 eps = 1E10 Ex, Exerr = dblquad(Excalc, thmin, thmax, gfun, hfun, epsrel=eps) Ey, Eyerr = dblquad(Eycalc, thmin, thmax, gfun, hfun, epsrel=eps) Ez, Ezerr = dblquad(Ezcalc, thmin, thmax, gfun, hfun, epsrel=eps) print Ex, Ey, Ez print Exerr, Eyerr, Ezerr

How to measure waveform distortion analysis (THD, THD+N...) in C++
I'm looking for how to measure distortion for waveform. Waveform data is received from the oscilloscope.
Currently I use FFTW(fftw_plan_dft_1d) to convert waveforms to spectrum. But I don't know how to perform distortion measurements?

How to disable "precharge" while generating a periodical signal using a Keithley2635B sourcemeter?
Good day,
I am trying to source custom waveforms using a Keithley 2635B sourcemeter. Using the trigger attribute
listv
and the timer attributedelaylist
it is easy to do but an unexpected behavior appears.As you can see on the attached picture, I am sourcing a very simple square signal (two points
+A
and–A
with the same delay) and repeat it a number of times usingsmua.trigger.arm.count = <Periods>
. Notice how the signal plateaus when switching polarity? It only happens when crossing 0, if I add an offset so that the signal keeps the same sign there is no issue. This plateau has a very consistent duration of 125 µs regardless of which frequency I try to source. Its impact is not negligible anymore at high frequencies.Below is the portion of the code that matters. Code that is upstream only processes the various parameters given by the user (frequency, amplitude, offset, etc) to create the appropriate
listv
anddelaylist
. Lua commands sent to the Keithley are wrapped with theWRITE()
function so that the rest of the program can be written in Python.# Source V K.WRITE("smua.source.func = smua.OUTPUT_DCVOLTS") # Configure actions to start immediately. K.WRITE("smua.trigger.arm.stimulus = 0") K.WRITE("smua.trigger.source.stimulus = 0") K.WRITE("smua.trigger.endpulse.stimulus = 0") # Configure the source action. K.WRITE("smua.trigger.source.action = smua.ENABLE") K.WRITE("smua.trigger.endpulse.action = smua.SOURCE_HOLD") K.WRITE("smua.trigger.source.listv(%s)" %SourceArray) K.WRITE("smua.trigger.arm.count = %d" %Periods) K.WRITE("smua.trigger.count = %d" %PointsPerPeriod) # Delay between outputs K.WRITE("trigger.timer[1].reset()") K.WRITE("trigger.timer[1].delaylist = {}".format(DelayArray)) K.WRITE("trigger.timer[1].stimulus = smua.trigger.SOURCE_COMPLETE_EVENT_ID") K.WRITE("smua.trigger.measure.stimulus = trigger.timer[1].EVENT_ID") # Start the trigger to generate the AC signal then turn off the output when done. K.WRITE("smua.source.output = 1") K.WRITE("smua.trigger.initiate()") K.WRITE(“waitcomplete()”) K.WRITE("smua.source.output = 0")
Edit: the sourcemeter documentation indicates that
in cases where the first sweep point is a nonzero value, it may be necessary to precharge the circuit so that the sweep will return a stable value for the first measured point without penalizing remaining points in the sweep.
so I assume the plateau I witness is precharge at the beginning of every repetition. Is there a way to disable it? Thanks in advance.

Find equivalent digital filter
I'm trying to find an equivalent digital filter for a simple RC filter. The bode plots don't line up and I don't know why.
====================================================
import numpy as np import scipy.signal as sig import matplotlib.pyplot as plt # Analog filter tau = 0.001 # param lti = sig.lti([1], [tau, 1]) # Equivalent digital filter fs = 20000 # param T = 1 / fs wd = 1 / tau wa = 2 / T * np.tan(wd * T / 2) dtau = 1 / wa dnum, dden = sig.bilinear([1], [dtau, 1], fs) dlti = sig.dlti(dnum, dden) w, mag, phase = sig.bode(lti) dw, dmag, dphase = sig.dbode(dlti) plt.figure() plt.subplot(211) plt.semilogx(w, mag) # Bode magnitude plot plt.semilogx(dw, dmag) plt.subplot(212) plt.semilogx(w, phase) # Bode phase plot plt.semilogx(dw, dphase) plt.show()