C++ type-punning when reading Numpy array via Pybind11
I was wondering if anyone knows if the below "type punning" is necessary or if a more efficient technique or approach is best in the last line?
The data is an 2D Numpy array of doubles which is passed by reference as the function argument.
side point: If this function is called indefinetly will a memory leak/ error eventually occur as the ptr_sum pointer is being created each time the function is run on the same identical memory address / location, Im not deleting the pointer "ptr_sum" as it seems to delete the underlying data in the memory location it points to?
double* ptr_sum = (double*)buf_summary.ptr;
void Pricing::price_all(py::array_t<double>& in_summary){
if (in_summary.ndim() != 2) {
throw std::runtime_error("Results should be a 2-D Numpy array");
}
py::buffer_info buf_summary = in_summary.request(); // summary
// Pointer definition
double* ptr_sum = (double*)buf_summary.ptr;
do you know?
how many words do you know
See also questions close to this topic
-
C++ increment with macro
I have the following code and I want to use increment with macro:
#include <iostream> #define ABS(x) ((x) < 0 ? -(x) : (x)) int main(int argc, char** argv) { int x = 5; const int result = ABS(x++); std::cout << "R: " << result << std::endl; std::cout << "X: " << x << std::endl; return EXIT_SUCCESS; }But output will be incorrect:
R: 6 X: 7Is it possible to somehow use macros with an increment, or should this be abandoned altogether?
-
Can anyone pls tell me whats wrong with my code ? im stuck for the last 3 hours ,this question is bipartite graph in c++
idk why im getting error ,can someone help ? im trying to prove if a graph is bipartite or not in c++
bool isBipartite(vector<int> graph[],int V) { vector<int> vis(V,0); vector<int> color(V,-1); color[0]=1; queue <int> q; q.push(0); while (!q.empty()) { int temp = q.front(); q.pop(); for (int i=0;i<V;i++) { if (!vis[i] && color[i] == -1) "if there is an edge, and colour is not assigned" { color[i] = 1 - color[temp]; q.push(i); vis[i]=1; } else if (!vis[i] && color[i] == color[temp] "if there is an edge and both vertices have same colours" { vis[i]=1; return 0; // graph is not bipartite } } } return 1; }it gives output "no" for whatever i enter
-
How to assign two or more values to a QMap Variable in Qt
I am getting confused of how to store the values assigned from 3 different functions and storing them in a single map variable
QMap<QString,QString> TrainMap = nullptr; if(......) ( TrainMap = PrevDayTrainMap(); TrainMap = NextDayTrainMap(); TrainMap = CurrentDayTrainMap(); }The PrevDayTrainMap,NextDayTrainMap & CurrentDayTrainMap returns a set of values with Date and the TrainIdName.I need to store all the values from prevday,currentday and nextday in the TrainMap but it stores only the currentday values to the TrainMap as it is assigned at the last.I am not sure what to do so that it doesn't overwrite.If I should merge what is the way to do it?
-
Reference count in lambda function
When debugging the reference count of the
m_killis shown below://"this" is pointer to a class holding the shared ptr std::atomic_bool return_type create() const { // 1 strong ref to m_kill const std::shared_ptr<std::atomic<bool>> flag = this->m_kill; // 2 strong ref to m_kill and flag auto op = [flag]() -> bool { if (flag->load(std::memory_order_relaxed)) return false; return true; }; // 3 strong ref to m_kill and flag return op; }; // 1 strong ref to m_kill and flag.Could you please help me understand what is happenning as this causes bugs in program, The reference count should be 2 not 1 at the end because the op holds one, wasn't it counted?
Ps: Let me know if you can't spot anything and will copy past the orginal code here (It has to do with TBB).
UPDATE
I asked a new question in here.
-
C: get sizeof typedef struct array inside of function when passed as parameter
I have a bit of code here that I wrote that demonstrates a problem I am having.
I am getting a different value for sizeof when I do sizeof of a pointer that represents a struct array. I think that when I do sizeof inside the function, it is just the sizeof the pointer, rather than the sizeof the array itself.
so here is my question: Is there any way to get the sizeof of a typedef struct array from inside of a function when it is passed in as a pointer?
#include <printf.h> typedef struct { int value_1; int value_2; } example_struct_t; example_struct_t struct_a[] = { { .value_1 = 4, .value_2 = 0, }, { .value_1 = 8, .value_2 = 9, }, }; void printsizeof(example_struct_t *struct_array){ printf("size of struct_array inside func: %lu\r\n", sizeof struct_array); } int main() { printf("This demo shows that a typedef struct array shows a different size inside of main vs inside of a function:\r\n"); printf("sizeof struct_array outside func: %lu\r\n", sizeof struct_a); printsizeof(struct_a); return 1; }this is how it is compiled and the terminal output:
~/ ❯ gcc -std=c99 -Wall -Wextra -Wpedantic -Werror struct_size_bug.c -o struct_size_bug ~/ ❯ ./struct_size_bug This demo shows that a typedef struct array shows a different size inside of main vs inside of a function: sizeof struct_array outside func: 16 size of struct_array inside func: 8 -
C pointer to a pointer confusion
I have the following code. I tried to change the value of y by using a pointer to a pointer (i.e. trying to do a de-referencing twice from the pointer that points to another pointer). this is my code, but for some reasons it doesn't work, could someone tells me where I have done wrong:
#include <stdio.h> void changeAddr(int **j) { (*j)++; printf("change pointer's address to: %p\n", j); (**j)++; // is there something wrong with this part? I want to increase the // value pointed to by the pointer pointed to by j (so I thought // it would be the value of y in main, but there is an error message // the error says: *** stack smashing detected ***: terminated printf("change the value pointed to by the pointer by using a pointer: %0d\n", **j); // this does not // give value 7 for some reasons } void inval(int *i) { (*i)++; printf("address of i: %p\n", i); printf("just increased: %0d \n", *i); changeAddr(&i); } int main() { printf("Hello World\n"); int y = 5; inval(&y); //printf("%0d\n", y); printf("%p\n", &y); printf("value of y in the end: %d\n", y); return 0; } -
pyarrow dependency custom build wheel
I am developing a python package using pybind11 which depends on pyarrow to interop between data types in python and C++. Therefore I build apache-arrow with the following script:
#!/bin/bash set -e VERSION=6.0.0 VERSIONSTR=${VERSION//./_} ARROW_DIR=arrow-apache-arrow-${VERSION} ARROW_TGZ=apache-arrow-${VERSION}.tar.gz INSTALLATIONFOLDER=/opt/arrow_${VERSIONSTR} BUILDFOLDER=build_arrow_${VERSION} if [ ! -f ${ARROW_DIR} ]; then if [ ! -f ${ARROW_TGZ} ]; then wget https://github.com/apache/arrow/archive/refs/tags/${ARROW_TGZ} fi tar xf ${ARROW_TGZ} fi mkdir -p ${BUILDFOLDER} cd ${BUILDFOLDER} cmake -DBUILD_TYPE=Release -DARROW_SIMD_LEVEL=AVX512 -DARROW_PYTHON=ON -DCMAKE_INSTALL_PREFIX=${INSTALLATIONFOLDER} ../${ARROW_DIR}/cpp make -j 12 sudo make install cd ../${ARROW_DIR}/python PYARROW_CMAKE_OPTIONS="-DARROW_HOME=${INSTALLATIONFOLDER}" python setup.py build_ext --build-type=release --bundle-arrow-cpp bdist_wheelwhich installs
apache-arrowand creates the wheel filepyarrow-6.0.0-cp39-cp39-linux_x86_64.whl. With this wheel file installed I can develop my package and all is ok. But how I can deploy a binary wheel for my package? How can I ensure that the consumer has installed the samepyarrowwheel package? Do I break other packages which depends on thepypiversion ofpyarrow? If I want thatmypackage.whlis compatible withpypi's verison ofpyarrowdo I have to build it with somemanylinuximage? -
Non contiguous array pointer traversing issue in pybind11?
Please does anyone have any idea why when I pass my 2 dim Numpy float64 (Double) array normally it prints row by row correctly, col1, col2, col3, col4 as I traverse the pointer by using pos.
However when I pass a particular numpy array of shape 42 rows, 4 cols it instead prints, all of col1, then all of col2, all of col3 and all of col4, rather than a row at a time, please can anyone spot what I'm doing wrong?
I welcome any and all tips as to the potential cause or a resolution :)
I was thinking perhaps its due to memory limitations, however other arrays which are larger don't have any issues
Please note I'm using pybind11 to pass my numpy array to C++, which then traverses the ptr to print out each row at a time, which has been working nicely for "array A" & "array B" except for the new "array C" which is 42 rows by 4 columns, all arrays are float 64.
void Pricing::print_array_simple(py::array_t<double>& in_results) { if (in_results.ndim() != 2) { throw std::runtime_error("Results should be a 2-D Numpy array"); } py::buffer_info buf = in_results.request(); double* ptr = (double*)buf.ptr; int array_number_rows =in_results.shape()[0]; //rows int array_max_size = in_results.shape()[1];//cols cout << "rows: " << array_number_rows << " Cols: " << array_max_size << "\n"; size_t pos = 0; std::cout << "\n---- START OF ARRAY ----\n"; for (size_t i = 0; i < array_number_rows; i++) { //rows for (size_t j = 0; j < array_max_size; j++) { //cols std::cout << std::left <<std::setw(12) <<ptr[pos]; pos++; } std::cout << "|\n"; } }Array that work correctly (72, 7) e.g. Array A or Array B:
CCY TENOR BID ASK MID LAST_BID_TIME_TODAY_REALTIME SETTLEMENT_DATE_RT 0 1.0 1.0 1265.2000 1266.7000 1265.9500 1.651646e+09 1.652051e+09 1 1.0 2.0 1265.3600 1266.3500 1265.8600 1.651661e+09 1.652656e+09 2 1.0 3.0 1265.2400 1266.2800 1265.7600 1.651662e+09 1.654729e+09 3 1.0 4.0 1264.4500 1265.5500 1265.0000 1.651662e+09 1.657494e+09 4 1.0 5.0 1263.5600 1264.6400 1264.1000 1.651662e+09 1.660000e+09 .. ... ... ... ... ... ... ... 67 6.0 8.0 6.7351 6.7411 6.7381 1.651662e+09 1.683500e+09 68 6.0 9.0 6.7870 6.7970 6.7920 1.651662e+09 1.714950e+09 69 6.0 10.0 6.8431 6.8614 6.8522 1.651662e+09 1.746486e+09 70 6.0 11.0 6.9734 6.9883 6.9809 1.651662e+09 1.778022e+09 71 6.0 12.0 7.1178 7.1277 7.1228 1.651659e+09 1.809558e+09C++ output:
---- START OF ARRAY ---- 1 1 1265.2 1266.7 1265.95 1.65165e+09 1.65205e+09 | 1 2 1265.36 1266.35 1265.86 1.65166e+09 1.65266e+09 | 1 3 1265.24 1266.28 1265.76 1.65166e+09 1.65473e+09 | 1 4 1264.45 1265.55 1265 1.65166e+09 1.65749e+09 | 1 5 1263.56 1264.64 1264.1 1.65166e+09 1.66e+09 | 1 6 1259.79 1262.01 1260.9 1.65166e+09 1.66795e+09 | 1 7 1256.05 1258.09 1257.07 1.65166e+09 1.6759e+09 | 1 8 1250.87 1254.47 1252.67 1.65166e+09 1.68359e+09 | 1 9 1242.32 1244.57 1243.44 1.65166e+09 1.71521e+09 | 1 10 1233.71 1243.59 1238.65 1.65165e+09 1.74675e+09 | 1 11 1234.94 1242.71 1238.82 1.65165e+09 1.77845e+09 | 1 12 1228.37 1236.78 1232.57 1.65165e+09 1.8099e+09 | 2 1 76.4175 76.425 76.4213 1.65166e+09 1.65179e+09 |This array doesn't work, 42 rows by 4 cols e.g."array C":
CCY TENOR BID ASK 0 1.0 0.0 1255.9000 1256.4000 1 1.0 1.0 1256.0000 1256.5000 2 1.0 2.0 1255.4000 1256.0000 3 1.0 3.0 1254.6000 1255.2000 4 1.0 4.0 1251.4000 1252.2000 5 1.0 5.0 1247.7000 1248.5000 6 1.0 6.0 1244.0000 1244.9000 7 2.0 0.0 76.2600 76.3000 8 2.0 1.0 76.4600 76.5100 9 2.0 2.0 76.7300 76.7900 10 2.0 3.0 77.0100 77.0700 11 2.0 4.0 77.8100 77.8800 12 2.0 5.0 78.5600 78.6500 13 2.0 6.0 79.4900 79.5700output in C++:
It iterates through the CCY's, then iterates through the Tenors, then iterates through the Bids & then the Asks.
---- START OF ARRAY ---- 1 1 1 1 | 1 1 1 2 | 2 2 2 2 | 2 2 3 3 | 3 3 3 3 | 3 4 4 4 | 4 4 4 4 | 5 5 5 5 | 5 5 5 6 | 6 6 6 6 | 6 6 0 1 | 6 0 1 2 | 3 4 5 6 | 0 1 2 3 | 4 5 6 0 | 1 2 3 4 | 5 6 0 1 | 2 3 4 5 | 3 4 5 6 | 1255.9 1256 1255.4 1254.6 | 1251.4 1247.7 1244 76.26 | 76.46 76.73 77.01 77.81 | 78.56 79.49 29.46 29.41 | 29.335 29.265 29.07 28.89 | 28.705 14442 14470 14507 | 14545 14650 14755 14860 | 52.41 52.63 52.82 52.97 | 53.38 53.72 54.02 6.6681 | 6.6861 6.7026 6.7151 6.7416 | 6.7571 6.7681 1256.4 1256.5 | 1256 1255.2 1252.2 1248.5 | 1244.9 76.3 76.51 76.79 | 77.07 77.88 78.65 79.57 | 29.485 29.44 29.375 29.305 | 29.12 28.94 28.755 14442 | 14480 14523 14565 14680 | 14785 14900 52.45 52.68 | 52.88 53.03 53.45 53.8 | 54.1 6.6711 6.6911 6.7086 | 6.7221 6.7486 6.7641 6.7761 |Python
Python code to create array C: lst = [] for i in range(1,7): for j in range(7): lst.append([float(i),float(j),123.1, 124.1]) lst arr_example = np.array(lst) arr_exampleUpdate: then pass arr_example into the function, it seems to work when I make the array like, this, I think it is because in python in reality I am using df.values to extra the np array from the pandas dataframe which causes issues, anyone have any ideas if this is likely, understand what causes the different memory order of the numpy array?
-
How python loads C++ shared library
context: I have a python program which relies on multiple libraries implemented in C++, exposed via pybinder. My impression is: when python
import xxx, it loads shared libraries (.so) into virtual memory. This is something I remember I read from book or webpage but cannot find it now.I want to know how I can verify it in Linux? I tried to
stracewith eventopen, the only shared libraries opened are something likepy/__init__.so(all python internal libraries), which doesn't look correct to me.In the code base, I have multiple libcurl with multiple versions, all of them are introduced by 3-rd parties libraries. I want to know if I have a way to enforce when running python applications, they don't intervene?
For example: python program A relies on C++ shared library B and C. B relies on libcurl (V1), statically linked. C also relies on libcurl (V2), also statically linked. Bad things could happen if B invokes libcurl (V2), since there might be conflict.