C File read getchar yields random EOF after 32KB

2018-01-11 20:22 Haito imported from Stackoverflow

I was tasked to make a file copy over ipc shared memory. The problem is that getc randomly yeilds EOF after 32k char.

FILE* file;
int znak;

file = fopen("./source","r");

if(file != NULL)
{
    while(feof(file) == 0)
    {
        znak = getc(file);

        if(znak != EOF)
        {
            czekaj(0);
            *adres = znak;

            sygnal(1);
        }
    }

    wait(0);        //Wait for your turn
    *adres = EOF;
    signal(1);      //Let other process go
}

Writing part as requested

int znak
FILE *plik;
plik = fopen("./plik_klient", "w");
fclose(plik);

.....



plik = fopen("./result","a");


if(plik != NULL)
{
    while(znak != EOF)
    {

        wait(1);        //Opuszczenie semafora

        znak=*adres;

        if(znak != EOF)
        {
            fputc(znak,plik);

            signal(0);
        }

    }
}

As a result of work other process reads the info and writes it into file.

-rw-r--r--. 1 ficekba inf-17   32769 01-11 21:15 result
-rw-r--r--. 1 ficekba inf-17 1000000 01-11 21:13 source

As you can see result file has exactly 32k

1 answer

answered 2018-01-11 20:35 chux
Code uses char znak when int znak is best.

getc() returns an int in the range of unsigned char and EOF. This is typically 257 different values: [-1 ... 255]. When code read the file source and may return a 255 and assigns that to a char znak, znak has the value of -1 which matches EOF in this case. This fools code into thinking copying is done. and so may end up with a rump result file.

Use int znak.

Also open the file in binary mode is source may be a binary file.
```
// file = fopen("./source","r");
file = fopen("./source","rb");
```

See also questions close to this topic

How to code this usb_modeswitch command using libusb?
sine usb_modeswitch is only available under linux, and I need it under windows and I believe that usb_modeswitch was written based on libusb and I have not used that lib before, I need to code a simple C program that does exactly what this usb_modeswitch command do:
```
usb_modeswitch -v 12d1 -p 157d -V 0x12d1 -P 0x157d --message-content "55534243123456780000000000000011062000000100000000000000000000" -s 60
```
I need this to switch a HUAWEI 3G DONGLE to modem mode (so that the serial com ports will show up in the device manager). Thank you.

What are some compiler options that I can use to get this disassembly?

I have found that the executable from the code

int main(int argc, char** argv) {
  printf("Hello World\n");
  return 0;
}

can be disassembled into (compiled using gcc -m32)

This code is from another person

; Variables:
        ;    arg_4: 12
        ;    arg_0: 8
        ;    var_4: -4
        ;    var_8: -8
        ;    var_C: -12
        ;    var_10: -16
        ;    var_18: -24
push       ebp
mov        ebp, esp
sub        esp, 0x18
call       _main+11
pop        eax                            ; CODE XREF=_main+6

mov        ecx, dword [ebp+arg_4]
mov        edx, dword [ebp+arg_0]
lea        eax, dword [eax-0x1f5b+0x1fa6] ; "Hello World!\\n"

mov        dword [ebp+var_4], 0x0
mov        dword [ebp+var_8], edx
mov        dword [ebp+var_C], ecx

mov        dword [esp+0x18+var_18], eax  ; method imp___symbol_stub__printf
call       imp___symbol_stub__printf
xor        ecx, ecx                           
mov        dword [ebp+var_10], eax
mov        eax, ecx                                    
add        esp, 0x18                           
pop        ebp
ret

but when I disassemble the executable from the same code (compiled using gcc -m32)

I got a really long dissembly code (here is the link)

My questions are

Why are the instructions from the same code different? Or, are they the same?
Is there any compiler or compiler option I can use that gives me the disassembly code just like that from the other person?
Do you recommend any reference that I can read to understand terms from the long assembly code? (such as __x86.get_pc_thunk.dx)

Best regards, Tyson

can i compare 2 SDL_Surface (whether they are the same or not)

While making a game with SDL2 in c,i have to compare 2 SDL_Surface to check a win condition but i couldn't find a way to do so
C++ Audio IPC data passing

I'm developing a data acquisition sw (crossplatform) and I need to send the data coming from the input to 2 different worker threads: one for plotting and one for storing to a file. I'm looking for the best mechanism to implement that: I thing that I need a publisher/subscriber pattern. I've looked at: - ZeroMQ - nanomsg - Boost Interprocess (Shared Memory and Queue)

What's the best solution?

Boost IPC managed_memory_segment bad_alloc

Upon trying to write to a piece of (portable) C++ code that uses shared_memory_segment in order to write to "shared memory", I encountered boost::interprocess::bad_alloc several times. From the Boost documentation:

This exception is thrown when a memory request can't be fulfilled.

So, I must be allocating too little memory. Here follows the code (only for writing, since reading here is irrelevant):

shared_memory.h:

#include <boost/interprocess/managed_shared_memory.hpp>
#include <boost/interprocess/containers/vector.hpp>
#include <boost/interprocess/containers/string.hpp>

#include <boost/interprocess/allocators/allocator.hpp>
#include <boost/interprocess/sync/named_mutex.hpp>

#include <string>
#include <exception>

namespace my_shared_memory
{
  typedef boost::interprocess::allocator<char, boost::interprocess::managed_shared_memory::segment_manager> CharAllocator;
  typedef boost::interprocess::basic_string<char, std::char_traits<char>, CharAllocator> IPCString;
  typedef boost::interprocess::allocator<IPCString, boost::interprocess::managed_shared_memory::segment_manager> StringAllocator;
  typedef boost::interprocess::vector<IPCString, StringAllocator> ShmVector;

  bool write_to_memory(std::string wsuid, std::string loop_val, std::string should_intercept, std::string post_data) ;

  const std::string shm_prefix = "shm_";
  const std::string mutex_prefix = "mtx_";
}

shared_memory.cpp:

#include "shared_memory.h"

namespace apl_shared_memory
{

  bool write_to_memory(std::string wsuid, std::string loop_val, std::string should_intercept, std::string post_data)
  {
    bool ret_val;
    std::string shm_name = shm_prefix + wsuid;
    std::string mtx_name = mutex_prefix + wsuid;
    boost::interprocess::named_mutex named_mtx{boost::interprocess::open_or_create, mtx_name.c_str()};
    size_t size = (sizeof(loop_val) + loop_val.size() + sizeof(should_intercept) + should_intercept.size() + sizeof post_data + post_data.size()) * 5;

    try
    {
      named_mtx.lock();
      boost::interprocess::shared_memory_object::remove(shm_name.c_str());
      boost::interprocess::managed_shared_memory segment(boost::interprocess::create_only, shm_name.c_str(), size);
      CharAllocator     charallocator  (segment.get_segment_manager());
      StringAllocator   stringallocator(segment.get_segment_manager());

      IPCString shm_loop_val(charallocator);
      IPCString shm_should_intercept(charallocator);
      IPCString shm_intercepted_data(charallocator);
      shm_loop_val = loop_val.c_str();
      shm_should_intercept = should_intercept.c_str();
      shm_intercepted_data = post_data.c_str();

      segment.destroy<ShmVector>("ShmVector");
      ShmVector *shmVector = segment.construct<ShmVector>("ShmVector")(stringallocator);
      shmVector->clear();
      shmVector->push_back(shm_loop_val);
      shmVector->push_back(shm_should_intercept);
      shmVector->push_back(shm_intercepted_data);

      named_mtx.unlock();
      ret_val = true;
    } catch(const std::exception& ex)
    {
      ret_val = false;
      named_mtx.unlock();
      boost::interprocess::shared_memory_object::remove(shm_name.c_str());
    }

    named_mtx.unlock();
    return ret_val;
  }

}

Yes, I realize I don't need unlock calls on all three places.

Problem seems to be in the line:

size_t size = (sizeof(loop_val) + loop_val.size() + sizeof(should_intercept) + should_intercept.size() + sizeof post_data + post_data.size()) * 5;

I thought it was overkill to add times 5 but that is apparently not the case. This code works fine on Debian 9 with gcc 6.3.0 and Windows 10 with Microsoft Visual Studio 2015 Community. However, on MacOS with Xcode 10.1, I get boost::interprocess::bad_alloc when I try to insert even the first element in the vector. The solution seems to multiply the size with 10 instead of 5 but this just appears wasteful.

I added segment.get_free_memory calls and determined that for a vector that contains the following three strings (without the quotes)

"true" "true" ""

I need 512 bytes with Xcode. The sizeof operator returned 32, 24 and 32 for IPCString, std::string and ShmVector, respectively. Therefore, it seems that I need 32+4 bytes for one "true" string and 32 for the empty string. Adding the ShmVector there, I need 36+36+32+32=136 bytes for the structures themselves. I calculated the size using sizeof(std::string), which is 24 here (an oversight), so that gives me (28+28+24)*5 = 400, which is not enough here.

My question here is how to determine how much memory I need for the segment? The data I want to write to the segment is known when the function is called and thus is its size.

Dbus ipc communication problems

I have a code where the client is a normal C application(non-root) which gets messages from a server which is running with root permissions . For IPC am using D-BUS. Everything runs fine if both are running as root, but i don't want the client to be root at anypoint. the DBUS which is been used is DBUS_BUS_SESSION.

TypeError while converting pdf to txt file

I have written a function that converts each pdf from a directory into text and I want to get the converted text from the pdf's as txt files. I am getting "TypeError: expected str, bytes or os.PathLike object, not tuple" error in my code. Can anyone please help me with this. Attaching code here:

import io
import os
import os.path
from pdfminer.converter import TextConverter
from pdfminer.pdfinterp import PDFPageInterpreter
from pdfminer.pdfinterp import PDFResourceManager
from pdfminer.pdfpage import PDFPage

    def extract_text_from_pdf(pdf_path):
        resource_manager = PDFResourceManager()
        fake_file_handle = io.BytesIO()
        converter = TextConverter(resource_manager, fake_file_handle)
        page_interpreter = PDFPageInterpreter(resource_manager, converter)

        with open(pdf_path, 'rb') as fh:
            for page in PDFPage.get_pages(fh, 
                                          caching=True,
                                          check_extractable=True):
                page_interpreter.process_page(page)

            text = fake_file_handle.getvalue()

        # close open handles
        converter.close()
        fake_file_handle.close()

        if text:
            return text

    def save_to_txt(lst):
            for i, ele in enumerate(lst): 
                txtfile = "{}.txt".format(i)
                files = extract_text_from_pdf(ele)
                with open(txtfile, "w") as textfile:
                    textfile.write(files) 

    if __name__ == '__main__':
        pdf_path = 'C:\\Users\\Lenovo\\.spyder-py3\\OCR'
        for root, _, files in os.walk(pdf_path):
            for filename in files:
                filepath = os.path.join(root, filename)
                extract_text_from_pdf(filepath)

        for f in filepath:
            save_to_txt(f)

The error is as follows:

runfile('C:/Users/Lenovo/.spyder-py3/updatedpy.py', wdir='C:/Users/Lenovo/.spyder-py3')
Traceback (most recent call last):

  File "<ipython-input-17-f6b3bb00c382>", line 1, in <module>
    runfile('C:/Users/Lenovo/.spyder-py3/updatedpy.py', wdir='C:/Users/Lenovo/.spyder-py3')

  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Anaconda3_64\lib\site-packages\spyder\utils\site\sitecustomize.py", line 705, in runfile
    execfile(filename, namespace)

  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Anaconda3_64\lib\site-packages\spyder\utils\site\sitecustomize.py", line 102, in execfile
    exec(compile(f.read(), filename, 'exec'), namespace)

  File "C:/Users/Lenovo/.spyder-py3/updatedpy.py", line 47, in <module>
    extract_text_from_pdf(file)

  File "C:/Users/Lenovo/.spyder-py3/updatedpy.py", line 22, in extract_text_from_pdf
    with open(pdf_path, 'rb') as fh:

TypeError: expected str, bytes or os.PathLike object, not tuple

C++ Why console output is automatically written along with struct data (fstream)

When I write an object into a binary file than the previous console output (which is already been cleared using clrscr ) gets automatically written into the file.

void initialize_chatroom(){
    clrscr();
    cout<<"\t WELCOME "<<session.name<<" | YOUR-ADMNO:"<<session.ADMNO<<" | CHANNEL:"<<session.channel<<" ";
    read_msgs();
    message d;
    cout<<"Enter Message";
    gets(d.body);
    strcpy(d.sender,session.name);
    d.ADMNO = session.ADMNO;
    send_msg(d);
}

the function to send message is

int send_msg(message d){
    ofstream m;
    m.open("msgs.dat",ios::out||ios::nocreate);
    m.write((char*)&d,sizeof(d));
    m.close();
    read_msgs();
    return 200;
}

Also i am not able to read from the file. Function to read file is

int read_msgs(){
    ifstream m;
    m.open("msgs.dat",ios::in||ios::binary);
    if(!m){
        cerr<<endl<<"Unable to connect to Messages database!! ";
    }
    else{
        m.read((char*)&msgs,sizeof(msgs));
        while(m){
            m.read((char*)&msgs,sizeof(msgs));
            cout<<endl<<" "<<msgs.sender<<" ("<<msgs.ADMNO<<") : "<<msgs.body;
        } 
    }
}

Here session is a global object And Structures are

struct session{
int ADMNO;
int channel;
int status;
char pw[40];
char name[40];
}session;

struct message{
     int msg_count;
     char sender[40];
     char body[100];
     int ADMNO;
    };

Can anyone tell why the output data on console is printed in the binary file and also how to fix that?

How to delete a certain string number from a text file in python
I've searched through stack overflow, but lots of the answers were to remove words from a text file, but not to remove certain lines. I'm programming a bot in discord and want the user to get a line from a file, and then for it to be deleted. I want to delete the last line from the text file so I'm not too sure on how to achieve this. Are there any answers???
```
@bot.command(pass_context=True)
async def spotify(ctx):
spotifynum = 0
author = ctx.message.author

with open("spotify.txt") as f:
    spotifylist = f.readlines()
    spotifylist = [x.strip() for x in spotifylist]
spotifylist = [i.replace('\n','') for i in spotifylist]
for item in spotifylist:
    spotifynum = spotifynum + 1


#variser
spotifynum = spotifynum - 1

spotifyacc = spotifylist[spotifynum]
```

Related questions

C File read getchar yields random EOF after 32KB

1 answer

See also questions close to this topic