Facing Hackers Attack in my Rails App hosted on AWS

I found some unauthorized access_log in our production.log file, where /proxy.php, /testproxy.php pages, /JBOSSServlet, /pp/anp/php type pages are hit too many times while our rails website page uses .html and .erb type pages.

How can I block those URLs into my rails app?

There is also a possibility that there is any unauthorized script running on my server, in this case how we find it and kill that processes.

I also try to use rack_attack gem to block the URLs which hits beyond the given limit, but I am not able to configure it correctly.

please help.