Node app stop responding when handling too many timeout requests

I have a node app running on a Ubuntu server, handle users request, and crawl other website's data, return it back to user.

Recently we've found an issue, that is when the website we are crawling isn't responding (Server down) for a long time (Like an hour),

My node app will stop responding, and affect other API's request (Other API won't be able to handle requests, it'll take a very long time), and the CPU will rise, eventually the whole server goes down, the only way to resolve is to restart the server, or, restart the node app before server goes down.

At the beginning I thought the reason is I didn't set timeout for crawling other website, so when requests come in, and because their server is down, lot's of connections hang in there and consuming server resource, but after I set a timeout for crawling the website (5s), this problem is still not solved.

I have totally no idea in how to solve this problem, so can somebody give me some advice? thank you

Environment:

  • node v4.6.1
  • Ubuntu 14.04.5 LTS

Packages:

  • pm2 v2.8.0
  • koa v1.1.2
  • request v2.4.8

Server on Google Cloud Engine:

  • CPU: 6 vCPUs
  • Memory: 5.5 GB memory
  • Disk: 50 GB SSD persistent disk