Page not loading map with api restriction

I'm getting an error "API error: RefererNotAllowedMapError" in my console when I try it render my map. I put a restriction on HTTP referers for my website, but it's still not working. Anyone know why this might be?