I think that it will overheat. My router is always warm.

How about a heatpipe cooler? You could run the heatpipes into the box through holes with the cooling fins outside the box. You could then seal the area where the pipes enter the box.