firewall - SSH Port closed although forwarded correctly

My network currently looks like this (simplified):

Note that Router #2 is connected to the LAN interface of Router #1. This should be familiar to anyone who has seen a standard static-IP setup with an additional firewall for a residence or other small building. Router #1 is actually my cable gateway, but since it is a fully functional router/firewall, I am going to refer to it as a router.

Now, I need to open various ports in both firewalls for incoming communication to my server - port 80 is a good example. So I've opened up port 80 in Router #2, and so far all incoming traffic at the public IP X.X.X.129 is being routed correctly.

The problem is that I also need my server to respond to incoming traffic at the public IP X.X.X.130 on the WAN interface of Router #1. Naturally, I can't just tell Router #1 to forward port 80 to another public IP. Port forwarding is only supported when the traffic is being directed to the LAN subnet.

I am willing to restructure my network topology if required, with the following conditions:

Router #1 cannot have its WAN IP reassigned - X.X.X.130 is mandatory.  
Router #1 cannot be moved or disconnected from the cloud.  
The server cannot be given a second IP address.  
I would prefer the server to have a private IP address - e.g. 10.0.0.10  
I'd like to keep Router #2, but it can have a private IP - e.g. 10.0.1.10

Following these rules, I need to get my server to receive incoming traffic on port 80 from both public IP addresses. Does anyone on SU know if this is possible? So far my only theories have been to set up a static route on either router, or to somehow combine my two subnets into a single subnet.

EDIT:
I have altered my diagram to depict Fred's solution. Seeing that I'm going to have to compromise somewhere, I figure the smallest and simplest compromise would be the most efficient. And while giving my server two IP addresses will certainly complicate the server configuration, the resultant simplicity in the network topology would be a pretty fair trade-off.

This solution will allow both public IP addresses to remain publicly visible, while allowing both routers to forward ports directly to my server's local IP addresses. Furthermore, everything on my network that is connected to Router #2 will be able to access Router #1's subnet, so the server will maintain local visibility from both local IP addresses as well (nothing else is connected to Router #1, besides Router #2 and the server).

My server motherboard does in fact have an integrated gigabit dual-NIC, so my hardware can handle this without any problems. I am going to attempt to configure my server to respond to both NICs today, and we'll see how it goes from there - but at this point I don't forsee any better solutions coming to light, and any further problems caused by this compromise can most likely be solved when they arrive. Of course, if there are any errors in the above network layout, please let me know.

Thanks a ton, guys!

Leave the setup as you have above, except put a new, 3rd firewall router in the DMZ of Router 1 and direct the .130 traffic to the DMZ where you place Router 3. Router 3 then forwards incoming traffic to the 10.0.0.10 address. The kicker here is the One IP Only requirement for the server. You will not be able to send packets back through Router 3 without a distinct IP on the server for that path (traffic via Router 3). The server's routing table will have a single default gateway for the single assigned IP, so no matter which way the traffic arrives (via Router 2 or Router 3), responses will go out the default gateway and are therefore translated to the public IP of that router. Perhaps UDP would work (traffic goes to .130 and comes back from .129), but I see no way for TCP to successfully make a connection on the IP that isn't routed through the server's gateway. I suggest you think really hard about the Only One IP requirement since allowing 2 IPs would make things much simpler.

Ok, assuming that both of your routers have NAT enabled on them, your current setup will not work. The problem you will have is your second IP address(1.129) is not visible publicly and thus can never be accessed from outside your network. The only IP the internet will see is router #1's IP(1.130).

If you want both IP addresses publicly available you will need to put a switch in front of router #1 connecting both router #1 & router #2 to the public internet.

Set up the second router as a switch instead of a router by disabling the DHCP server (if one is running) and plugging the cable from router 1 to router 2 into one of the second router's LAN ports. You will probably also need to set router 2 to an unused static IP on the 10.0.1.x subnet, and move the server to the 10.0.1.x subnet as well. Then clear router 2's port forwarding settings and tell router 1 to forward to the server's new 10.0.1.x IP, and you should be fine.