Can you tell us the brand and model of the router, as well as what the lights do on the front? Either the old/dead router or the replacement if you already did or decide to just replace it and need help with that.
Replacement's probably the easiest thing to do if you can't reach it at all. Starting fresh is as easy as troubleshooting and it might be dead.
If working on the old, are the computers connecting to it? Varies with your devices, but all should have an icon that indicates a connection or not somewhere on the display. That indication and the lights on the router usually tell the story.
If you want to try something with the old router you need the password you set on it, or if you didn't set one, it's in the manual (really important to set your own complex password on these and write it down). The inside facing IP address, the one you use to reach the router is needed as well, but that can be discovered. If you have the password but not the IP, assuming the wired PC is windows, use that and download [Invalid or Expired Link Removed] which you can install or just run it uninstalled (it will offer both options). Just type in 192.168.1.0/24 and hit the green button. If nothing is found, try 192.168.0.0/24 and repeat. If you can't get that program onto the computer, the IP address is almost always 192.168.1.1 and you can just type that into the web browser, then if nothing, try 192.168.0.1
If you don't have the password, we have to set it back to factory default, then you might as well go new. As was mentioned, the ISP provided modems, or cheap modems are not very reliable. Newer ones work well. Cisco/linksys are pretty solid, but set you back for about $100, and a provider will always give you the $30 variety.
Always start working from the wired computer first. It's easiest. Wireless is much more voodoo. You do want to setup the security on wireless.