The above places are the best to buy fresh fish to take home. If you wish to eat it in a restaurant, however, there are plenty around the Old Harbour that buy their fish fresh every day, such as Mar Bar, Icelandic Fish and Chips (both, as there are two) and Salka Valka, just to name a few. You can even partake in sea-angling tours, where you can catch your dinner, and either prepare it for yourself or take it to Mar Bar, where they will cook it for you! If you are seeking fresh fish in Iceland, you really won't be looking too long, as fish is one of the two main national dishes, alongside lamb, and Icelanders have high standards for both.
Hope that helps