What kind of lighting are you using? I think that is where a lot of potential for improvement lies. With textured subjects, the angle(s) of your light source(s) with regard to the camera's optical axis can make a significant difference in contrast and the amount of detail you can see.
Using flash will also allow you to "kill the ambient" and freeze any motion caused by the water moving around in the tank. Even though your shutter duration may be fairly slow (at your camera's flash sync speed), if the only light that makes a major contribution to the image is from your flash(es), then the short duration of the flash will freeze the motion.
The fact that your subjects are usually inside aquariums does present additional challenges, but they're not insurmountable. Much of what is discussed regarding lighting in this question about photographing fish in an aquarium would also apply to your situation.
Your 12 MP APS-C D90 has the same pixel density as a 27 MP FF sensor would, so you'd actually be losing a bit of ground with the D610 at 24 MP, but the 6% linear difference would be negligible. You'd just get more in the frame with the same lens. The D810, on the other hand, has a 36.3 MP sensor. In terms of linear resolution, that's a 23% increase over the 90D which could be significant if your lens is up to the challenge.
Which brings us to the lens choice, which is where all macro photography should start.
We should probably pause here to define a few terms:
- MFD - Minimum focus distance is measured from the subject to the imaging plane. That is, the focus distance is measured from the film or digital sensor to the subject.
- WD - Working distance is measured from the front of the lens to the subject. Working distance can be defined as the minimum focus distance less the distance from the imaging plane to the front of the lens when the lens is fully extended at MFD. If a lens has an MFD of 8" and it is 5" from the sensor to the front of the lens when the lens is focused at MFD, the WD would be the remaining 3".
- MM - Maximum magnification is the largest the lens can project a focused image of the subject onto the film or sensor. If a lens has a MM of 1.0X, it can project an image of the subject that is the same size on the sensor as the subject is in real life. If a lens has a MM of 0.5X, it can project a half-size image of the subject onto the sensor. This magnification is measured at the sensor.
- RR - Reproduction ratio is another way of expressing the amount of a lens' maximum magnification. It is the ratio of the size of the projected image compared to the size of the subject. 1:1 is the same thing as 1.0X MM. 1:2 is the same as a 0.5X MM, 5:1 is the same as a 5.0X MM (very specialized lenses), and so on.
- Enlargement ratio - When we view images from most of our cameras, we enlarge the resulting image far beyond the size of the sensor or film. If we have a full frame camera (36x24 mm sensor or film) and a lens with a MM of 1.0X, when we view the image at even 4x6 inches, we've used an enlargement ratio of about 4.25X (linear), so the subject will appear 4.25X wider and taller on a 4x6 inch print than its actual size. It would appear 8.5X it's actual size for an 8x12 inch print, and so on.
Most macro lenses are only capable of 1:1 reproduction at their minimum focus distance.¹ If you have to shoot from further away, you lose some of that magnification. The further back you have to shoot from, the smaller the subject will be in the frame.
- If you are using the AF Micro-Nikkor 60mm f/2.8D lens you have an MFD of 8.7" and a working distance of 3.56 inches.
- If you are using the AF-S Micro-Nikkor 60mm f/2.8G ED, Nikon lists the MFD at 7.2" which leaves a working distance of only about 2 inches when the lens is extended at MFD.
The density of the glass through which you are shooting will affect this somewhat, and the density of the water on the other side of the glass will affect it even more. The refraction due to the air/glass/water interface will increase magnification, which will also reduce MFD slightly.
To be able to use maximum magnification with subjects that are more than 2-3 inches from the side of the aquarium, you'll need a longer focal length macro lens than your current 60mm lens.
The following working distances are for each lens at 1.0X (1:1) maximum magnification:
Tamron 90mm Macro (there have been several well regarded versions - I'm using the Model F017 specs) has a working distance of about 5.1"
Nikon AF-S 105mm f/2.8G Micro has a WD of about 5.3"
Sigma 150mm f/2.8 EX DG APO HSM Macro has a WD about 7.3"
Sigma 180mm f/2.8 EX DG OS HSM Macro has a WD of about 10"
Other macro lenses in each of the focal length ranges are similar. The longer the focal length, the greater the MFD at 1:1 and, assuming the lens lengths are relatively proportional to their focal length, the greater the working distance.
There are plenty of other questions/answers here regarding technique for doing macro work, but we'll mention a couple of things that many folks new to macro work can easily miss:
- To get maximum magnification the lens must be set at the minimum focus distance. Typically one manually sets the lens' focus at MFD and then moves the camera forward or backwards until the subject comes into focus.
- To get maximum detail a sturdy tripod or other solid camera mount is required. When using the MFD technique mentioned above, a macro rail between the tripod and the camera can be invaluable. Macro rails can range in price from around $20-30 to several hundreds of dollars, but you can find pretty good ones for less than $100.
¹ There are some macro lenses that have higher reproduction ratios than 1:1. With such lenses, the highest reproduction ratio is always at MFD, and 1:1 would be at a longer focus distance than MFD.
