Researchers at the Harvard T.H. Chan School of Public Health studied more than 500 couples who were trying to get pregnant.
These concerns may have led some women to shy away from eating fish when attempting to become pregnant, the researchers added.
There has not been much research on the possible links between seafood and fertility, but the few studies that have investigated the relationship may have focused on the potential harms of seafood on fertility, such as exposure to mercury and other environmental chemicals that could have reproductive consequences, according to the study authors.
For future studies, Dr Gaskins would like a research team track the seafood consumption for couples who are undergoing infertility treatments. This study proves that the male seafood intake was just as important as the female one.
All the study participants were asked to keep a diary recording their daily seafood intake and sexual activity.
The findings may be partly explained by the effects on semen quality, ovulation and embryo development, scientists believe. Women's consumption of seafood, however, is hindered by their fears of mercury contamination.
French and American health authorities recommend eating two or three portions of fish per week, and in particular low mercury fish (bass, anchovies, mackerel, sardines, salmon, haddock, cod, herring, and mullet), however, the study points out that 50 per cent of pregnant women consume less than they should.
Though oysters are widely accepted as an aphrodisiac for their nutritional properties, there's little scientific evidence that shellfish have any real biological effect. Most pregnant women are however anxious about eating seafood because of the mercury in them.