In the coastal ecological project aimed at wave attenuation through mangroves, incorporating oyster reefs as a complementary component to establish an “oyster reefs + mangrove” wave attenuation system could enhance the survival probability of mangrove, as well as improve the overall effectiveness of wave dissipation. This study investigates this innovative combined wave attenuation system consisting of oyster reefs and mangroves with various configurations. Laboratory experiments were conducted in a wave flume using artificial models of mangrove and oyster reefs to examine the impact of the system on the wave attenuation effect, thereby providing a scientific foundation for coastal ecological restoration projects. The results indicate that the wave attenuation coefficient increases with the incident wave height, decreases with water depth and period, and enhances with the height of oyster reefs and the density of mangroves. Notably, oyster reefs play a dominant role in wave attenuation within the system. The system’s maximum wave attenuation coefficient of 0.42 is achieved when three layers of oyster reefs are combined with staggered dense mangroves. Moreover, although the summed wave attenuation coefficient of individual mangrove and individual oyster reefs is higher than that of the system, it is noteworthy that the overall wave dissipation exhibited by the system exceeds that of either individual component alone. Additionally, incorporating oyster reefs within individual mangrove structures significantly enhances wave dissipation capacity by up to 0.26. In coastal ecological restoration projects, to enhance wave energy reduction, maximizing the height of oyster reefs in areas with low water depth and pairing them with the staggered dense arrangement of mangroves is recommended. For areas with high water depth, the most effective wave reduction can be achieved through a combination of taller oyster reefs and either staggered or tandem clusters of dense mangroves.