Green shipping and electrification have been the main topics in the shipping industry. In this process, the pure battery-powered ship is developed, which is zero-emission and well-suited for inland shipping. Currently, battery swapping stations and ships are being explored since battery charging ships may not be feasible for inland long-distance trips. However, improper infrastructure planning for battery swapping stations and ships will increase costs and decrease operation efficiency. Therefore, a bilevel optimal infrastructure planning method is proposed in this paper for battery swapping stations and ships. First, the energy consumption model for the battery swapping ship is established considering the influence of the sailing environment. Second, a bilevel optimization model is proposed to minimize the total cost. Specifically, the battery swapping station (BSS) location problem is investigated at the upper level. The optimization of battery size in each battery swapping station and ship and battery swapping scheme are studied at the lower level based on speed and energy optimization. Finally, the bilevel self-adaptive differential evolution algorithm (BlSaDE) is proposed to solve this problem. The simulation results show that total cost could be reduced by 5.9% compared to the original results, and the effectiveness of the proposed method is confirmed.