You need to be cautious when swimming in the sea around the island as the current runs very near to the shore and would sweep you away in a second. I was a flying fish fisherman on Yakushima and the strength of the tides on the fishing boat never ceased to amaze me.
There are however areas around the island particularly suited to swimming and of course relaxing on the beach. Here are some of them:
NAGATA INAKAHAMA BEACH
There is a very strong current not too far from the shore all the way along this beach. If you swim here, you need to do so with care. Paddling is better!
For more informatiom about Yakushima check out the one and only Yakumonkey Guide