Up until the Meiji restoration(modernisation of Japan), coastal Chinese states have always interacted with the Japanese without much problem. Japanese were a barbaric tribe without much resources. Some trade went on freely between both sides until the Ming emperor imposed restrictions. Then they became like the Somalis and turned to piracy.
The priority of the Ming military was defending against the mongols so military strength along the coastal line was relatively weak. Many enterprising hokkiens saw an opportunity in this and took part in the raids.
In fact Chinese historians today estimated that up to 80% of these Japanese pirate raiders were actually hokkiens. At least one instance of the local government officials who collaborated with the pirates.