When the protagonist is too strong in a certain world, the system will generate a "balancer" (such as a special enemy that combines virtualization + curse spirit + devil fruit abilities). The protagonist will randomly forget part of the real world memory every time he travels through (it is eventually revealed to be a self-protection mechanism)
