04-06 uses the 5vy, you have a 5pw. The 5pw fits the 5pw frame. Technically speaking you in theory could shoehorn a 4c8 (07-08) engine into a 5vy (04-06) frame, but that doesn't help in your situation.
Some professionals crammed a crossplane in one, but honestly, with that amount of post fab you could put any engine in that frame. They had to change the motor mounts, the harness, the ecu, the subframe, move the swingarm and do a whole host of other stuff.
Is it possible to put any engine you want in any frame you want? I mean, technically speaking, I suppose that it is. For the money though, you could just buy another bike.