Are the cush drive rubbers missing? (part 92075 in the image above)
That would explain being able to wobble the sprocket around.
According to the parts diagram above, the axle should go in from right to left as seen from the rear of the bike. Nut on the sprocket side.
Image of chain adjuster orientation for the 2005 Ninjette below...
