imsmooth,
As far as I can remember, the hardware limit switches are used only for homing the axis. After that, everything is aligned by keeping track of step counts sent to the motors. So maybe you are losing steps somewhere.
Here are a few things to check:
1. Make sure the belt driving (toothed) pulleys on the X and Y motors are secured to the motor shaft -- setscrews tightened. Some folks have had them work loose. Check other threads here for recommendations on "lock-tite" or something like that for keeping the setscrews "set" after tightening to the motor.
2. One user reported having issues with the ABS printed belt clamps -- there's one under the "spider" that rides the Y axis belt, and one under the extruder assemblage that rides the X axis belt. Make sure each is securely gripping the respective belt.
3. Some users have reported losing steps and having to increase the current applied to the stepper motors (which determines the holding and accelerating torque of the motors, and which is set in the firmware), or reducing the accelerations of the X and Y axes (which can be done by sending appropriate G-codes -- which start with M by the way

) to the machine -- once you get accelerations dialed in right, you can send another code to the printer to copy those settings to EEPROM on the microcontroller so the board powers up with the new settings.
Let us know how you progress. And I'm sure some others will chime in with other ideas...
Best Regards,
Dale