i get that you don´t have to point the controller directly at the tv.No, not at all. The gyroscopes make it so space is relative; you can have the controller pointing anywhere. This isn't the Wii/PSMove.
i think my problem is more that when i move around, and my hands are in different positions that there is a disconnect for me how to reliably move my hands/wrists.