If your device supports touch interaction, PDF Annotator will by default scroll the document when you use your finger on the document area.
If your finger draws on the document instead of panning the document, there's a chance you have enabled the Draw with Touch setting.
Note: The Draw with Touch setting is only visible when your device supports "touch".
In the first screen shot, Draw with Touch is enabled (the button has a blue background).
In the second screen shot however, Draw with Touch is not enabled.
In Full Screen Mode, the Draw with Touch setting is available in the default set of Action Buttons. Click the Expand button ...
... to find the Draw with Touch button.