The point is that the keypad has to show. So it will either push out the layout (scrolling out up the top), squeeze out the rest of the screen (which is the current situation), or need to cover up the bottom part of it.
Apparently, this last option is the one you favor.
What you can do then is temporarily make the arrangement (either column or row or whatever) smaller by changing its height to be less than what it contains. If the component is top justified, then everything that exceeds the nominal height would be clipped from view, leaving room for the keypad.
You need to capture the end of keypad entry to restore the arrangement height however, once the keypad is dismissed.
Alternatively, you can have a sub component that is just the same height as the keypad, and make that component invisible while the keypad is present–that is the solution I personally use in my apps.