This is how I solved the problem. As Venky said, you have to add
android:windowSoftInputMode="adjustResize"
to your tag in the AndroidManifest.xml file. But in our case, it wasn’t enough. Make sure you do this as well with your views, webviews etc. Then we finally made it work.