Resolution of primary sceen screwed when booting with second screen pluged in

I have a fresh Ubuntu 16.04 install on my HP notebook. I installed the nvidia drivers to enable for a multiscreen setup with a second desktop monitor.

When I boot my system with only the notebook screen and plug in the secondary monitor afterwards, all works fine.

However when I boot with the secondary monitor attached the resolution of the primary screen is al screwed up. Furthermore I cannot click on anything.

Does anyone have any clue to help me further? I tried reinstalling the nvidia drivers and some other things. But with no luck so far.