Dual DisplayPort KVM switch with HDMI monitor

I recently got a 2 port Dual DisplayPort KVM Switch so that I could use both my work laptop (the docking station comes with 2 DisplayPort outputs, hence the choice for a DisplayPort switch) and my desktop with my existing 2 HDMI monitors.

I’m using 2 DisplayPort to HDMI passive converter cables from the KVM switch to the monitors and I can use both monitors fine when I’m on the laptop. However, when using the desktop, one of the 2 monitors isn’t detected by the video card. My video card has a DisplayPort output (working with one monitor). The second port is an HDMI. I realized that I couldn’t passively convert an HDMI signal to DisplayPort, so I’m now using an active HDMI-to-DisplayPort converter from the card to the switch. For some reason the monitor still isn’t being recognized. Any ideas what I could be doing to fix this setup (HDMI output to DP switch to HDMI monitor), without having to change the current KVM switch?