Yes, this is a stopgap measure. Eventually, you need to change all those http:// URLs to https://. However, with one HTTP header, it insures that all resources will be requested over SSL first. Consequently, it will make your site more secure immediately and provide a good start to implementing Content Security Policy and using its other cool features.
For more details on the upgrade-insecure requests, check the W3C Working Draft document.
As for browser support, the upgrade-insecure-requests feature is supported by Chrome v43 and Opera v30 (at the time of writing). You can also check the latest browser support information at CanIUse. As with other features of Content Security Policy, the browser adoption will probably be pretty quick.