Should I be engaging my pelvic floor muscles the entire 15 minutes?

Having the weight inserted will force your pelvic floor muscles to activate simply through normal bodily movement. Also, gravity and your normal movement will occasionally make the weight begin to slip out. Once you feel this slip, perform a Kegel (engage your pelvic floor muscles) to keep the weight inside. This will also increase your strength. The more your pelvic muscles engage, the stronger you will get.