If we remove the module which will call mpc52xx_spi_remove
it will free 'ms' through spi_unregister_controller.
while the work ms->work will be used. The sequence of operations
that may lead to a UAF bug.
Fix it by ensuring that the work is canceled before proceeding with
the cleanup in mpc52xx_spi_remove.