The mice will just chew through it and climb back inside. If you've got larger gaps, use copper or steel wire wool filling overlaid with wire mesh, securely fix the mesh to the wall, and then caulk ...