I figured out what was happening.
ServerA and ServerB both know a tape as ABCDEF. You then relabel the tape as XYZ on ServerA. If you inventory that tape on ServerB, it will still show up as ABCDEF. I assuming it's getting that from the SQL database based on the tape's Media ID (stored on the tape's header and never rewritten).
I then ejected the tape and deleted ABCDEF from the media pool (you have to move it to "Retired Media" first) on ServerB. Reloaded the tape and submitted the inventory job. It correctly saw it as XYZ.