The simple answer is to put something in front of the green Naim logo to block the IR signal from the remote on the device you don’t want to use.
If this is a long term setup you can change the remote channel on the device you do want to use so that only that one is paired with your handset.