I had the exact same problem with my 500PS switch. At first it wouldn’t turn off so I needed to unplug it. Soon after I got a loud hum. The bottom line was that I needed to send it to Focal-Naim America, in Montreal. They fixed it by replacing the power switch. It was a very cheap part causing a very big problem.
Skip it sounds like the latching mechanism on the press button switch has got stuck. The gap between the switch button and the (deep) fascia is tight and it may be that dust or something has jammed it up. It has happened to one or two on here before. Can you power down by removing power and then investigate further?
Had the same issue with a NAP300PS - very slight re-alignment of the internal switch mount required (maybe 1mm) to prevent the binding and no issues since and no need to replace the switch. Dealer can probably do this for you? If comment is inappropriate Richard, please accept my apologies and remove.
Then with the end of a knife I spread the brass spring so that it was not overly compressed. You can see the small brass spring on the left. See below for the compressed spring. I did not get a picture of the uncompressed spring but you can easily imagine and you can check it before you put it back together.
The biggest problem by far was getting the 500PS out of the Fraim and unlocking the Burndies. The rest was a breeze. You only slide the power supply out of the case by a couple of inches to get to the spring. Many thanks to Dale G. at Serious HiFi for the coaching.