Well, there's the Intel Compute Stick. I have two STK1A32SC as micro-servers for my GOG surveillance state scripts (they are not plugged via HDMI into any monitor, but running Linux, with a VNC server, and connected on WiFi to my LAN). m3 and m5 variants are far more potent but equally more expensive.

The online specs say it comes with a x5-Z8300 processor, however both my sticks have a x5-Z8350 in fact and boost up to 1.92 GHz when needed.

If you want something a bit cheaper, there's also the ASUS VivoStick PC with equivalent hardware, however there's a caveat if you plan to run Linux on it.

The VivoStick fan is driver controlled, meaning if you remove the default Windows installation, then the fan will always be on, never powering down. Intel Compute Sticks have a hardware controlled fan which throttles/stops regardless of which OS you use.

You can run very low-end indie games on them, but I wouldn't recommend that as a main use. I've played stuff like Braveland, Human Resource Machine and even Kingdom Rush (at 720p) on it and they are ok in terms of performance. Older games which have open source linux engines like Quake 2 and even 3 will also work quite well at 1080p.

I own a tablet with that cpu and I bet that it wont overheat in either of the options. The Pc stick will run hotter but probably not too hot, as it consumes very little power, my whole tablet (12") uses around 10W at the wall while gaming.
I am running windows 10 (wich is freaking horrendous) because I lack the knowledge to fix the wifi driver on Linux.
Keep in mind most mini pc are a little faster, compared to the Pc stick because they support faster ram and storage, but they are much more expensive, as most you have to buy ram and storage as well, they are called "barebones".

There are other options:
Atomic Pi SBC, I first heard of it on youtube channel ETA PRIME and I want to buy one, infortunatelly only available on USA. Very cheap and might be what you need.
https://dlidirect.com/products/atomic-pi

If you know how to build computers (hardware), check the pc parts shop for motherboards with soldered CPU. J1900 is a good option, they are usually very cheap and upgradeable. Some of them support a dedicated GPU if needed. Just ensure the CPU is a quad core, as some dual core are very weak, like the n3350 and n3060.
They pop-up on add websites from time to time very cheap as well.
https://www.asrock.com/mb/Intel/J4105B-ITX/index.us.asp

New quad core pentium n5000 cpu's are much faster than the atoms x5 or the older celeron n3350. In my country there are mini pc with n5000 as cheap as with older CPU's
https://www.msi.com/Desktop/Cubi-N-8GL

There are some newer laptops with Linux or Freedos, usually for cheap, since they come with a screen, keyboard and battery, it may be a better option. In my country I´ve found some with the Atom cpu for 120 euros (local supermarket, no-name brand), you may have to search a little to find good deals.

Don't forget tablets, they can be used as a normal pc with keyboard, mouse and big desktop monitor hooked. If you never used "the internet" with a touch screen, it´s amazing and works flawlessly with Chromium and Linux Mint.
My tablet can easily run n64 games and some PSP as well, not sure about Bnes, I may try it if you want.
Many indie titles run just fine, I completed guacamelee at 60fps with no problem.

Those are very underpowered systems, as you know, but some of them can be a bit tweaked for increased performance. Sometimes they are limited in power consumption in a locked bios, do some research in the model you want. 6W is the maximum for the X5 z8350 atom, but on some devices it´s loked to much less.
Keep in mind, the 6W limit is quite low, the GPU cant be used 100% or the CPU will throttle to 0.8 GHz, so only lower resolutions are possible, in more "demanding" titles.
Dual channel RAM memory helps a lot on those little machines, instead of single channel. If the Pc has or can be upgraded to dual channel, is a big factor in performance. (the z8350 dont support dual channel but celerons and pentium do).

My tablet has 4gigs of Ram, but probably 2gigs is not a problem, unless browsing and gaming at the same time.
Pretty sure Hollow Knight won´t run on this thing properly, have to check it but 60fps locked may be a no go.
If you want to increase disk space, a memory card is a must, I run some steam games on memory card and it´s not a problem. A faster A1 spec card may be better.

If you don´t mind buying used, a whole big post can be made about old computers, like core2duo stuff. they wiil use a lot of power though.

Funny thing is, I atually *have* one of these, but I don't have any way to power it, so it's useless.

Isn't powered at 5v trough GPIO pins?