I recently bought a tiny board and LCD:
https://www.sparkfun.com/products/12857https://www.sparkfun.com/products/12086What's nice about these is that they don't have many parts that emit electromagnetic radiation, and the ones that do run at frequencies that are easy to shield. Right now I'm running it in a makeshift enclosure made out of a cardboard box coated with radio-absorbent paint, with the AC adapter on the other end of a 25' extension cable, and it's the first time that I've enjoyed using a computer. The drawbacks are that the CPU is really slow, the screen's color depth is terrible, and it only runs Android or Linux, but I still like it better than my Thinkpad.
The next step will be to build a better case. I can easily share plans when I'm done if anyone's interested.