Embedded Linux is something not running on a big desktop PC. Mostly such devices have limited memory and not enough power to do compilation of its code. Additionally such devices might not be individual installations they need to be duplicated and might be used mass production.
Nowadays memory is not such a big issue, since most devices accept memory cards and have therefore >2GByte of space. This is enough for a full complete and no more no more tweaked Linux system to be able to get installed to reduced memory .
If time is not an issue standard Linux distributions as Gentoo can be installed on the device and keeping maintained compiling its new source code.
However embedded Linux devices are created using a powerful PC where the embedded Linux devices can be attached. The powerful PC devices does the configuration and compilation and downloads the result to the embedded Linux devices.
It is common that the powerful Linux PC using an other CPU architecture as a x86 and the small embedded device something as an ARM processor. Therefore cross compilation is necessary.
There are different solutions available to do this:
From scratch http://www.linurs.org/limux.html the hard way but good to learn
Linux Target Image Builder http://ltib.org/
Open embedded http://www.openembedded.org/wiki/Main_Page
Simply use Gentoo and consider to use crossdev and distcc. Since maintenance, updating and expanding is an issue Gentoo is a good way to use for that.