Most phone extensions, or handsets, are plug-and-play. Just plug them into any jack and LinuxMCE will start using them. If you want to use a regular old-fashioned style phone, you will need an adapter that plugs into the network jack (RJ45) and gives you a regular analog POTS connection (RJ11) that you can plug your phone into. These adapters are also plug-and-play.
You can change the settings for your phone extensions here, or add new phones that are not plug-and-play.