The US isn't the only one that taxes its citizens no matter where they are in the world. Americans can get out of taxes provided that they fulfill some requirements and you can read more about this in the Tax Guide for Overseas Americans. If you need help with your taxes, there are plenty of tax guides out there or alternatively look into hiring an accountant who specialises in expat taxes.
Peruvians and foreign residents also have to declare taxes on money they make in other countries. Whether or not you can avoid paying taxes to Peru depends on your situation. See SUNAT for more info. In addition, having paid taxes to your home country does not exempt these funds from being taxed again. This is the law, what is done in practice is different. I don't know anyone who declares their worldwide income to Peru so that they have double taxation.
If you're going to hit the tax office in Peru, one tip to remember is that speaking Spanish will help you greatly. You'll be treated differently than if you speak English all the time, it'll help you assimilate to Peru, and it'll get you information faster. If you're looking to learn Spanish, check out Fluenz Spanish, Rosetta Stone, and Synergy Spanish.
The Ultimate Peru List recommends: