Cultural influences on our behavior are partly reflected in neuro-anatomical changes in our brains, altering our abilities to perceive and interpret information. This paper points out possible consequences of such ``cultural abilities'' for user interface design, and outlines challenges for systems that adapt to their users. Specifically, we describe how culture could influence our design decisions of when, how, what, why, and where to adapt.