Extreme Programming (XP) is such a methodology so that either you do all of XP or none at all. There is no in-between thing. But all of the utopian theory of XP may not be true in all cases. I interacted with many Project Managers due to the nature of my profession and I found some of the major difficulties for the implementation of XP project.
1.Extreme Programming is hard to do. You may have difficulty to get many developers accept this practice, and it takes a lot of discipline to keep doing them all. Your customers may not like the idea of so much involvement. Management may also find problems to get used with the practice that changes dynamically during the project life cycle.
2.XP team does not believe the idea of fixed price, fixed scope kind of terms with the customer. Other processes are more likely to encourage the creation of detailed planning from the beginning.
3.XP encourages pair programming. There may be a huge amount of duplication of codes that clubs with unit test. As a result, it is going to take long time to run and leave the database with lot of duplicate data.
4.XP is code centric rather than design centric development. The lack of XP design concept may not be serious for small programs. But, it can be problematic when programs are larger than a few thousand lines of code or when many people are associated with the project.
5.XP does not measure or plan Quality aspect of development. Managers for the big project believe that quality planning helps properly trained teams produce high-quality products.
6.Another basic problem is that software today are very large and complex by nature. My practical experience is that it is hard to design by using incremental method that XP practises.
7.XP emphasizes on refactoring during software development process. But, as far as my experience goes, too much refactoring may not be that important all the time and can be a waste of time rather than being productive in positive sense.
Since, XP programming is not structured, it may be difficult to find significant number of defects for testers by just mere looking at the screen. On the contrary, traditional programming enables testers to find improved number of defects because it is well documented. Thats what I witnessed in a couple of mid sized projects at our company.