It's a good question.
I think that you can probably do crossfit as the basis of a bodybuilding program. I'd suggest that you give it an honest try - do the WODs, perhaps using a "ME" version (doing a WOD, then a maximum effort lifting day for one compound lift, then a WOD, then rest) and see what happens. It should be pretty clear after a month or two of effort if you're gaining, losing, or staying the same. Then you can evaluate if it's the right method for you. If it is, just keep going. If not, drop it. If some parts are useful, take what's working for you. You can always add bodybuilding-style workouts to your routine as you approach a contest or otherwise as needed. If you're already doing lots of compound exercises, you should be able to slide right into crossfit.
I'm currently trying crossfit on a slightly-modified schedule along with a dedicated full-body workout day once a week for my MMA training. I figure it should be clear within 2 months if this combination is superior to what I've been doing. If not, I can always go back to what I was doing before. If it is, well, it was worth trying.
For a summary of the Black Box ME approach I mentioned above check this thread:
Whether you do crossfit or not, good luck with your bodybuilding and reaching your goals. With so many couch potatoes around we exercising folks need to stick together. :D