A server is its own community. Each one has its own flavor, its own pace, its own rhythms. Once you make a server your home, you get used to seeing the same guild names. You become familiar with the tone of /trade. You might learn some server history. You start seeing the same people around the mailbox.
Being on a familiar server is like shopping at the neighborhood grocery store. If you're there often enough, you start to recognize the regulars, like the cashiers, and they start to recognize you, too. You might not know everyone in the aisles, but you assume that they live or work nearby. You might run into them again. You may have acquaintances in common. You deal with many of the same idiosyncrasies of that particular neighborhood.
Enter cross-realm zones. Suddenly I feel as if my local grocery store is gone and I'm shopping at the airport. Who are these people? Will I ever see them again? There's no sense of community. Internet anonymity is back in full force.
I settled onto a particular server after some research as well as some trial and error. The server I escaped from was an unpleasant place to be, and I was thrilled to find a great realm like my current one. I'm very happy with the community here, and I've done my small part to keep it a good, welcoming, positive place. I don't want that sense of community to become fractured. I don't want to start thinking of the toons around me as anonymous strangers in an airport.
The bonds of friendship we form with other players are a large part of what keeps many of us tethered to WOW. That sense of community we build with each other encourages us to log in. The sense of community and accountability is something many players have been trying to cling to in this post-LFD game. Now it feels as if the anonymity of the dungeon finder is in almost every aspect of the game, from questing to doing dailies to standing at the mailbox. It's bad enough to lose a quest mob or a node to someone on my server, but a complete stranger I've never heard of and will never see again? When someone from my server does something irritating, I can look at the guild name and roll my eyes and think, "Oh, right. That figures." There's no opportunity for that moment of community with some random nobody from another server. At this point, I just get a moment of frustration, instead. People from other servers aren't community. They're competition. And they're cementing that sense of anonymity that I came to this realm to get away from.
I don't want a game full of complete strangers. I don't want to disregard other players as anonymous blobs of pixels. For me, CRZ makes WOW feel more like a single-player game, because I'm less interested in interaction with the changing parade of strangers around me. Without CRZ, I'm more interested in the toons around me, and which guilds they're in, and whose alts they might be. I'm more likely to be friendly and helpful. I'm more likely to be tolerant and patient. I'm more emotionally engaged. With CRZ, I don't know them and they don't know me, we'll probably never encounter each other again, and I care less. I'm less engaged.
With LFD and LFR, the convenience of the change in gameplay outweighs the impact on community. The sacrifice is worth it. With CRZ, it's not worth it.
P.S. On another note, I'd assumed that as you merged RP realms with RP realms, you would merge EST realms with EST realms. I expected you to match up PVP-PST realms, and PVE-CST realms, and so forth. I don't know why you're mixing up the time zones, but please reconsider. It affects the fishing tournaments, the Darkmoon Faire, and the coordination of events.