No. I haven't found having two sets of speakers compromises the sound.
It may be that my system/speakers are not revieling enough (or my hearing) to notice. My room is pretty dead so that may be a factor.
I like being able to switch between the two very different sound characteristics of the speakers. Sometimes I'm want to rock out and the towers can go loud. Mostly 70db listening, jazz and ambient is magical on the tangband/SVS dual subs.
I play only one set of speakers at a time. I do not notice any problems or interference with placement beside each other, but this is not a highly resolving system (not very expensive in comparison to most systems here).
The single drivers (sealed speakers, no ports) are wonderful at the midrange and treble. Terrible bass, as in non-existent bass. With the dual sealed subs filling in the bass its AMAZING. These are an experiment for me into full-range single drivers. Basically sourced out some existing speaker boxes online (craigslist) that matched the internal volume recommendations from the driver manufacturer, reinforced the interiors, glued and screwed an extra layer of 5/8" ply inside (now walls are 1 1/8" thick, solid rap test) and an extra 3/4" ply for front baffle (so now front baffle is 1 1/2" thick), finish outside to my choice and done. VERY VERY happy with them. Now my main speakers.
Such an interesting speaker combo! I assume that you only play one at a time - I'm considering having another pair of speakers (a set of omnis), but I taking my large speakers out each time and moving in the omnis seems exhausting. Do you feel that having two pairs side by side compromises anything?