You're right, one of the biggest issues of modded is that minecraft is single threaded, and putting too much on one thread will eventually cause lag. Not to mention that Minecraft, nor Java, as really built to handle as many mods as we have in these modpacks and from all kinds of different authors that are far from flawless.
Now, the method you are suggesting is a very popular method to handle massive vanilla servers. To not only split each "game mode" into different sub-servers, but also splitting each mode into different servers as well. Balancing out the load on the entirety of the processor. It's a great idea and once it works it works very well... BUT, it's never been done with modded. From my knowledge none of our competitors use a setup like this presumably because the infrastructure isn't really there.
With the exception of Oceanblock actually, most of our other servers run off Sponge which is a standalone API to Spigot, meaning we are very limited on the plugins we can have on those servers as the number of plugins available is much less than it would be for Spigot. Which is why we've made a bunch of custom plugins for those servers, and then run the "standard" protection, permission and utilities plugins.
But Oceanblock does run off a fork of Spigot called Archlight, so we technically could get the same plugins as a vanilla server would but the problem is that those plugins only have support for vanilla and vanilla items. So a lot of things would be very broken- taking item handling for example, syncing inventories across servers when the plugins don't recognize the items as items, and not understanding the NBT tags used... that seems like it would be a disaster. Not to mention that Archlight isn't as widely developed as spigot, so who knows how well it would handle all of that.
And even then, if by some miracle we got all of that to work flawlessly, then the biggest issue is still that every mod in the modpack sees this server as one server. Even though it might be two, three, or however many. And just as an example, FTBClaims would probably break all together. Either it would act as one world, so if you claim and chunkload one chunk on one server, it would appear to be claimed and chunkloaded on the 2nd server too. But if you weren't on the 2nd server when you claimed it, then that server didn't register that those chunks are claimed, so while they appear claimed they might not act as it. Or they might. Impossible to tell.
But that's just one example, I'm sure many mods handling the world or players in general would toss a fit if introduced to a setup like this.
In conclusion, it's a good idea and it would be even better if it was practical. But it would cause even more issues than we are currently experiencing.
We do run top end hardware to try to offer as smooth of an experience as possible, and staff do what they can to limit and help lag. And frankly, while I know we've had some hiccups these last few days I still think Oceanblock is running superb given the size of the modpack and how intense some of the late game is.
I have tweaked some of the settings in hope that it won't be restart-looping as much anymore(not the same as crash-looping)