The secret to writing (or playing) characters that are smarter than you are is that you can take your time coming up with what they do. Maybe in-game your character has a razor wit and would have a snappy comeback for any situation. Out of game you've got a list of pre-prepared retorts you can bust out as needed.
So I guess that's actually several questions, and they each have different answers.
Why does combat feature heavily in D&D? It doesn't. Or at least, not necessarily. How much or little it features is dependent on your DM.
Ok, so why has it historically been featured heavily? Because of D&D's lineage. The game evolved mechanically from wargames, where combat was the whole thing, and thematically from works like Conan the Barbarian and Tolkein, where fighting monsters featured prominently.
Why so many types of monsters, then, if works like The Hobbit only had a half dozen or so? Because The Hobbit is a single story, whereas D&D is a framework for creating lots of stories. Maybe one short campaign or a campaign arc has as many monsters as a Tolkein story, but then you go on to the next arc, the next campaign, and you need something new. You can obviously recycle lots; orc bandits are different from orc soldiers are different from orc cultists. But with (tens of?) thousands of games going on continuously, year after year, there's always a demand for new content to slot in, and monster design is often a handy thing for DMs to outsource. Hence, there are a lot of kinds of monster because there is demand for them.
When Lwaxana scanned the bar in DS9 s01e17 The Forsaken to figure out who stole her broach, she has to look at each person in the room to read them. Perhaps the range for Betazeds is very high or unlimited, but using it requires them to be aware of the person's presence, or to specifically focus on them? Or perhaps there's general vibes, but truly reading someone requires active focus?
The DM can not metagame, definitionally