A good pair of brown trousers has a bit of weight to the fabric even before you unfold them off the rail, enough drape that they don't cling to your hand like cheap polyester does. Hold the waistband and give it a gentle pull. If it springs back without going slack, that's a decent sign. If the colour looks like it's been dipped in tea rather than chosen on purpose, put it back and keep looking.
If you've never owned a pair, start somewhere between chocolate and camel, not the very dark end and not anything leaning orange or ginger, which is a different colour wearing a disguise. A straight or slightly tapered leg in a mid weight fabric will do more for you than a fashion cut, because the shade itself is doing plenty of work already. I made the mistake once of buying a pair in a deep rust thinking it would read as brown under any light, and under office lighting it looked like I'd wandered out of an autumn catalogue shoot. Mid brown, nothing shouting, is the one to trust first.
Once you've got that pair, you'll notice how much easier it makes the rest of getting dressed. Cream, white, soft pink and any washed out blue all sit well against brown in a way that black never quite manages, it's softer on the eye and doesn't need the rest of the outfit to be quite so tidy. A cream knit with brown trousers looks like you planned it even on a day you didn't. Tan or burgundy accessories pick it up without matching it exactly, which is the bit that makes an outfit look put together rather than coordinated to death.
Brown trousers earn themselves a season rather than a single occasion. They start making sense properly from September, carry you through winter better than black does because they don't look quite so funereal under grey skies, and keep going into early spring before anything lighter takes over. I've worn mine to work, to a long lunch, and once to a wet Tuesday dog walk that left the hem muddy and somehow improved them. They're not really a summer trouser, there's always one shade that insists on trying, and it never quite convinces.
Next time you're looking, check the trouser against your own skin tone in daylight near a window, not under shop lighting, because that's where a brown either looks rich or looks like cardboard.