For me, it would be these:
Surveyors (any variety) - stupidly annoying, hard to hit unless lining for attack or reparing a bot, they tend to fly all around the place and their ion blasts make short work of pretty much any shield.
Buzzards - flying enemies are generally annoying as hell in FPS games, these guys are no exception. Again, quite hard to hit sometimes, the need to track them all over the sky (and possibly overlooking that badass who kills you) and generally being an annoyance. The game would be far better without flying enemies… only Rakks seem to be a fair game, they are not so annoying as the rest of flyers and in groups they can still do a number on a Vault Hunter, although they are very weak individually.
Rabids - oh my… these guys are insanely fast, have stupid appetite and never stop beating you. Especially stalkers, skags are not that big deal. Huge health pool, too. Stupidly designed foes tanking a LOT of damage and being capable of downing a VH with just a few chomps.
Constructors (any variety) - yes, they can be killed pretty easily and have a huge crit spot. Provided you survive all those heatseekers these things hurl at you… another stupidly designed enemy. Some of these literally spam heatseekers while doing nothing else. What else… well, these guys are immune to slag and have massive HP pools which turns these goddamned things into huge bullet sponges… it’s simply a DPS race to see whether the VH can kill a Constructor faster than it can spam heatseekers. Sometimes Constructors build robots (that is no problem, can be dealt with just fine) or use their laser (also no big deal if there’s cover nearby). But dat heatseeker spam…
Skeleton Mages - ugh, stupid design strikes again. Mages give so little window to shoot them and although that “ghost” allows a VH to track them, it disappears shortly before the mage stops to uncloak and shoot (so the player still does not know where the Mage really is until it uncloaks and shoots - that’s too late though). Maya has it easiest, but the window to PL them is still damn short. Luckily, the Mages are physically weak, they fall apart after one or two good hits from a solid weapon (preferably a shotgun).
Eternal/Immortal Skeletaurs - the only enemies in the entire game a VH can’t get a Second Wind off. GG, Gearbox. You find few of these things alone (with no other enemy types around), get downed and are forced to die even if you down one of these skeletaurs. It should be possible to get rid of that cursed sword in FFYL to get an SW.
Jackenstein (final boss of the Hammerlock DLC) - oh man. A prime example of a bad boss design in BL2. If you make a boss that can be hurt only in certain places and nowhere else, you are doing it wrong. It really does not need any further comment. Critical spots on bosses and enemies are fine because you can hurt the foe everywhere on his/her/its body/mass, it’s just that hitting crit spots deals more damage. But making Jackenstein who is completely immune to damage except being hit in specific locations… no, that’s really stupid design. Sidenote: Crystalisks are similar in design, but far more bearable because their vulnerable spots are huge and can be dealt with easily. I never had too much trouble with Crystalisks, but Jackenstein caused me to rage more than once… Oh, and this guy also deflects EVERYTHING that hits him anywhere but weak spots… including rockets. GG.
Anything that deflect bullets - BUL Loaders, PWR Loaders, Crystal Skeletons. Ah, bullet deflection really is a bad idea to have on an enemy. Especially Crystal Skeletons, which deflect ALL bullets that hit them (the Loaders have only arms that deflect bullets and so they can be dealt with - BULs are easier that PWRs). Only two cases of this make some sense and are somewhat acceptable - Handsome Jack (his personal surveyor makes a reflect shield for him, but is easy to track and shoot down) and Hyperius (shield-based tactic).