@JoeKGBX and the original poster. Please take note of my experiences below.
I had some connection issues just last week (late November/early December 2015) connecting to someone on my friendslist for Borderlands 2. I had him join my game instead. Here is what I did. Not sure it was all necessary. Just part of my troubleshooting.
(back button and start button aren’t labeled on the xbox one controller but they essentially are in the same place as on the 360)
I quit and restarted Borderlands 2 (xbox button, start button, quit, reopened the game). I ESPECIALLY restart the game after my xbox has been turned off with Borderlands still open.
I also ensured that my network visibility was set a certain way. At the title menu when looking at the rotating character onscreen, I hit the back button and set it to invite only, or public, or friends, depending on the situation.
I may have also invited him and/or used party to help.
I am a little paranoid of using any xbox button features when trying to get someone connected. Why? Because I noticed how when I use the xbox one button that little red Xs appear next to my character as if I had some sort of disconnection. I don’t know if it means anything but it certainly is worrisome.
When I go back into the game the Xs go away and multiplayer does seem to work, I think. But it’s always a little odd to see that.