Connection issues ONLY with friends

I’ve been recently experiencing ‘Connection Lost’ errors while trying to join a friend’s game, or when they try joining mine.
My internet connection has never been amazing, but the Co-Op did manage to at least work a few months ago, but now it just boots me out of the friend’s game right after character selection, with the message that the connection has been lost and I should check my internet connection.

I’ve tried this with two different friends and got the same result. Joining a game through matchmaking on the other hand, appears to be working perfectly.

If anyone has any recommendations or solutions for my situation, I’d be really happy to know them.
Thank you in advance! ^^