Issues setting up co-op with friends

Can’t seem to reliably set up a co-op session with friends. Takes multiple tries including full game restarts. Very hit and miss. Really frustrating to have friends in voice party but not able to join the game.

Anyone else experiencing this issue?