Joining my brother completely lags my game after the patch

So as soon as the patch hit if I join my brother, who’s in the room with me on a different xbox, the game is absolutely unplayable for the person not hosting. Are they gonna fix this or am I stuck having to play alone?