One of two possible reasons hit my head…
The first could be because of Brady being a very attractive guy and Asher being in the adult world, Asher may have had a lot of fellow models in his following and those men are attractive themselves. If Brady and him had some sort of conflict about that, that could explain the massive unfollowing. Sometimes with these pretty boys, egos come inflated.
The second thing could be Asher being in the adult industry, might be trying to go low key or out of the industry and might have some people who he may need to disconnect from.