Fence dispute - zero allotment boundry wall

Hi everyone. I'm dealing with a fence dispute with my neighbor. I built a house in a western suburb of Melbourne and moved in April. I sent a fence notice to my neighbor back in February, but since I didn't get a response, I went ahead and built the fence. Now, the neighbor is starting construction and needs to remove part of my fence for their garage, which is on a zero allotment boundary wall. I've given the builder permission to remove the fence, but the neighbor is resisting, saying the builder isn't handling it, so I have to remove the fence myself. If anyone has encountered a similar issue, please share how you resolved it. Thanks in advance