we went through a similar ride when we applied for our loan. At first the process was go through our builders broker (MC) who started applying for loans everywhere asking us to keep signing papers for him to apply for the loan, giving him all these papers about income etc. until we finally had enough. The only benefit the broker was giving us was that we could pay $1500 of the initial deposit instead of the 5%. Which made no difference as we'd have to pay the 5% eventually anyway.
My brother got me in touch with a lending manager at CBA who was fantastic. She met us after 6 and within the hour we had pre-approval with everything we needed clearly lined out and how everything would work. We told our broker that we went directly to the bank and he said he could do the same thing and we gave him the flick so we could deal directly with the bank.
All you have to do, if you haven't signed any contracts with your broker is, send them a friendly e-mail releasing them of their services.