I have settled on my land and I am building a new home. I am a first home buyer and unfortunately I went with a low deposit agent as this is our journey to home ownership

I signed the contracts months ago - these contracts presented to us by the 'marketing agent' aka the low deposit company was between us and the builder who offers a full turn key package.
so we are all settled and ready to go however when its come to me upgrading or variations, the builder has informed me that they need to get permission from the 'low deposit agent'. apparently they said if its okay or not.
I'm upset by this as I was not told that this would happen when I signed off or that the 'low deposit company' would have to be and stay involved throughout our build.
is this normal or even acceptable.
I have no contracts or signed documents saying this.
my contract is between me and the builder only