About 500, including many from Wyndham and Melton, and their families - ranging in age from

children to the elderly - took their frustration to the streets in a peaceful protest.

The group assembled on the corner of Collins and Williams streets on Tuesday, demanding the State Government scrap its proposed $95,000 per hectare 'tax' on landowners who sell their land within the urban growth boundary extension area.

Speakers including affected residents, lobby group Taxed Out spokesman Michael Hocking, Liberal MP David Davis and Greens MP Colleen Hartland addressed the crowd.

Disgruntled landowners agreed to a series of actions, including calling on the Government to scrap a green wedge "land grab",

convene a population summit and meet with community groups. The gathering then tried to convey its demands to Planning Minister Justin Madden, who was believed to be dining at the nearby Intercontinental at the Rialto.

The crowd marched along Collins Street to its destination, forcing police to redirect traffic, before continuing its chanting.

Residents held placards and called for Mr Madden and Premier John Brumby to "go". The planning minister did not emerge from the building.

Mr Hocking said it was unfair to make landowners pay for infrastructure that wouldn't be delivered until "2019 to 2029".

"For 150 years not one person has had to pay for infrastructure tax," he said.

"All the necessary services are in place so that tunnels, highways, parks, art galleries and hospitals are all paid for by the community, through state and federal tax."

Mr Hocking said proposed changes would make the ownership of land "mean nothing".

"A precedent will be set if this legislation is passed. The logical extension of this will be an exit tax for all landowners in Victoria," he said.

"Victoria will be the only state in Australia to tax landowners for future infrastructure that will not be constructed for decades, if ever."