You can probably ask them to pull the wires there but not install or terminate them for a patch panel.
Because you specified a patch panel, they probably quoted for the installation of the rack and the patch panel, as it’s not there and therefore they need it to complete the task completely.
You’ll end up with loose unterminated wires you can then just put an RJ45 plug on and wire directly to a switch or whatever.
I’d just manage the actual patching with VLANs on the switch. Unless you plan a more complex setup with some jacks going directly to a server or other routers/switches, it should be plentiful to just have 24 live ports you can plug devices into. Fair amount of switches can be simply wall mounted without a rack.
I’m actually not looking to do anything very complicated at all. Just connect the internet box directly to the router (via the patch panel). Router itself connected to a switch, and then all my other devices connected to the switch (with every connection to the switch going via the patch panel). I don’t have any plans to do anything with different networks, just the one local network.
The reason for having the patch panel is so I have a simple plug-and-play way of connecting everything up. I didn’t think it would be any different to having a standard walk jack, but with 24 ports instead of 1–4.