An "on-net" connection means that the one service provider is providing both the VoIP telephony and the internet access service.
This is better because:
- lower latency (no need to traverse third party networks)
- single vendor to deal with - no one passing the buck on reasons for call quality issues
- alleviates some privacy concerns in that internal calls remain on the single network
- helps ensure call quality assurance