User-agent: * Disallow: /CustomErrors/ Disallow: /WebPortal/ Disallow: /*.xml$ Disallow: /*? Disallow: /id/