User-agent: * Disallow: /wp-admin/ Disallow: /pdfs/ # Block the /pdfs/directory. Disallow: *.pdf$ # Block pdf files from all bots. Albeit non-standard, it works for major search engines. Disallow: /members/ Disallow: /pdf-download/ Disallow: /home-2 Disallow: /ncav-media-center/value-investing-basics/